The power of good tools

Ok, sort of another minor DragonFly milestone … except that no code in DragonFly has changed. Previously, we pulled our job description from a file, and used it to create the job for us. Now, without changing a single line in the code, we have done this:

dragonfly@dragonfly:~/utilities$ ./build_job.pl --job=http://dragonfly:3001/jobs/xml/14 --program=ptb.xml --debug D[27064]: os = 'linux' D[27064]: directory = '/home/dragonfly/utilities' D[27064]: opening temp file in directory .... D[27064]: parsing XML from job='http://dragonfly:3001/jobs/xml/14' D[27064]: parsing XML from program='ptb.xml' D[27064]: project='p1388' D[27064]: keeping default environment D[27064]: delete environmental variable 'PGI_BITS' [current value = ''] D[27064]: add environmental variable 'alpha' [current value = ''] D[27064]: add environmental variable 'beta' [current value = ''] D[27064]: add environmental variable 'gamma' [current value = ''] D[27064]: append environmental variable 'PATH' [current value = '/home/dragonfly/bin:/opt/gridengine/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'] with value = '/usr/strange/bin' D[27064]: insert environmental variable 'PATH' [current value = '/home/dragonfly/bin:/opt/gridengine/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/strange/bin'] with value = '/opt/bin' D[27064]: param substitution: Parameter '_NCPU_' = '165' D[27064]: Using MPI: stack = 'openmpi124' D[27064]: - MPI mpibin = '/opt/openmpi124/bin' D[27064]: - MPI mpirun = 'mpirun' D[27064]: - MPI mpiargs= '-np _NCPU_' D[27064]: - MPI runcmd = '/opt/openmpi124/bin/mpirun -np 165 ' D[27064]: executable = '/home/dragonfly/bin/ptb.exe' The web server that served the job description did it from the database where the job description was saved. Ok, a minor thing you might say. Perhaps. Or perhaps not. Regardless, it is interesting that without changing a line of code in the application that generates the jobs, we were able to change the source of the jobs. Which if you think about it, changes some other things as well … makes them … I dunno … easy (as compared to what they are now). This change was effected with a 5 line template,, a 10 line method in the server. The client didn’t change at all. Think about this from a parallel and high performance programming perspective. Wouldn’t it be nice to have this sort of power, to be able to retarget calculations between various computational engines without changing code, and still get great performance? I have a suspicion that this is not a vision that will come easily over time. A milestone, nonetheless.