New Year

Starting Jan 1st, I will be moving to 3/4 time at UND. As such I resigned my NDSU position effective Dec 31st. I will continue work for the archaeology lab at NDSU as a grant writer through the spring sememster, but in an unbenefitted position.

The week of Dec 17th I will be on campus M, T, W, F. I will be on vacation until after the new year. Starting the week of Jan 1. I will be on the UND campus M,T, Th, F every week. Hopefully funding will continue to be available, through the sememster and into the summer.

Recently was I was given admin access to Shale, and as such I have been installing new software. Shale now has two 3D renderers on it. The first, POVRAY is an open source renderer used by a number of 3D modeling programs. The second is Yafray, which is used by Blender – a very nice and feature rich 3D modeling and animation application. The nice thing about Blender is that it is completely free as it is open source.
www.blender.org

I also installed Mr. Bayes phylogentic software for the biology department, and have successfully completed its test run. Because Mr. Bayes and Paup are now ready for use on Shale, Dr. Simmons of the biology dept will be offering a course in Computational Biology next fall.

I have been looking into the HDF issues concerning the supernova research going on in physics. It turns out that the NCSA has released Java libraries for accessing data stored in the HDF4 and HDF5 formats. As such, I decided to put the IDL code aside, and rewrite the applciation in java. Nice thing about that is, is that we don’t have to purchase the IDL developers kit for Shale. At least for now.

Lastly, I have been assisting with organizing Interent2 presentations for the state legislature IT subcomittee. It looks like we’ll have some good presentations. From Ron Marsh and Tim Young’s Eclipse internet broadcast, to the med school’s rural networking inititatives. And it looks like Keith Henry will be able to present his I2/HPC/Visualization-based pharmacology work as well.

System utilization figures for the last 30 days show that Shale has been at 50% utilization around 50% of the time, and is regularly over 60% utilized, and has peeked on several occasions at around 80%. There were two occasions where jobs needed to wait in the queue as each required more resources than what was available at the time.

I am still working on getting the Fluent computational fluid dynamics software running on Shale. Everything is in place now except the license server. Because the compute nodes cannot talk to outside servers, the Fluent license server must be setup on Shale’s master node. As such, I am waiting for the licenses to be reissued. Once this has taken place, Fluent should be available for use within a day.

I am looking forward to the new year. I have some interested in submitting an NSF grant for a shared memory system, and I hope to be digging more deeply into some Chemistry-based HPC projects. I have the I2 presentations coming up January 18th, as well as the installation of the new Viz3D system and the birth of my new baby girl to which to look forward. She’s due Jan 18 as well!

Happy New Year!

Aaron

Computational Polyglot

I am amazed at the vast numbers of computer languages that are used within the realm of HPC and scientific computing. One pretty much has to be a computational polyglot to make any headway. There are so many languages. Fortran77, Fortan90, Fortran95… thank god I haven’t run into any Fortran 2003 yet. But I’m sure its only a matter of time. But let’s not forget C and C++ or the scripting languages like Perl, Python, or PHP. I’ve run into all of these languages within since I started here last December. I’ve even dealt with Java on a couple of occasions. It’s enough to make one go mad.

Yesterday I was introduced to the Interactive Data Language (IDL) or as Wikipedia says “IDL is often confused, at the acronym level, with another (unrelated) “IDL”: the Interface description language.”. What there’s more than one IDL???? No wonder why the help page I was reading didn’t make any sense.

Anyway, I sometimes wonder why people go through the trouble of inventing a new computer language. It occasionally makes me wish that everything was written in Assembly.

So the good news is that I was able to get the ZeusMP/2 code running on Shale this week. ZeusMP/2 is a software package developed by Laboratory for Computational Astrophysics at UCSD. Among its many uses is “hydrodynamic blast wave” analysis for supernovas. Getting it to run also meant installing the Fast Fourier Transform, Hierarchical Data Format (HDF), JPEG, and ZIP imaging libraries on Shale. All put together, it was a mass of Fortran 77, Fortran 90, C, and MPICH (and I swear, maybe even a bit of C++) code. Let’s just say that the configure and make files for all of these applications and libraries are a blast of fun to deal with. Thanks Robert for your help.

But there was still one more twist. The HDF files that are produced by Zeus still are only data files, they still needed to be converted to images…. and of course, the program that’s needed to convert from HDF data sets to HDF images is written in a completely different language that requires a completely different compiler… one we don’t have, IDL.

Ok, so I’m venting a bit. How many languages can there really be? You almost have to have an anthropology education to keep them all straight… oh wait… hmmm… But when I think about it, I’ve actually learned a bit this week. How some of the code required the gcc compiler, the pgf compilers, the pgcc compiler, and the g77 compiler… so many compilers. It can be frustrating, but it is really a great feeling when you finally find the answer to why the code does not compile.

Did you know that the following line of C code is accessible from Fortran 77, but not Fortran 90?
void poisson_solver_(int *myid_w, int *nprocs_w, int *nx, int *ny, int *nz, double data[*nz][*ny][*nx])

Anyway, its all good. We’ll be getting a IDL compiler soon, so we’ll be blowing up stars any day now.

Projects in the Pipeline

Progress on feeding jobs into the Shale cluster has gone slower that I expected or would have liked. Training and traveling have interfered with returning to troubleshooting. Additionally, the Supercomputing in Plain English video workshop eats up half of my Wed, leaving me with just 1 1/2 days some weeks to work on pending issues. Between the supercomputing symposium, the video workshop, and working half time at NDSU, I’ve only been able to put int 3 days of HPC consulting in the past 3 weeks and even then, I was working on a grant proposal too.

However, that should change in the next week. Dr. Simmons Paup jobs are finally working as expected. I will be writing a script that will let her post process the data much more efficiently. 10Gb of text files is somewhat burdensome. Just today, I was able to get Zeus-MP software compiled on Shale including the extra libraries the software required that Shale didn’t have. This software is used to calculate gas hydrodynamics and will be used by Dr. Young (physics) for his research into supernova.

Hopefully, next week I will be able to get MrBayes, another piece of Phylogeny analysis software compiled for Shale. Biology uses comparisons between data produced by MrBayes and Paup to determine if the data from each is likely to be correct. MrBayes is also an MPI application. I’ve had it installed on Shale for awhile, but in the past it has produced a segmentation fault when run on the cluster. After talking with Dr. Hoffman a bit, he and I figured that it was probably compiled inappropriately. I did use the -O3 compiler optimization in the MrBayes make file, so I’m guessing that the optimization is the issue. As I am here 3 days next week (the first time in three weeks) I should have time troubleshoot the compiling issues.

Software licenses have continued to be an issue where the computational fluid dynamic software seats are concerned. With the new software license review process, purchasing software has become… a long process for items that don’t readily appear on the shelf at your Best Buy. Mechanical Engineering had ordered the parallel CFD seats in late July. The license review occurred right away, but the company didn’t respond as quickly as I would have liked.

In order to shake things loose, I called the company representative hoping that he might be able to facilitate the company’s response. Whether I was responsible for getting the ball rolling I am not sure, but shortly there after the UND General Counsel received a response. After a couple of follow-up emails between myself and everyone involved, the new license agreement was approved by both parties around the end of September. Of course, it’s almost the middle of October, and the CFD software seats still haven’t been delivered, but at least purchasing was able to order the software.

In the mean time, graduate students from Mechanical Engineering have been running the CFD software in a serial environment on the CRC’s new 64bit Windows-based visualization system. As such, the students should be ready to make the transition to the cluster as soon as the parallel seats arrive.

There is more information to share, particularly the Gaussian install and my trip to the 2007 OU Supercomputing Symposium, but this blog entry is getting long, so I will post more on these subjects next time.