Monday, December 3, 2012

More on DMP vs SMP

To paraphrase one of my professors at  UC Davis, once you understand a problem well enough to formulate an elegant question about what it is you don't understand, you're halfway to figuring it out.  It was with this in mind that I was contacted by one of my readers with some questions about some of the differences between SMP and DMP.  Paraphrasing and obfuscating a bit, our conversation went something like this:

Reader

Could you explain a bit more about the difference between DMP and SMP?  I understand your card sorting analogy, but what does that mean for hardware?  Multi-cores?  Dual CPUS?  Both?

AppliedFEA

The answer depends a bit on your solver and solution type. The executive summary is that SMP makes most things much faster, but DMP makes some things much much faster, so long as you have enough memory to spare. What solver (Abaqus/Nastran/optistruct) And solution type (statics/frequency/transient etc) are you targeting? Depending on the answer to the previous questions DMP may not be available at all, such as if you're running Nastran SOL 106.

Reader

We run a lot of static stress analysis and modes.  We use implicit solvers such as ABAQUS/Standard, Optistruct, or MSC/MARC.  We run some ABAQUS/Explicit.

AppliedFEA


To break outside the card analogy, an FEA solution requires the solution of a sparse stiffness matrix. Depending on the solution type, there are several levels at which the solution can be broken up and solved on more than one processor at a time, given that the system solving the problem has more than one CPU core. At the lowest level, SMP/parallel, only the vector operations are parallelized. Essentially every solution type in Nastran and Optistruct can make use of this low level parallelization.

You're in luck, however, as some of your solution types have the ability to be parallelized at a higher level.
For statics:
In Nastran, the domainsolver card allows a statics (sol 101) job to split with the stat method, which can assign each processor a certain number of grids or degrees of freedom. Abaqus does it's DMP split similarly, albeit solely at the element level.
For modes (eigensolver):
In Nastran SOL 103 you can use the MODES dmp method, which can split the job up at the grids, DOF, or frequency level (ie each process finds modes in a portion of the total frequency range). Abaqus can, again, split things up over the Element level.

If you're finding modes of models with large numbers of degrees of freedom, consider using an an automated multi level substructuring eigen solver instead of the more typical lanczos. Both Optistruct and MSC Nastran can use the university of Texas developed AMLS eigen solver, which can be purchased separately. All three solvers have also developed their own similar solvers, in Optistruct they call it AMSES, in Nastran they call it ACMS, and in Abaqus AMS. Speed increases of 10x or more are not unknown.

So to tie back to my typical advice, just make sure you are off the hard drive, DMP (split the level up at a high level, with each split requiring more memory) until you run out of memory or CPUs, and if there are any CPUs left after you run out of memory, use parallel/SMP to use those to split up the split at a low level.

No comments:

Post a Comment