I occassionally listen to the 99% Invisible podcast.is It covers a number of interesting topics, mostly focused on civil engineering, but one of the earliest ones focused on something different: the sounds that create the feel of computer programs.
Designing the interface of a computer program so that it creates an intuitive, physical connection with users is very difficult. Unlike old stereo equipment, a computer program cannot give real tactile feedback when a dial has been turned all the way or when a switch has been turned on or off. A display can do a very slick job of trying to appear real, or skeuomorphic, but it is still a picture under glass. Most cell phones and video games can give some shaking to create crude feedback, which can tell you that something happened, but it can't tell you much about what that something specifically was. The only remaining way for a computer to give intuitive feedback is the sounds that a program makes. Think about the Windows Critical Stop sound or the early mac equivalent uh-oh sound; if you have a strong emotional reaction to something your computer does its probably to these sorts of sounds. So it's not surprising that smart interface designers will spend a significant amount of time honing the sound effects in their programs.
Something I thought was especially interesting, when listening to the the podcast, was that the sort of sounds that users gravitated to most were the ones that were the recorded sounds of something mechanical happening, such as a vise grip being released, rather than something synthesized. Even when using a device that would only make a big mechanical sound if it was being destroyed, people still enjoyed the unsynthesized sounds of real things happening.
Of course, things do have a way of coming full circle.
Friday, October 12, 2012
Wednesday, October 10, 2012
The Need for Speed - Part 4 of 7: Performance tuning parameters: Nastran
Previously I've covered how to find the optimal memory and cpu settings for Abaqus and Optistruct. While the procedure for those FEA solvers may have seemed complicated, MSC Nastran (and for the most part its spinoff Nx Nastran) take this complexity to a whole new level. The fundamental ideas are the same:
Limits of Nastran estimate Procedures
If you follow the official recommendations, there are in fact two ways to estimate the memory needed to not use the hard drive. The first method is to use the 'estimate' program. It will generate output as seen below:
Reading input file "./.bdf"
...
Submit with command line arguments:
memory=532.9mb
Estimated Resource Requirements on a 32-bit system:
Memory: 532.9 MB
Disk: 9125.4 MB
DBALL: 6099.1 MB
SCRATCH: 3049.6 MB
SCR300: 845.3 MB
SMEM: 25.0 MB
Well this looks very helpful, except it's very very wrong. This is the output for a model I know used memory=33GB of memory to run at full speed. Estimate didn't even try to recommend an aggressive smemory setting to reduce the amount of data written to the Scratch and Scr300 files.
The other recommended way to find the optimal memory settings is to run the model until the .f04 file displays UIM (User Information Message) 4157.
*** USER INFORMATION MESSAGE 4157 (DFMSYM)
PARAMETERS FOR PARALLEL SPARSE DECOMPOSITION OF DATA BLOCK SCRATCH ( TYPE=CSP ) FOLLOW
MATRIX SIZE = 1163947 ROWS NUMBER OF NONZEROES = 29408809 TERMS
NUMBER OF ZERO COLUMNS = 0 NUMBER OF ZERO DIAGONAL TERMS = 0
SYSTEM (107) = 32770 REQUESTED PROC. = 2 CPUS
ELIMINATION TREE DEPTH = 8372
CPU TIME ESTIMATE = 915 SEC I/O TIME ESTIMATE = 3 SEC
MINIMUM MEMORY REQUIREMENT = 36169 K WORDS MEMORY AVAILABLE = 2807326 K WORDS
MEMORY REQR'D TO AVOID SPILL = 36432 K WORDS MEMORY USED BY BEND = 36170 K WORDS
EST. INTEGER WORDS IN FACTOR = 143016 K WORDS EST. NONZERO TERMS = 299087 K TERMS
ESTIMATED MAXIMUM FRONT SIZE = 2429 TERMS RANK OF UPDATE = 64
If we examine the .log file for this .f04, we see that mode=i8 was chosen, meaning each word is 8 bytes. Multiplying the Memory Reqr'd to avoid spill, aka the estimate for in-core memory, we find that UIM 4157 estimates that 36.432 megawords x 8 bytes/word=291 Megabytes of memory will be required of the 2.807 Gigawords x 8 bytes/word=22.46 Gigabytes made available to hicore by the difference between the mem and smem memory settings. So what did this model actually use? If we read the Total Memory and Disk Usage Statistics section in the .f04 file, we see the following useful output data:
*** TOTAL MEMORY AND DISK USAGE STATISTICS ***
+---------- SPARSE SOLUTION MODULES -----------+
HIWATER SUB_DMAP DMAP
(WORDS) DAY_TIME NAME MODULE
2185800160 16:44:56 FREQRS 655 FRRD1
...
+------------------------------ DBSET FILES -----------------------------+
FILE ALLOCATED ALLOCATED HIWATER HIWATER I/O TRANSFERRED
(BLOCKS) (GB) (BLOCKS) (GB) (GB)
MASTER 5000 2.44 155 0.076 0.093
DBALL 2000000 976.56 3 0.001 0.004
OBJSCR 5000 0.31 387 0.024 0.032
(MEMFILE 32768 16.00 31291 15.279 0.000)
SCRATCH 2000000 976.56 1 0.000 0.000
SCR300 2000000 976.56 1 0.000 0.000
==============
TOTAL: 0.130
This model actually reached a hiwater of 17.48 GB, which means that UIM 4157 was off by a factor of roughly 60. Therefore I usually find little value in UIM 4157.
A practical procedure for finding optimal memory settings in Nastran
I therefore propose a two step process for finding what Nastran actually needs
How much memory you can set on the memory command will be dependent on machine and system. If you have a 32-bit machine or operating system, such as standard Windows XP, then the maximum will be 2 GB. If you are on a server you can set the mode=i8 flag and can use as much memory as is installed in the machine. The following will assume that a user knows how much memory is physically available in a system, the installed memory minus the operating system overhead and that used by other users.
The memory settings in Nastran are:
+------------------------------ DBSET FILES -----------------------------+
FILE ALLOCATED ALLOCATED HIWATER HIWATER I/O TRANSFERRED
(BLOCKS) (GB) (BLOCKS) (GB) (GB)
MASTER 5000 2.44 155 0.076 0.093
DBALL 2000000 976.56 3 0.001 0.004
OBJSCR 5000 0.31 387 0.024 0.032
(MEMFILE 32768 16.00 31291 15.279 0.000)
SCRATCH 2000000 976.56 1 0.000 0.000
SCR300 2000000 976.56 1 0.000 0.000
==============
TOTAL: 0.130
For instance, this job required 15.28 of the 16GB allocated to Smemory.
Then check the hiwater. This will tell you how much hicore (memory-smemory) you need.
*** TOTAL MEMORY AND DISK USAGE STATISTICS ***
+---------- SPARSE SOLUTION MODULES -----------+
HIWATER SUB_DMAP DMAP
(WORDS) DAY_TIME NAME MODULE
2185800160 16:44:56 FREQRS 655 FRRD1
This particular job was run with mode=i8, and needs 2,185,800,160 words x 8 bytes/word=17.48GB of hicore. Therefore this job, rerun, would use memory=17.48GB + 15.28GB=32.76GB and smemory=15.28GB, with mode=i8. Note that if hicore>hiwater, that means that not enough memory-smemory was set, and that this amount should be increased if it can.
Setting SMP and DMP
Nastran has a long history of SMP type parallelization. Use either the 'SMP' or 'parallel' keywords to set parallel execution, which will work for nearly any modern installation and will use no more RAM.
DMP can also be used in solutions 101,103,108,110,111, and 112. Check the domainsolver executive control statement for more details on how to divide up the problem. When executing, use the 'DMP' keyword to set the number of domains. If there are other machines on the network that have the same version as Nastran and can accept the rsh command, then they can also be used to run Nastran in DMP. Use the hosts=node1:node2:node3:node4 keyword. More details on how to configure this can be found in the MSC Nastran Installation and Operations guide, 'Runing Distributed Memory Parallel (DMP) Jobs'. Similar to Optistruct, each DMP requires its own memory allocation, so memory required is Memory x DMP.
GPU is a very new development for Nastran, and unfortunately is only supported in SOL 101.
- Get off the hard drive
- DMP until you run out of RAM
- SMP until you run out of processors
Limits of Nastran estimate Procedures
If you follow the official recommendations, there are in fact two ways to estimate the memory needed to not use the hard drive. The first method is to use the 'estimate' program. It will generate output as seen below:
Reading input file "./.bdf"
...
Submit with command line arguments:
memory=532.9mb
Estimated Resource Requirements on a 32-bit system:
Memory: 532.9 MB
Disk: 9125.4 MB
DBALL: 6099.1 MB
SCRATCH: 3049.6 MB
SCR300: 845.3 MB
SMEM: 25.0 MB
Well this looks very helpful, except it's very very wrong. This is the output for a model I know used memory=33GB of memory to run at full speed. Estimate didn't even try to recommend an aggressive smemory setting to reduce the amount of data written to the Scratch and Scr300 files.
The other recommended way to find the optimal memory settings is to run the model until the .f04 file displays UIM (User Information Message) 4157.
*** USER INFORMATION MESSAGE 4157 (DFMSYM)
PARAMETERS FOR PARALLEL SPARSE DECOMPOSITION OF DATA BLOCK SCRATCH ( TYPE=CSP ) FOLLOW
MATRIX SIZE = 1163947 ROWS NUMBER OF NONZEROES = 29408809 TERMS
NUMBER OF ZERO COLUMNS = 0 NUMBER OF ZERO DIAGONAL TERMS = 0
SYSTEM (107) = 32770 REQUESTED PROC. = 2 CPUS
ELIMINATION TREE DEPTH = 8372
CPU TIME ESTIMATE = 915 SEC I/O TIME ESTIMATE = 3 SEC
MINIMUM MEMORY REQUIREMENT = 36169 K WORDS MEMORY AVAILABLE = 2807326 K WORDS
MEMORY REQR'D TO AVOID SPILL = 36432 K WORDS MEMORY USED BY BEND = 36170 K WORDS
EST. INTEGER WORDS IN FACTOR = 143016 K WORDS EST. NONZERO TERMS = 299087 K TERMS
ESTIMATED MAXIMUM FRONT SIZE = 2429 TERMS RANK OF UPDATE = 64
If we examine the .log file for this .f04, we see that mode=i8 was chosen, meaning each word is 8 bytes. Multiplying the Memory Reqr'd to avoid spill, aka the estimate for in-core memory, we find that UIM 4157 estimates that 36.432 megawords x 8 bytes/word=291 Megabytes of memory will be required of the 2.807 Gigawords x 8 bytes/word=22.46 Gigabytes made available to hicore by the difference between the mem and smem memory settings. So what did this model actually use? If we read the Total Memory and Disk Usage Statistics section in the .f04 file, we see the following useful output data:
*** TOTAL MEMORY AND DISK USAGE STATISTICS ***
+---------- SPARSE SOLUTION MODULES -----------+
HIWATER SUB_DMAP DMAP
(WORDS) DAY_TIME NAME MODULE
2185800160 16:44:56 FREQRS 655 FRRD1
...
+------------------------------ DBSET FILES -----------------------------+
FILE ALLOCATED ALLOCATED HIWATER HIWATER I/O TRANSFERRED
(BLOCKS) (GB) (BLOCKS) (GB) (GB)
MASTER 5000 2.44 155 0.076 0.093
DBALL 2000000 976.56 3 0.001 0.004
OBJSCR 5000 0.31 387 0.024 0.032
(MEMFILE 32768 16.00 31291 15.279 0.000)
SCRATCH 2000000 976.56 1 0.000 0.000
SCR300 2000000 976.56 1 0.000 0.000
==============
TOTAL: 0.130
This model actually reached a hiwater of 17.48 GB, which means that UIM 4157 was off by a factor of roughly 60. Therefore I usually find little value in UIM 4157.
A practical procedure for finding optimal memory settings in Nastran
I therefore propose a two step process for finding what Nastran actually needs
- Give Nastran as much memory as your system has
- Find out what Nastran actually used
How much memory you can set on the memory command will be dependent on machine and system. If you have a 32-bit machine or operating system, such as standard Windows XP, then the maximum will be 2 GB. If you are on a server you can set the mode=i8 flag and can use as much memory as is installed in the machine. The following will assume that a user knows how much memory is physically available in a system, the installed memory minus the operating system overhead and that used by other users.
The memory settings in Nastran are:
- memory=XGB, where X is the memory in Gigabytes. The total memory allocated
- smemory=XGB, where X is the smemory in Gigabytes. Smemory is short for Scratch Memory. Remember that the memory available to hicore, or the main solver, is mem-smem. Therefore smem must always be smaller than mem, and increasing smem without increasing mem will reduce the amount of data available to the main solver
- mode=i4 (default) or i8. This sets 4 bytes/word or 8 bytes/word. Set it to 8 bytes word on any 64 bit machine with more than 16GB of memory so that Nastran can take full advantage of the memory in the machine. Note that if a model requires memory<16GB when mode was set to i8, you can take that memory required, divide it by half, and set mode=i4.
- 32-bit machines : memory=2GB smemory=1GB
- 64-bit machine with 8GB of memory: memory=6GB smemory=3GB mode=i4 (Try not to use more than ~80% of system memory)
- 64-bit machine with 8GB to 16GB of memory: memory=8GB smemory=4GB mode=i4
- 64-bit machine with >16GB of memory: memory=(.8xtotal installed memory) smemory=(memory/2) mode=i8
+------------------------------ DBSET FILES -----------------------------+
FILE ALLOCATED ALLOCATED HIWATER HIWATER I/O TRANSFERRED
(BLOCKS) (GB) (BLOCKS) (GB) (GB)
MASTER 5000 2.44 155 0.076 0.093
DBALL 2000000 976.56 3 0.001 0.004
OBJSCR 5000 0.31 387 0.024 0.032
(MEMFILE 32768 16.00 31291 15.279 0.000)
SCRATCH 2000000 976.56 1 0.000 0.000
SCR300 2000000 976.56 1 0.000 0.000
==============
TOTAL: 0.130
For instance, this job required 15.28 of the 16GB allocated to Smemory.
Then check the hiwater. This will tell you how much hicore (memory-smemory) you need.
*** TOTAL MEMORY AND DISK USAGE STATISTICS ***
+---------- SPARSE SOLUTION MODULES -----------+
HIWATER SUB_DMAP DMAP
(WORDS) DAY_TIME NAME MODULE
2185800160 16:44:56 FREQRS 655 FRRD1
This particular job was run with mode=i8, and needs 2,185,800,160 words x 8 bytes/word=17.48GB of hicore. Therefore this job, rerun, would use memory=17.48GB + 15.28GB=32.76GB and smemory=15.28GB, with mode=i8. Note that if hicore>hiwater, that means that not enough memory-smemory was set, and that this amount should be increased if it can.
Setting SMP and DMP
Nastran has a long history of SMP type parallelization. Use either the 'SMP' or 'parallel' keywords to set parallel execution, which will work for nearly any modern installation and will use no more RAM.
DMP can also be used in solutions 101,103,108,110,111, and 112. Check the domainsolver executive control statement for more details on how to divide up the problem. When executing, use the 'DMP' keyword to set the number of domains. If there are other machines on the network that have the same version as Nastran and can accept the rsh command, then they can also be used to run Nastran in DMP. Use the hosts=node1:node2:node3:node4 keyword. More details on how to configure this can be found in the MSC Nastran Installation and Operations guide, 'Runing Distributed Memory Parallel (DMP) Jobs'. Similar to Optistruct, each DMP requires its own memory allocation, so memory required is Memory x DMP.
GPU is a very new development for Nastran, and unfortunately is only supported in SOL 101.
Friday, October 5, 2012
Friday Video Mash-Up: BeamNG
As I mentioned earlier, BeamNG does an amazing job of simulating physics in video games. Here is a video of it in action.
BeamNG offers some caveats in their description of their video: "If things seem too bouncy, please remember we're using real life spring and damper rates, and it probably looks wrong because nobody ever does anything this extreme with a real car". For comparison, here is a real crash test:
So what do we notice when we compare the two? I see two things:
Same model, but with a cross section cut thruough it
Looking at this model, it appears that the rapid permanent set of the material happens in a much more realistic fashion. This can easily be explained by the much higher levels of detail allowed by a model which does not have to run in real time. But the body panels are still bouncing around! Why is this? For insight, I first looked at the Abaqus input deck to see what was being used to try to stop the body panels from bouncing around. Specifically, what I'm look for is 'damping', the name for what keeps a bell from ringing forever. Unsurprisingly, I found nothing: this model has no damping in it outside of the suspension. Why would this model not include realistic damping to create a more realistic simulation?
The answer will be very familiar to FEA analysts who do significant amounts of work with explicit FEA solvers. Damping of the kind typically found in structures is of a form that is proportional to the deformation of the metal, and as can be seen in the Abaqus Analysis Users's Manual ver 6.10, section 23.1.1, small amounts of this 'stiffness proportional' damping can reduce the stable time increment, i.e. how much time can pass in one simulation step without the FEA model exploding. This reduces the speed of the simulation, and for a relatively minor amount of structural damping can reduce the speed of the simulation by a factor of ten or more. So for an FEA analyst simulating a car crash or cell phone impact, not only is damping not going to have a very large effect over the very short time scales of an impact, it will significantly increase the time it takes to arrive at a solution. This is why damping was left out of this model and why it behaves strangely. Damping can be so hard to add to explicit FEA solvers that a typical approach is to just filter the results after the fact.
Which brings us back to BeamNG. In a recent interview, the developers spoke at length of the challenges faced by video game developers trying to do structural simulation in real time, stating that "Mass-spring systems have very bad stability, they tend to explode, and are very CPU intensive." My guess is that BeamNG may have experimented with more realistic damping rates for more of the vehicle, but after noticing the massive changes in stability in their simulation removed most of it, leaving damping only in the suspension.
Of course, all this is nitpicking, and I wish them the best of luck in their ambitious endeavor. If I may offer some unsolicited advice, I'd suggest that they be the ones to make the first game that uses their technology. All the successful video game engines and rendering software that I can think of were noticed after the developers made an original product that captured the interest of the public. Consider Pixar's RenderMan, the Doom engine, the Unreal engine, the Crytek Cryengine, and many others. These may never have been used as widely had the developers of these engines not used their intimate knowledge of what their technology can do to make something great. Fortunately, it looks like BeamNG is already heading down this path, and I look forward to a new way to use my computer to simulate smashing things.
BeamNG offers some caveats in their description of their video: "If things seem too bouncy, please remember we're using real life spring and damper rates, and it probably looks wrong because nobody ever does anything this extreme with a real car". For comparison, here is a real crash test:
So what do we notice when we compare the two? I see two things:
- In the real crash, the metal quickly deforms into a complex shape, and doesn't bounce around much after that, similar to the process of smashing an aluminum can.
- In the real crash, there is some bouncing around of the thin metal on the outside of the vehicle. The bouncing of these 'body panels' is visible immediately after impact but quickly stops. By comparison, BeamNG does indeed seem to bouncy.
Same model, but with a cross section cut thruough it
Looking at this model, it appears that the rapid permanent set of the material happens in a much more realistic fashion. This can easily be explained by the much higher levels of detail allowed by a model which does not have to run in real time. But the body panels are still bouncing around! Why is this? For insight, I first looked at the Abaqus input deck to see what was being used to try to stop the body panels from bouncing around. Specifically, what I'm look for is 'damping', the name for what keeps a bell from ringing forever. Unsurprisingly, I found nothing: this model has no damping in it outside of the suspension. Why would this model not include realistic damping to create a more realistic simulation?
The answer will be very familiar to FEA analysts who do significant amounts of work with explicit FEA solvers. Damping of the kind typically found in structures is of a form that is proportional to the deformation of the metal, and as can be seen in the Abaqus Analysis Users's Manual ver 6.10, section 23.1.1, small amounts of this 'stiffness proportional' damping can reduce the stable time increment, i.e. how much time can pass in one simulation step without the FEA model exploding. This reduces the speed of the simulation, and for a relatively minor amount of structural damping can reduce the speed of the simulation by a factor of ten or more. So for an FEA analyst simulating a car crash or cell phone impact, not only is damping not going to have a very large effect over the very short time scales of an impact, it will significantly increase the time it takes to arrive at a solution. This is why damping was left out of this model and why it behaves strangely. Damping can be so hard to add to explicit FEA solvers that a typical approach is to just filter the results after the fact.
Which brings us back to BeamNG. In a recent interview, the developers spoke at length of the challenges faced by video game developers trying to do structural simulation in real time, stating that "Mass-spring systems have very bad stability, they tend to explode, and are very CPU intensive." My guess is that BeamNG may have experimented with more realistic damping rates for more of the vehicle, but after noticing the massive changes in stability in their simulation removed most of it, leaving damping only in the suspension.
Of course, all this is nitpicking, and I wish them the best of luck in their ambitious endeavor. If I may offer some unsolicited advice, I'd suggest that they be the ones to make the first game that uses their technology. All the successful video game engines and rendering software that I can think of were noticed after the developers made an original product that captured the interest of the public. Consider Pixar's RenderMan, the Doom engine, the Unreal engine, the Crytek Cryengine, and many others. These may never have been used as widely had the developers of these engines not used their intimate knowledge of what their technology can do to make something great. Fortunately, it looks like BeamNG is already heading down this path, and I look forward to a new way to use my computer to simulate smashing things.
Wednesday, October 3, 2012
The need for speed: Part 3 of 7 - Performance Tuning Parameters: Abaqus and Optistruct Linear
Previously we've discussed the three main priorities when trying to use the optimal settings for an FEA solver.
So given what we now know, how do we accomplish the optimal use of computer resources using specific commercial FEA software?
A Note on system limitations
The following blog post assumes a user is already knowledgeable about the limits of the system they are running their FEA simulations on. Techniques for checking available system resources vary considerably between windows and Unix/Linux based environments, and identical hardware may have different limitations based on 32-bit vs. 64-bit operating system limits, among other things. I plan to cover this topic in more detail in a later post.
ABAQUS
Of the three codes I typically cover, Abaqus is the easiest to properly configure for speed. All Abaqus needs to know is how much memory is free on a computer, how many processors are free on a computer, and what machines it can use. There is almost nothing else to set.
To begin, start by doing a datacheck. Assuming you know how much memory and how many cpus are in your computer or server, and that you know how to execute using the command line, start with the command:
abq6101.bat memory="1 gb" datacheck job=jobfile.inp
note that abq6101.bat is what I use on my local machine. On linux servers it will typically be /opt/abaqus/Commands/abq61... depending on version.
Hopefully your job has been carefully prepared and the Abaqus pre-processor will finish without any errors. On linux machines, or windows machines running cygwin, you can monitor the progress of the datacheck run with the tail command. Ex:
tail -n 100 -f jobfile.dat
The important data will be held in a section that reads:
PROCESS FLOATING PT MINIMUM MEMORY MEMORY TO
OPERATIONS REQUIRED MINIMIZE I/O
PER ITERATION (MBYTES) (MBYTES)
1 1.86E+012 691 5623
For this run, at least 691 mb must be available to the abaqus solver, and no more than 5623 mb needs to be used by it.
As an aside, Abaqus handles DMP in a very different way than Nastran or Optistruct. Whereas Nastran and Optistruct multiply their memory usage by a factor of memory requestedxDMP, Abaqus will report the total memory needed by all the DMP processes. This is because Abaqus has put significant development effort on a style of DMP that's not quite full DMP, one that is not akin to a full box of cards being sorted independently on different tables or in different rooms, but is more like several people at one table, but with significant effort being put into sorting out who gets what chunk of cards ahead of time. In this way, with some effort by the first person to grab the deck of cards, many people can be kept busy on a smaller table, and a single deck of cards can even still be worked on a great deal of tables. When there are several decks of cards to sort this becomes less efficient than the Nastran/Optistruct style, but Abaqus' approach is the best for attacking one deck.
Returning to the memory settings, because the memory total is the true memory total, one need only set it to whatever is the maximum actually available on a computer and Abaqus will handle the rest. Abaqus is even smart enough to only use as much memory as it needs, as long as that is less than what you allocate it.
So what happens if we have a desktop computer with 8GB of ram and a 64-bit operating system with virtual memory, and we try running it with memory ranging from the bare minimum to the maximum?
So what happened? Wasn't adding memory the most important thing? Well here's the big surprise: a modern operating system helps a great deal when you do one of two suboptimal things:
So let's say you now have enough memory, how do I use more cpus? Again, the syntax is simple
abq6101.bat memory="6 gb" cpus=2 job=jobfile.inp
In this case we are requesting 6 GB of memory and 2 cpus. What are the performance benefits from additional cpus? Typically, the first cpu nearly doubles performance, 4 cpus is a little more than triple speed, and so on as the returns diminish. More detail can be found here.
How about Gpus? Although I don't have a system that can take advantage of it, in the latest abaqus 6.12 release it is selected with the syntax
abq6121 memory="6 gb" cpus=2 gpus=1 job=jobfile.inp
The above will use two cpus, 6GB of memory, and 1 GPU (1 video card). Note that there are significant limits on the solvers that gpus can be used on, specifically only the sparse symmetrix solver in implicit solutions. This rules out explicit and extremely nonlinear problems, such as those with coefficients of friction higher than 0.2
Finally, the use of more than one host is enabled in the abaqus_v6.env file. HP-MPI or some other interconnect must be enabled, and infiniband will yield measureably higher performance than 1GBit ethernet. When using extra hosts, check to make sure that rsh commands are enabled on ssh through the use of key based logins. To enable additional hosts, include in the abaqus_v6.env file the following line:
mp_host_list=[['host1',4],['host2',4]]
The first entry is the name of the host on the network, the second the number of cpus that can be used on that host. Any run that uses more than one cpu and has extra hosts available in the abaqus_v6.env file will take advantage of them.
One final thing to remember is that many of the previous memory and cpu settings where originally only set in the abaqus_v6.env file. It may be worth it to check what parameters remain in the file, either from yourself or your adminstrator, as many of them have changed over the last few versions.
Optistruct
Optistruct can be optimized in a similar, by first running a datacheck and then using the correct memory parameters.
If one is in a hurry, one can simply run optistruct with the below settings
radioss -core in -cpu 2 jobfile.fem
If sufficient memory is available to run in-core with minimum disk use, it will use it, and in this example it will use two cpus in SMP style. Note that if insufficient memory is available for in-core it will error out.
If we want to see what a job will need, then we can instead run
radioss -check jobfile.fem
Assuming your model is built well, the .out file will eventually read like this:
MEMORY ESTIMATION INFORMATION :
-------------------------------
Solver Type is: Sparse-Matrix Solver
Direct Method
Current Memory (RAM) : 116 MB
Estimated Minimum Memory (RAM) for Minimum Core Solution: 154 MB
Recommended Memory (RAM) for Minimum Core Solution : 154 MB
Estimated Minimum Memory (RAM) for Out of Core Solution : 189 MB
Recommended Memory (RAM) for Out of Core Solution : 211 MB
Recommended Memory (RAM) for In-Core Solution : 1515 MB
Recommended Number of Nodes for OS SPMD Parallel Run : 1
(Note: Minimum Core Solution Process is Activated.)
(Note: The Minimum Memory Requirement is limited by Assembly Module.)
(Note: Use param,HASHASSM,yes to avoid assembly module memory bottleneck)
DISK SPACE ESTIMATION INFORMATION :
-----------------------------------
Estimated Disk Space for Output Data Files : 18 MB
Estimated Scratch Disk Space for In-Core Solution : 208 MB
Estimated Scratch Disk Space for Out of Core Solution : 1895 MB
Estimated Scratch Disk Space for Minimum Core Solution : 1952 MB
In the above example, we would need a total of 1515+208MB of ram for a maximum reduction in hard drive use. The syntax for this would be
radioss -len 1723 -ramdisk 208 -cpu 2 jobfile.fem
As noted before, additional -cpu cpu's add some measure of speed but do not require more ram.
If a job is of a type that can also DMP, then they can be requested using the syntax:
radioss -len 1723 -ramdisk 208 -cpu 2 -mpi -np 4 jobfile.fem
A few things I've found: hpmpi sometimes has issues, so take care with the documentation to show it the path to the hpmpi files if necessary. Also note that the total memory demands are now 4x1723=6892 MB, and the total number of cpus needed is 4x2=8. Solution sequences that make use of DMP well tend to be direct frequency and lanczos eigensolutions, although eigensolutions will nearly always be faster when using a modern multilevel eigensolver.
Closing Remarks
As we've seen, different solvers can be very different in how they understand the tuning parameters given to them by users. Additionally, there is typically a significant amount to be learned before even solving a model by simply invoking a checkout run. As we will see in the upcoming Nastran entry, checkout runs are unfortunately not as capable in MSC/MD Nastran.
- Get off the hard drive
- DMP until you run out of RAM
- SMP until you run out of processors
So given what we now know, how do we accomplish the optimal use of computer resources using specific commercial FEA software?
A Note on system limitations
The following blog post assumes a user is already knowledgeable about the limits of the system they are running their FEA simulations on. Techniques for checking available system resources vary considerably between windows and Unix/Linux based environments, and identical hardware may have different limitations based on 32-bit vs. 64-bit operating system limits, among other things. I plan to cover this topic in more detail in a later post.
ABAQUS
Of the three codes I typically cover, Abaqus is the easiest to properly configure for speed. All Abaqus needs to know is how much memory is free on a computer, how many processors are free on a computer, and what machines it can use. There is almost nothing else to set.
To begin, start by doing a datacheck. Assuming you know how much memory and how many cpus are in your computer or server, and that you know how to execute using the command line, start with the command:
abq6101.bat memory="1 gb" datacheck job=jobfile.inp
note that abq6101.bat is what I use on my local machine. On linux servers it will typically be /opt/abaqus/Commands/abq61... depending on version.
Hopefully your job has been carefully prepared and the Abaqus pre-processor will finish without any errors. On linux machines, or windows machines running cygwin, you can monitor the progress of the datacheck run with the tail command. Ex:
tail -n 100 -f jobfile.dat
The important data will be held in a section that reads:
PROCESS FLOATING PT MINIMUM MEMORY MEMORY TO
OPERATIONS REQUIRED MINIMIZE I/O
PER ITERATION (MBYTES) (MBYTES)
1 1.86E+012 691 5623
For this run, at least 691 mb must be available to the abaqus solver, and no more than 5623 mb needs to be used by it.
As an aside, Abaqus handles DMP in a very different way than Nastran or Optistruct. Whereas Nastran and Optistruct multiply their memory usage by a factor of memory requestedxDMP, Abaqus will report the total memory needed by all the DMP processes. This is because Abaqus has put significant development effort on a style of DMP that's not quite full DMP, one that is not akin to a full box of cards being sorted independently on different tables or in different rooms, but is more like several people at one table, but with significant effort being put into sorting out who gets what chunk of cards ahead of time. In this way, with some effort by the first person to grab the deck of cards, many people can be kept busy on a smaller table, and a single deck of cards can even still be worked on a great deal of tables. When there are several decks of cards to sort this becomes less efficient than the Nastran/Optistruct style, but Abaqus' approach is the best for attacking one deck.
Returning to the memory settings, because the memory total is the true memory total, one need only set it to whatever is the maximum actually available on a computer and Abaqus will handle the rest. Abaqus is even smart enough to only use as much memory as it needs, as long as that is less than what you allocate it.
So what happens if we have a desktop computer with 8GB of ram and a 64-bit operating system with virtual memory, and we try running it with memory ranging from the bare minimum to the maximum?
- Using less memory than you actually have
- Using more memory than you actually have
So let's say you now have enough memory, how do I use more cpus? Again, the syntax is simple
abq6101.bat memory="6 gb" cpus=2 job=jobfile.inp
In this case we are requesting 6 GB of memory and 2 cpus. What are the performance benefits from additional cpus? Typically, the first cpu nearly doubles performance, 4 cpus is a little more than triple speed, and so on as the returns diminish. More detail can be found here.
How about Gpus? Although I don't have a system that can take advantage of it, in the latest abaqus 6.12 release it is selected with the syntax
abq6121 memory="6 gb" cpus=2 gpus=1 job=jobfile.inp
The above will use two cpus, 6GB of memory, and 1 GPU (1 video card). Note that there are significant limits on the solvers that gpus can be used on, specifically only the sparse symmetrix solver in implicit solutions. This rules out explicit and extremely nonlinear problems, such as those with coefficients of friction higher than 0.2
Finally, the use of more than one host is enabled in the abaqus_v6.env file. HP-MPI or some other interconnect must be enabled, and infiniband will yield measureably higher performance than 1GBit ethernet. When using extra hosts, check to make sure that rsh commands are enabled on ssh through the use of key based logins. To enable additional hosts, include in the abaqus_v6.env file the following line:
mp_host_list=[['host1',4],['host2',4]]
The first entry is the name of the host on the network, the second the number of cpus that can be used on that host. Any run that uses more than one cpu and has extra hosts available in the abaqus_v6.env file will take advantage of them.
One final thing to remember is that many of the previous memory and cpu settings where originally only set in the abaqus_v6.env file. It may be worth it to check what parameters remain in the file, either from yourself or your adminstrator, as many of them have changed over the last few versions.
Optistruct
Optistruct can be optimized in a similar, by first running a datacheck and then using the correct memory parameters.
If one is in a hurry, one can simply run optistruct with the below settings
radioss -core in -cpu 2 jobfile.fem
If sufficient memory is available to run in-core with minimum disk use, it will use it, and in this example it will use two cpus in SMP style. Note that if insufficient memory is available for in-core it will error out.
If we want to see what a job will need, then we can instead run
radioss -check jobfile.fem
Assuming your model is built well, the .out file will eventually read like this:
MEMORY ESTIMATION INFORMATION :
-------------------------------
Solver Type is: Sparse-Matrix Solver
Direct Method
Current Memory (RAM) : 116 MB
Estimated Minimum Memory (RAM) for Minimum Core Solution: 154 MB
Recommended Memory (RAM) for Minimum Core Solution : 154 MB
Estimated Minimum Memory (RAM) for Out of Core Solution : 189 MB
Recommended Memory (RAM) for Out of Core Solution : 211 MB
Recommended Memory (RAM) for In-Core Solution : 1515 MB
Recommended Number of Nodes for OS SPMD Parallel Run : 1
(Note: Minimum Core Solution Process is Activated.)
(Note: The Minimum Memory Requirement is limited by Assembly Module.)
(Note: Use param,HASHASSM,yes to avoid assembly module memory bottleneck)
DISK SPACE ESTIMATION INFORMATION :
-----------------------------------
Estimated Disk Space for Output Data Files : 18 MB
Estimated Scratch Disk Space for In-Core Solution : 208 MB
Estimated Scratch Disk Space for Out of Core Solution : 1895 MB
Estimated Scratch Disk Space for Minimum Core Solution : 1952 MB
In the above example, we would need a total of 1515+208MB of ram for a maximum reduction in hard drive use. The syntax for this would be
radioss -len 1723 -ramdisk 208 -cpu 2 jobfile.fem
As noted before, additional -cpu cpu's add some measure of speed but do not require more ram.
If a job is of a type that can also DMP, then they can be requested using the syntax:
radioss -len 1723 -ramdisk 208 -cpu 2 -mpi -np 4 jobfile.fem
A few things I've found: hpmpi sometimes has issues, so take care with the documentation to show it the path to the hpmpi files if necessary. Also note that the total memory demands are now 4x1723=6892 MB, and the total number of cpus needed is 4x2=8. Solution sequences that make use of DMP well tend to be direct frequency and lanczos eigensolutions, although eigensolutions will nearly always be faster when using a modern multilevel eigensolver.
Closing Remarks
As we've seen, different solvers can be very different in how they understand the tuning parameters given to them by users. Additionally, there is typically a significant amount to be learned before even solving a model by simply invoking a checkout run. As we will see in the upcoming Nastran entry, checkout runs are unfortunately not as capable in MSC/MD Nastran.
Thursday, September 27, 2012
The need for speed: Part 2 of 7 - Performance tuning parameters: Common Topics
What are the most important things to keep in mind when changing the settings for an FEA solver?
An FEA solver must first create a system of linear equations which describe the FEA problem to be solved. This is sometimes called assembling the stiffness matrix, although other matrices may be assembled, such as mass or damping. These equations are stored in matrix form, and must then be operated on by a mathematical solver in order to arrive at the solution to the FEA problem. In approximate order of increasing memory requirements, the typical solvers seen in structural FEA are:
So what sort of memory should we use to store the system of equations? Given the layout of a typical computer's memory hierarchy, the most practical, high speed place to store the system of equations so that they can be quickly solved is the random access memory or memory of the computer.
At this point those with a background in computers may be wondering what if the system of equations is too large to fit in the free memory of the computer? A typical computer program would have the operating system store whatever didn't fit into virtual memory, or hard drive space that behaves as memory. For most programs, this is the best thing to do if there is insufficient memory. FEA solvers, however, are different. FEA solvers predate operating systems with virtual memory, and have developed their own specialized methods of making due with less than the ideal amount of memory. What an FEA solver does instead is if the memory allocated to it is less than what the system of equations needs, then it will revert from what is known as an in-core solver to an out-of-core solver which will load a portion of the system of equations and solve it before putting it away in a scratch file and loading a new chunk to work on.
So now we have two interesting things to remember:
So now we've given a general answer to the first question, how to get off the hard drive: allocate enough memory. Now what?
Now we DMP (distributed memory process).
Solving a large system of equations is analogous to sitting at a table and ordering a shuffled deck of cards. If we have only one deck of cards to sort, and only one desk, and only one person sorting them, then there are no decisions to be made about how to allocate resources. Now let's say you have a second person that can help you, and a completely separate deck of cards to sort. If you have enough space on the desk, then they can spread out the second deck and begin sorting without slowing you down. This is equivalent to a DMP run on a single computer: enough memory (desk space) and enough people (CPUs) and a problem that can be divided up (seperate decks of cards). This is confusing for many at first, including myself, because a computer with more than one CPU that shares the same pool of memory is technically an SMP computer system. So from a computer science perspective, a desk with a piece of tape down the middle is an SMP system doing DMP style work, whereas two separate desks in two separate rooms is like true DMP.
So what sort of FEA solutions lend themselves well to DMP style splitting of workload? Many, in fact, although speedup can be very solver and solution type dependent. DMP can be used by lanczos eigensolvers, and very easily used by direct frequency analyses, where there is a frequency that needs to be found at each frequency and each frequencies' solution can be found independently. It can also be used by ABAQUS for gaussian elimination, iterative, and explicit solvers, although the overhead of 'passing cards' from room to room can begin to dominate the solution times.
So what if you have only one computer, and you don't have enough memory to split the solution up any more? This is where SMP can be of use, and is in fact one of the oldest ways that more than one CPU has been used to accelerate FEA solution times, dating to Nastran running on CRAY supercomputers. SMP is equivalent to having only enough room on the desk to work on the one deck of cards, but more than one person reaching in and working on ordering the cards. As you can imagine, returns quickly diminish for this type of parallelization, but if the CPUs are available there's no reason not to take advantage of it. Nearly any solver and nearly any solution type can take advantage of this style of parallelization, with the notable exception of Abaqus, which has instead focused on parallizing in a manner that divvies up the cards intelligently ahead of time.
There is one final, more exotic, method of SMP that is sometimes available. With the growing general abilities of the vector processors inside of the graphics cards in computers and video game systems, programmers have begun to use them as a new type of person working to sort the deck of cards. Only in this instance, its more like a trained bag of mice, mice which still need a person to bring the deck of cards to the table and spread them out, but can handle much of the hard work from there. This type of acceleration, oftentimes referred to as GPU acceleration, will only become more prevalent as the limits in using more and more transistors to make a single cpu faster become apparent. As of now though, support is very limited, and only the most basic static solution of Nastran and the Gaussian solver, non complex, symmetric solver of Abaqus is supported, and only on post Windows XP operating systems. The latest Intel chips,
Sandy Bridge and later, have a functionality similar to this that is much better supported in Nastran than GPU acceleration.
We've now covered, in very general terms, how to get off the hard drive and make the most of our ram, and having done that make the most of our CPUs and even our graphics cards. Next we will cover solver specific tuning parameters that take advantage of these concepts.
- Get off the hard drive
- DMP until you run out of RAM
- Parallel whatever cores you have left
An FEA solver must first create a system of linear equations which describe the FEA problem to be solved. This is sometimes called assembling the stiffness matrix, although other matrices may be assembled, such as mass or damping. These equations are stored in matrix form, and must then be operated on by a mathematical solver in order to arrive at the solution to the FEA problem. In approximate order of increasing memory requirements, the typical solvers seen in structural FEA are:
- Forward Integration time step: explicit transient dynamics, typically short duration transients, such as vehicle impact
- Iterative: statics, linear and nonlinear, typically dense and blocky models, such as an engine block with mild contact nonlinearity
- Gaussian Direct Elimination, non complex: statics, linear and nonlinear, steady state or implicit transient. One of the most common solver types
- Gaussian Direct Elimination, complex : frequency response, the steady state response of a structure when loaded by loads which all have the same forcing frequency, such as a speaker generating a single tone or vehicle NVH analyses across a frequency range
- Eigenvalue solvers, modern methods (automated multilevel substructuring): natural frequency and buckling loads, also used for modal type solutions to direct frequency and transient responses
- Eigenvalue solvers, older methods (Lanczos, etc.)
So what sort of memory should we use to store the system of equations? Given the layout of a typical computer's memory hierarchy, the most practical, high speed place to store the system of equations so that they can be quickly solved is the random access memory or memory of the computer.
At this point those with a background in computers may be wondering what if the system of equations is too large to fit in the free memory of the computer? A typical computer program would have the operating system store whatever didn't fit into virtual memory, or hard drive space that behaves as memory. For most programs, this is the best thing to do if there is insufficient memory. FEA solvers, however, are different. FEA solvers predate operating systems with virtual memory, and have developed their own specialized methods of making due with less than the ideal amount of memory. What an FEA solver does instead is if the memory allocated to it is less than what the system of equations needs, then it will revert from what is known as an in-core solver to an out-of-core solver which will load a portion of the system of equations and solve it before putting it away in a scratch file and loading a new chunk to work on.
So now we have two interesting things to remember:
- The same level of detail may take different amounts of memory to solve depending on solution type
- The solver will need to know how much memory is actually available so it can intelligently decide how to store the system of equations, either in-core or out-of-core
So now we've given a general answer to the first question, how to get off the hard drive: allocate enough memory. Now what?
Now we DMP (distributed memory process).
Solving a large system of equations is analogous to sitting at a table and ordering a shuffled deck of cards. If we have only one deck of cards to sort, and only one desk, and only one person sorting them, then there are no decisions to be made about how to allocate resources. Now let's say you have a second person that can help you, and a completely separate deck of cards to sort. If you have enough space on the desk, then they can spread out the second deck and begin sorting without slowing you down. This is equivalent to a DMP run on a single computer: enough memory (desk space) and enough people (CPUs) and a problem that can be divided up (seperate decks of cards). This is confusing for many at first, including myself, because a computer with more than one CPU that shares the same pool of memory is technically an SMP computer system. So from a computer science perspective, a desk with a piece of tape down the middle is an SMP system doing DMP style work, whereas two separate desks in two separate rooms is like true DMP.
So what sort of FEA solutions lend themselves well to DMP style splitting of workload? Many, in fact, although speedup can be very solver and solution type dependent. DMP can be used by lanczos eigensolvers, and very easily used by direct frequency analyses, where there is a frequency that needs to be found at each frequency and each frequencies' solution can be found independently. It can also be used by ABAQUS for gaussian elimination, iterative, and explicit solvers, although the overhead of 'passing cards' from room to room can begin to dominate the solution times.
So what if you have only one computer, and you don't have enough memory to split the solution up any more? This is where SMP can be of use, and is in fact one of the oldest ways that more than one CPU has been used to accelerate FEA solution times, dating to Nastran running on CRAY supercomputers. SMP is equivalent to having only enough room on the desk to work on the one deck of cards, but more than one person reaching in and working on ordering the cards. As you can imagine, returns quickly diminish for this type of parallelization, but if the CPUs are available there's no reason not to take advantage of it. Nearly any solver and nearly any solution type can take advantage of this style of parallelization, with the notable exception of Abaqus, which has instead focused on parallizing in a manner that divvies up the cards intelligently ahead of time.
There is one final, more exotic, method of SMP that is sometimes available. With the growing general abilities of the vector processors inside of the graphics cards in computers and video game systems, programmers have begun to use them as a new type of person working to sort the deck of cards. Only in this instance, its more like a trained bag of mice, mice which still need a person to bring the deck of cards to the table and spread them out, but can handle much of the hard work from there. This type of acceleration, oftentimes referred to as GPU acceleration, will only become more prevalent as the limits in using more and more transistors to make a single cpu faster become apparent. As of now though, support is very limited, and only the most basic static solution of Nastran and the Gaussian solver, non complex, symmetric solver of Abaqus is supported, and only on post Windows XP operating systems. The latest Intel chips,
Sandy Bridge and later, have a functionality similar to this that is much better supported in Nastran than GPU acceleration.
We've now covered, in very general terms, how to get off the hard drive and make the most of our ram, and having done that make the most of our CPUs and even our graphics cards. Next we will cover solver specific tuning parameters that take advantage of these concepts.
Tuesday, September 25, 2012
The need for speed: Part 1 of 7 - Intro
How can I make my FEA simulation run faster?
Ask less of your computer, use more of your computer, or use a faster computer(s).
Once you've meshed the geometry, applied the material properties, applied the loads, assemble the load cases, and the model runs sometime today, doing more work to make the model run faster is probably the last thing on your mind. Tuning parameters and measures of the solution difficulty of a model can seem dauntingly complex, and the payoff from a model that runs faster may not seem worth it. However, a working knowledge of how to manage the detail and run times of a model can have some major payoffs:
In a series of posts, I'll outline the practical steps an analyst can take to speed up their run times. This is obviously an extremely complex topic to fully cover. Fortunately, a little knowledge can go a long way, and I will focus on the topics that typically have the largest impact. Upcoming posts will cover:
Use more of your computer:
Ask less of your computer, use more of your computer, or use a faster computer(s).
Once you've meshed the geometry, applied the material properties, applied the loads, assemble the load cases, and the model runs sometime today, doing more work to make the model run faster is probably the last thing on your mind. Tuning parameters and measures of the solution difficulty of a model can seem dauntingly complex, and the payoff from a model that runs faster may not seem worth it. However, a working knowledge of how to manage the detail and run times of a model can have some major payoffs:
- knowing ahead of time if a model has surpassed the ability of a computer to quickly solve
- minimizing storage and archiving requirements
- ability to run the same model several times to assess design sensitivities and optimize the performance of a design
In a series of posts, I'll outline the practical steps an analyst can take to speed up their run times. This is obviously an extremely complex topic to fully cover. Fortunately, a little knowledge can go a long way, and I will focus on the topics that typically have the largest impact. Upcoming posts will cover:
Use more of your computer:
- Performance tuning parameters: Common Topics
- Performance tuning parameters: Abaqus and Optistruct/Radioss
- Performance tuning parameters: Nastran
- A faster computer: performance metrics
- Reducing output requests
- Optimizing model detail and solution type
Thursday, September 20, 2012
MPCs: a quick summary
Many years ago one of my coworkers built a model in Optistruct and was disappointed to see the RBE that he'd built failed to provide the restraint he expected. He called up the technical support at Altair, and after some back and forth about the RBE element the support representative finally lost his patience: "It is not an element, it is a system of equations!"
So what are these equations, and what do they do?
First of all, they are indeed different from an element. Although FEA software may refer to them as a type of element, they have a fundamental difference from a normal element. A normal element adds to the global stiffness, mass, and damping matrices. An MPC (multi point constraint) is more like an SPC (single point constraint), in that it modifies instead of adds to the matrices that describe the model in order to describe the special behavior of these MPCs.
So all that sounds interesting, but it hasn't really told you much about what they do. I've thought of different ways to describe them to my coworkers, and the best I've come up with is this analogy. Basically, you're including a group of grids in a type of government, and you're letting them know what kind of leader they have. So what's an RBE2? An RBE2 is a monarchy, where the control grid is king, and tells the other grids what to do. An RBE3, on the other hand, is more like a democracy, where the President does what the people tell it to do. So what works and what doesn't work when we try to make use of these MPCs?
What Works
So what are these equations, and what do they do?
First of all, they are indeed different from an element. Although FEA software may refer to them as a type of element, they have a fundamental difference from a normal element. A normal element adds to the global stiffness, mass, and damping matrices. An MPC (multi point constraint) is more like an SPC (single point constraint), in that it modifies instead of adds to the matrices that describe the model in order to describe the special behavior of these MPCs.
So all that sounds interesting, but it hasn't really told you much about what they do. I've thought of different ways to describe them to my coworkers, and the best I've come up with is this analogy. Basically, you're including a group of grids in a type of government, and you're letting them know what kind of leader they have. So what's an RBE2? An RBE2 is a monarchy, where the control grid is king, and tells the other grids what to do. An RBE3, on the other hand, is more like a democracy, where the President does what the people tell it to do. So what works and what doesn't work when we try to make use of these MPCs?
What Works
The above works because the king can tell the dependent nodes what to do. There are constraints on grids, but not on the same ones as the RBE2.
The above also works because the dependent grid knows what to do because the nodes that vote for it at least know what they want; if the elements beneath them were unconstrained then the whole set of elements would not know what to do.
This one works because the voters still know what they want: the constraints tell them what they want! The RBE2 above the RBE3 also knows what to do because the president is telling it what to do. Ok, My analogy is getting old. I'll switch to talking about independent/dependent grids.
As you can see above, I lied, the deputies are out! Through the use of UM grids the RBE3 is not the dependent element, and can now be told what to do.
The above also works, as the UM grids allow the top RBE3 to not have its dependent grid be the same as the bottom RBE3 dependent grid.
Finally, the above works because the top RBE2 is only connected to independent grids.
You can download the above examples at RBEThingsYouCanDo.bdf
What Doesn't Work
Well this won't work, the RBE2 is trying to tell the dependent grids what to do, and the constraints are doing the same thing.
The above also doesn't work. There are nodes which are dependent nodes in two separate RBE2's.
The above also has problems. The UM grids have been set, making them no longer independent, but those grids also have constraints, so again we have grids where MPCs and constraints are both trying to tell a grid what to do.
Finally, the above is also bad. You can't make a grid the dependent grid of more than one RBE3.
You can download the examples of things that don't work at RBEThingsYouCantDo.BDF
There are fun tricks you can do once you're feeling comfortable with RBE3s. The two most interesting are:
-Applying load without adding stiffness. If you have a certain total load that needs to be applied across a cross section of a model, you can use an RBE3 to apply it. This is particularly useful at the ends of a beam.
-Constraining the overall motion of a larger body. If it's known that something is sitting on a very soft support, such as rubber or water, an RBE3 can be created across the supported region, and through the setting of the UM card you can constrain the former dependent grid to control the overall average motion of the model.
There is, of course, something that analysts commonly do when they need to make a part of their model very stiff but don't have the time or experience to construct MPCs without errors: create elements with very very stiff properties that will behave similarly to MPCs. The upside to this approach is that it will typically yield a very similar answer, the downside is that it may take a few iterations to find a stiffness that is stiff enough, yet not so stiff the stiffness matrix becomes ill-formed.
So wouldn't it be great if there was an MPC that automatically made a very stiff element that was just stiff enough? Something that automatically created a perfectly rigid body but then had stiffness that connected it to the dependent grids? Well there is in fact an element that does this. This type of element is typically called a Lagrangian element, and it can be selected in certain load cases in Nastran and is the behavior for Kinematic (RBE2 style) and distributing (RBE3 style) elements in Abaqus. I'll cover this topic in more detail in a later post.
Update 28 Sep 2021: Due to a Google security update the example file links expired, should be working again.
Subscribe to:
Posts (Atom)