The dyno can be a very important tool for the performance enthusiast. It has the ability to report what your car is putting to the ground over the whole RPM range. However, some people and manufacturers ďtwistĒ dyno results for their purposes. They can do this because there is quite a bit of variability from one dyno run to the next even with the same car in the same configuration and even taken just minutes apart.
There are many reasons for this. The first is that the dyno must be periodically calibrated. If it isnít, then you donít know what you are getting. In addition, environmental conditions affect the results of both the car and the dyno. This is why it is critical to know the temperature, humidity, altitude, etc. of your location. If you donít, then you run the risk of being way off in your numbers. Perhaps the biggest effect is your carís variation over temperature. For example, people usually do 2 to 3 dyno runs at a time. Take a look at the dyno runs of this BMW M3 in the Dynoperformance data base. There was a huge difference of 15 HP at 6250 RPM between its first and third run. I remember when I had gone to dyno an automatic 98 vette (LS1 power pack and corsa exhaust mods). I did three pulls with run1 = 285 RWHP, run2 = 297 RWHP, run3 = 301 RWHP. Why is there so much variation? The car needs to be warmed up correctly and running in its optimum configuration. Each car runs differently at different temperatures, and it is very important to run the car in the same configuration every time you do a dyno pull (and at its optimum temp if you are looking for peak power capability).
If the dyno is well calibrated, it does not lie. The three runs of that 98 vette did produce those numbers. It is technically not a lie to say that it pulls 285 RWHP, because during that run it did. However, letís say I had done only one run with that car in stock configuration and got 275 RWHP after waiting for a few hours on that dyno day for my turn (so the car was cold). Then I drove away and put exhaust on and went back to the dyno and did a pull as soon as I got there with the car nice and warm. Now I pull 290 RWHP. Did I really gain 15 HP from the exhaust? No. But now you can see how easy it is for a manufacturer to make some huge claim for their product. Most people will not dyno their car enough times before and after a mod to remove this variation.
On top of this variation, there is a statistical variation that occurs with the car even under the same conditions. For example, letís say a car gives its peak power at a particular coolant temp. If I do X runs at that temp with everything else the same, I will see a statistical variation from run to run. What that variation is depends on what statistical theory you apply. This is why it is very important to do many dyno runs and average the runs to get a real idea of what is happening. If your deviation from run to run is 3 HP, then there is no way that you can statistically differentiate, with a single dyno run, a gain of 3 HP. So, if I had a nice baseline of 10 dyno runs that produce an average of 300 RWHP with a deviation of 3 HP and I go and install high performance spark plugs and pull 303 with a single dyno run, I have not proven there was any gain from those plugs. However, if I do 10 runs after the modification with an average of 303 and a 3 HP deviation, then I can conclude that there was a gain from the plugs.
There are other variations also, but most of them take care of themselves if you make sure you get a decent number of runs before and after a mod and you know the configuration of the car and the environmental conditions. The bottom line is be careful when looking at peopleís dyno runs and be suspicious of small gains that a manufacturer may claim.