Synthetic, or real-world temperature testing?

[Arcmanov]

While reading through some HSF reviews on FrostyTech, I came across their testing methodology.

While I understand and appreciate their reasoning for applying their methods as trying to "remove the variations in heat loads generated by a CPU",
there is really no substitute for real-world, on-CPU testing.  Consequently, I am not prepared to accept their test results and 'Top 5' listings as a true representation
of HSF performance.  I need more 'meat on da grill' so to speak, when it comes to PC component testing, especially when it comes to CPU/HSF thermal
comparisons. 

It's not a coincidence that the majority of HSF reviews are conducted using actual PC components, running Windows, showing the effects
of overclocking and system/OS performance, versus the rise in CPU temps.

This is just my personal opinion, but sound off techies.  What do you think?

Synthetic, or real-world testing?

[Philosophical45]

*just to put my n00b 2cents* hmm to be fair.. i would say both. The cooling test methodology on frostytech i must say is innovative at least. They can tell whether the cooler was made to handle the most extreme of temps or just built for the average expected temperature. Real world testing would be alot more relevant to the end user, the one that just wants to know how good this cpu cooler can cool my Q6600 or something like that. But the disadvantages lie in the fact that not all end users keep their cpu's at stock speeds. Some heatsinks may handle cpu's at stock speeds exceptionally well, but may totally suck at non-stock/oc'd speed. So to be fair, i can't choose one over the other, as one may have advantages over the other and vice versa.

[Spazosaurus]

You really have to ask yourself...what is the difference between synthetic and real world testing in this context? I consider their method more valid than testing a cpu under actual load simply because of the fact that a great many of the variables are removed.

Consider this in your typical "real world" setup...

First you have the chip that interprets the temperatures (usually some type of Winbond chip that may or may not measure temps accurately.) Then you have the actual sensor itself, both of which may or may not give an accurate readout. Compounding this is the fact that each model mobo carries different sensors and different monitoring ic's and its a fact that different programs can give you a different readout even while obtaining their info from the same chip.

Then there's the matter of these sensors measuring only Tcase temperatures and not Tjunction. Which is more accurate or which one is the better value to go by...of course not forgetting that is assuming that the correct temperature is being measured.

Temperatures are reported by your OS/bios are a good guide as to what's happening for the average user but for the real hardcore crowd out there I think they look towards a solution similar to that of frosty tech's.

In the case of the Xigmatek cooler though, I don't doubt that its performance surpasses the majority of solutions out there because of its direct contact approach. I mean, the logic of it just makes sense. You have the copper heat pipes coming into direct contact with the CPU cutting out the "middle man" (usually aluminum which makes first contact and conducts heat only to pass it on to the copper.) Switching the first point of contact from the less conductive aluminum to the higher conducting copper is...well...that's standard 5 science. While the results may not be completely accurate, it serves as a good guide...which brings me to another point...

They have an established method of measuring temps. They use the same method of testing for everything. That means that there is a standard. Now even though this standard MAY achieve incorrect results, it still serves as a valid method of comparative testing in relative terms.

[Arcmanov]

Duly noted.  Especially the Tcase/junction issue.  I guess this can be resolved by using an actual temperature probe between the CPU
and the HSF base.

Different CPUs have different power and thermal envelopes.  How can FrostyTech's setup accurately replicate that if, as Philo45 said,
I just want to know if a certain HSF will cool my CPU better than another?  Based on FrostyTech's results, am I now to believe that
I 'wasted' my money on my current HSF, when the majority of reviews elsewhere seem to indicate otherwise?

I am sorry, but I just can't accept that. 

[W1nTry]

I am sure this debate will go on for some time, however as a first hand user of the Xigmatek HSF, I have already seen a nice decrease in temperature over stock. I had to remove the side intake fan yet I have seen my ambient/idle core temp drop from mid-high 40's to low 30's and my temp at full load has dropped from 58 to 50. So I can say I have seen on avg at least a 10 degree drop if not more. I am positive better results would have been seen if I didn't mess with the HSF orientation so much. I misjudged the space needed when I first attempted to put it on and as a result actually had to REMOVE it altogether. I did NOT reapply thermal compound I just left it as it smudges and all (not best practice I Know but I was a bit lazy). Then of course there was the the swivlling of the HSF WHILST it was ON THE CPU as I was moving my RAM to the 2ndary slots as the HSF was so large it was over the the 1st ram slot and I didn't want to have issues should I ever have to replace a module. So the long and short is, when applying the HSF I possibly broke most of the DO NOT's and it still performs excellently. Also its QUIET. The 120mm fan rotates @ 1400RPM TOPS and you can't hear it. I would have to say Frostytech in the LEAST pointed me in the right direction.

[daniboy79]

interesting test rig... my only concerns right now are;

how do they factor in the lack of a 'real' environment? that rig is probably gonna sit on a nice desk in an Air Conditioned room rather than in a relatively closed PC case, and no other source of heat (Hard Drives, RAM modules, PSU, etc.). And don't even get me started on the Humidity, Dry/Wet Bulb effect during tests.

[W1nTry]

I have to say though, it strikes me as arguing symantics by saying that its not in a real rig, etc. If you think about it, EVERY environment you go into, the conditions will vary and as such so will performance. However if you want to test the properties of a substance ergo the copper/aluminium and its implementation, you cannot factor in the environment to see how it performs. Not to mention, with age and wear even your benchmarking platform as 'standard' as it may be will deteriorate, and change. By using a synthetic platform, the conditions can be better regulated and more absolute values attained. This isn't SOFTWARE, it's still HARDWARE, so regardless of the mobo, ram, HD or heat thereof, it WON'T change the specific heat capacity of the HSF or how well the materials involved should conduct. All the surrounding components affect the AMBIENT temperature of the AIR surrounding the HSF and thus its ability to transfer heat to its surrounding. I am sure that's the idealogy used. So whilst I agree that there should be actual IN-RIG tests to support the 'synthetic' benchmark, I can't see it as being a discredit to the absolute values they have recorded.

[daniboy79]

how brilliant!!

[Arcmanov]

I have to say though, it strikes me as arguing symantics by saying that its not in a real rig, etc. If you think about it, EVERY environment you go into, the conditions will vary and as such so will performance. However if you want to test the properties of a substance ergo the copper/aluminium and its implementation, you cannot factor in the environment to see how it performs. Not to mention, with age and wear even your benchmarking platform as 'standard' as it may be will deteriorate, and change. By using a synthetic platform, the conditions can be better regulated and more absolute values attained. This isn't SOFTWARE, it's still HARDWARE, so regardless of the mobo, ram, HD or heat thereof, it WON'T change the specific heat capacity of the HSF or how well the materials involved should conduct. All the surrounding components affect the AMBIENT temperature of the AIR surrounding the HSF and thus its ability to transfer heat to its surrounding. I am sure that's the idealogy used. So whilst I agree that there should be actual IN-RIG tests to support the 'synthetic' benchmark, I can't see it as being a discredit to the absolute values they have recorded.

Very valid argument here, which I am at pains to find fault with.

[Spazosaurus]

I have to say though, it strikes me as arguing symantics by saying that its not in a real rig, etc. If you think about it, EVERY environment you go into, the conditions will vary and as such so will performance. However if you want to test the properties of a substance ergo the copper/aluminium and its implementation, you cannot factor in the environment to see how it performs. Not to mention, with age and wear even your benchmarking platform as 'standard' as it may be will deteriorate, and change. By using a synthetic platform, the conditions can be better regulated and more absolute values attained. This isn't SOFTWARE, it's still HARDWARE, so regardless of the mobo, ram, HD or heat thereof, it WON'T change the specific heat capacity of the HSF or how well the materials involved should conduct. All the surrounding components affect the AMBIENT temperature of the AIR surrounding the HSF and thus its ability to transfer heat to its surrounding. I am sure that's the idealogy used. So whilst I agree that there should be actual IN-RIG tests to support the 'synthetic' benchmark, I can't see it as being a discredit to the absolute values they have recorded.

I'm pretty sure this is the general idea I put forward as well...

[W1nTry]

I'm pretty sure this is the general idea I put forward as well...

Ahh but not in so few words XD

[Jdcrys_Shinkuu]

my thoughts are more simplistic in nature...
treat this scenario like the dyno testing of a car. the dyno test is a controlled environment showing the true if u will performance of the car while the on the road test will prove the actual performance of the car. remembering that every scenario will be different like every road will be different based on a lot of factors. so while i appreciate the controlled test (as what the product should operate at) it serves as a perfect guide as to what u should have it run like. prime example office servers are put into an airconditioned room to maintain the recommended temperature. this recommendation came from some one who saw the optimal performance in a controlled environment. so yes i prefer the synthetic