GATT Benchmarking Guide (PLEASE READ but DO NOT POST RESULTS HERE)

[W1nTry]

Since we have a rather enthusiastic group of members (at least in the hardware dept) it came to my attention that the 'old' benchmark ranking system has become antiquated for lack of a better word. In the very diverse world of Hardware we have today where choice is KING, its time we introduced a PROPER system to rank the best of the best here in GATT. I have already bounced the idea off another admins head and he seems quite receptive. Basically we intend to introduce catergories to benchmarking so that everyone has a decent shot at holding a top ranked spot. Just to give you an idea of what I mean, lets first break down what essentially makes up a PC today:

1. CPU - The world runs predominantly on X86 architecture ala AMD or Intel (maybe even Via). There are other types of CPUs (RISC) ala Alpha, Itanium, PPC, etc, however they are a minimum and totally if not virtually non-existent in GATT. From here there are single core, dual core, quad core. There is also 64-bit, 32-bit and God forbid someone has an 8-16 bit still running somewhere... Then there is Power consumption, die sizes and the age of the CPU. Now it'd be quite ubsurd to facilitate all these catergories thus we will take those that best reflect the diversity on GATT. SO for instance so far i've planned to have Brand, age and perhaps power consumption into the consideration for catergories for Benchmarking.

2. RAM - This is not really a part (though crucial) that would have a great impact on the catergories for benchmarks... basically we have SD-RAM, DDR-RAM, DDR2-RAM, RD-RAM, and a few others. However unless someone can come up with a really solid justification as to how this will differentiate a PC then it remains out of consideration.

3. Motherboard - There is an even greater diversity in brand here than most other components, with a HUGE mix of chips in the play. Thus this particular part along with CPU and Video will help determine what AGE catergory a PC falls into.

4. Video -  This is an extremely important player in the Benchmark arena. Since most Benchmarks are graphical in nature... (at least those on GATT, since its gamers we're interested in) this will help determine age, brand, POWER and overrall rank in GATT.

5. Sound - often neglected and unfortunately doomed to stay this way as well... at least until Beo or the likes intros some nice audio benchmarking s/w to add to the fray to give it a unique catergory

6. PSU - This will not directly affect benchmarks, but rather is a support component for achieving your benchmarks... as such it will be factored into power but otherwise minimal.

7. HDs - Minimal impact on most benchies... thus perhaps it will aid in determining age.

8. Optical and all other peripherals - by themselves they mean nothing...put it altogether and yuh just may have something... like a really encompassing gamig experience (read - sound + controllers/steering wheel/racing seat/gam mat/etc). The will be a catergory for sheer innovation or miscellaneous excessiveness...

Now that we've more or less gone over all the parts and how they determine how and where a rig falls.. let me give an idea into the catergories for the Benchmarking Forum:

TOP DOG - The FLAT OUTRIGHT MOST WTFPWNAGE machine in all of GATTDOM BAR NONE......

Stuck in the middle - The mainstream system that JUST couldn't make it into the BIG BOYS TOYS catergory

Lemmings - Intergrated/budget builds that can play games....mostly

Best in Breed - Best AMD/Intel/ATI/Nvida Platform Rig

Best in Show - Best looking gaming rig

The light bulb - most innovative rig

Golden oldies - Rigs >= 3/4 yrs old that still hold a candle to the wind against their succesors

PWNED - The outright slowest, crappiest, lamest, WTFPWNEDBYEVERYONE rig on GATT (cannot belong to golden oldies catergory).

There will be additional rules obviously to help discern what rig goes where, and where a rig can belong to multple catergories and there is even talks on a GATT t-shirt or something to that effect to the TOP DOG or perhaps 1st place in each catergory... at least once a year. Also there will be a LONG list of requirements to be considered for ranking. This will include LOTS of screen shots and pics of the rig in its entirty. Also it has be be owned by the member or the member's  household to be elligible. I will update this thread with the full rules, rank system, guidelines, requirements etc as time goes along. I hope this is not TOO much for the enthusiasts in GATT. So prepare ur rigs gentlefolk...

Please remember: When running synthetic benchmarking applications like 3dmark and pcmark, ALL settings are to remain at default. Wherever possible, arrange your screenshots in such a way that as much information as possible is available for scrutiny. You are free to run your own custom benchmarks at custom settings and place screenies in your personal benchmark thread, however for the purposes of the GATT Members' Top Rigs board, please adhere to the default settings in order to maintain a level playing field for all competitors.

[Arcmanov]

Wonderful ideas here.

Also, we need to standardise what programs are used for benchmarking, how they should be run,
and most importantly the format for submission of results, so it can be easy to display em in tabluar
form perhaps?

Lets get it started.
I have TWO rig just snapping at their leashes............

[.:Jedi:.]

sounds great.....and just when im on my new build. check it out in the hardware & tweaking board.

[Nephilim]

we need to standardise what programs are used for benchmarking, how they should be run,
and most importantly the format for submission of results, so it can be easy to display em in tabluar
form perhaps?

Might i suggest the defaults on 3dmark06? This way we can get a standard across the board...i know that some of you all will be like 'WTF who does use default?!?'
And within each de/vision we can have specific adjustments. e.g. high end run at certain setting such as to really push the rig. This way we can see how rigs in specific categories measure up, a controlled variable if you will...

[Eroo]

3dmark05 and 3dmark 06 I think should be the defaults.

[TriniXaeno]

a brilliant idea.

lol @ lemmings.

I just know I'm gonna dominate that "Stuck in the Middle" category.

Couldn't my realm be given a better name? How about "Price/Performance King" lol

Second 3dmark 2005 and 2006 as the standard. They separate CPU and GPU scores and are pretty much the gamers standard the world over. While there are arguably better and more intense benchmark tools on the scene, 3Dmark can't be beat for it's simplicity.

The free version at defaults should be a solid indicator of a rigs gaming prowess.

Thoughts?

[W1nTry]

Well i'm still to flesh out the final specs... but as for standardizing on the apps....
The staples of course are:

3DMark05/06 I believe in the interest of older rigs we include at least 01SE as well. Cause lets face it, rigs with Gefore3 and older cyah compete in the 05/06 arena and we do have catergories for those, alternatively we could have different benchmarks for different catergories.. and well I do have to admit I like see a RIDICULOUS score in 01 on my rig so I may well stick 01/05/06 as the defacto 3DMarks.
Default settings for everything (although I beleive 1280x1024 should be the standard res, but hey we'll leave it stock)

PCMark05

Sisoftware Sandra - memory bandwidth test & potentially the overall performance matrix against a STANDARD system in the mid-performance region as the defacto

CrystalMark - latest revision - System test but it shows a nice breakdown of the various parts (ALU, FPU, Mem, etc)

SuperPI - Standard, there are alot of optimized versions for SSE, SSE2, SSE3, MMX, etc... we go use vanilla

Now we have to mention proofing. Man cyah be using online benchmark results to fool pplz... so the staples:
CPU-Z: CPU, Mem

We could use Everest for system info, but we could just as easily use Sisoftware Sandra and since we doh wanna have a million downloads I say Sandra for system details

Screen shot of your 'My Computer' Properties (allowed to blank off serial # details) NOT name of rig

temperature screenshots are also welcome (esp is overclocking). Overclocking will have to be a catergory by itself... or rather there will be a special place for the most effective OC (I see Beo winning that hands down... witha 4GHz OC....) But we will want default benchmarks (AKA RIG ON STOCK) + OCed Benchmarks. I think it best to also have a BEST STOCK RIG catergory... as for top place in each catergory, IF your rig is OCed, there will need more PROOF that the system is stable and thus the benchmarks are credible... name Prime95. Prolly look at a 12hr Prime95 torture test, or even 24hrs, please provide feedback on this matter.

So this is an update... of the apps so far and the conditions/proof requirements... also 2 catergories added - Best Stock (best NONE OCed rig) AND Nitrous Methane (Greatest improvement in performance due to OCing).

[TriniXaeno]

* NoobGoneWild head explodes from all those apps.

* NoobGoneWild prays for a simple and sweet process

[W1nTry]

* NoobGoneWild head explodes from all those apps.

* NoobGoneWild prays for a simple and sweet process

LMAO, i'll simplify fear not, however it will be STRINGENT!!!

[syncrod]

SuperPi and wPrime runs
http://www.hwbot.org/user.do?userId=8297

[UltimateGamer]

This is an excellent idea, can't wait, its about time the gloves came off.

Thus far it seems like arcman is going to be king

[TriniXaeno]

yeah, arcman's rig is pretty intense.

reminds me, I have to buy a lotto ticket.

[W1nTry]

LOL @ remembering to buy a lotto ticket... that aside, I downloaded some more apps last night.. RMAA - sound benchmarking app, povray and a few others that I will evaluate and then recommend for our bench list. I am working on it slowly but surely, but have no fear it will see the light of day soon. I have another suggestion, which is the use od EVEREST as our info collecting app (mobo, card, etc) as opposed to Sis... but i'll weigh the two and see which is preferable. On the other hand, the more forms of verification we have the better imho. As for arc rig being top dog atm... yeah i'll have to agree... but we'll see... never know who has a MONSTER waiting in the wings to put a downer on arc day :p

[W1nTry]

PLEASE READ THIS THREAD/PARTICULAR POST before issueing your results

Alright, i've looked over some benchmark apps (again for the millionth time) and I think we can reduce the list of apps we use to as follows (NGW Brace yuhself, it sounds like reduce but its not that much of a reduction )

For verification (first and foremost of course) screenshots of CPU-Z AND EITHER a screenshot (or several) of PCMark05 or Everest System details page/s:
CPU-Z CPU/Mainboard/Memory
Either or for the following
PCMark05 - There is a system option that generates a webpage with your FULL system specs (you may save this webpage as a pdf and attach it as well)
Everest
Screenshot of your System Properties page (you can sensor out the serial # information)


Video Benchmarks (Screenshots of the results required):
3DMark2001SE (with latest patch 330 I believe). This benchmark will NOT be run @ default, but with the resolution set to 1280x1024, everything else default.
3DMark06 (with latest patch 120 I believe). This benchmark should be run @ default settings

System Benchmark (Screenshots required):
PCMark05
ScienceMark - Choose the comparison to the Dual socket AMD Opteron setup as the comparison
CrystalMark

Part specific Benchmarks (screenshots required):
SiSoftware Sandra - memory bandwidth test/ALU/FPU
HDTach - Use only Long test (32MB) and comparison drives are WD Raptor 74GB & Seagate 250GB SATA (i'll be more specific as soon as I check it back)
Super-PI - you MUST perform the 1M&2M calculations in the LEAST. If you wish to perform ALL feel free.

I would like to add RMAA for the sound benchmark, however this program comes with some pretty lenghty testing procedures and I will add it later on as its a bit involved even for the more enthusiastic gamer. LASTLY it's preferred that each gamer have his/her own Benchmark THREAD with a simple Summary of their results on the OVERALL benchmark page. This way it prevents the main ranking thread from getting too cluttered or LARGE with a million screenshots. ALSO DO NOT POST FULL SIZE SCREEN SHOTS!!! ONLY THUMBNAILS[/color] that connect to the full size image eg. USE IMAGESHACK.

The Benchmark ranking thread will be created soon and I will post my results as the template for the results. Similarly anyone can view my Benchmark page to get an appreciation for how they CAN (just a guide, you do as you like once its logical and easy to follow) be presented in YOUR OWN THREAD.

Also, if your rig is OCed, then we will require additional information for verification:
Screenshots of the following:
Prime95 running for a minimum of 12hrs
Frequency of Vid card OC (CPU-Z takes care of CPU frequencies, mem speed, etc), I suggest ATItool or a capture of the Nvtune or Catalyst Overdrive page.
Also captures of your system Temperature is suggested.

Lastly if anyone has any comments, requests, suggestions on what additional apps could be used or perhaps things they rather NOT use, please post here.

[TriniXaeno]

OMG!

[W1nTry]

OMG!

LMAO, I was waiting on this!!! I figure ppl will be posting requests for how-to and where to get etc soon... O_o

[Saxito Pau]

yeah, you should post a series of links for downloading all this software. I personally have most of them already so I safe.. I won't do any postable benchies pending the acquisition of my video card...

but yes, this is a LOT of stuff to post... wow... *fans NGW*