4500 DRAMs stuff up a 1U serverMetaRAM makes it possibleBy INQUIRER news desk: Tuesday, 26 February 2008, 7:19 AMIF YOU WANT high memory density in modern servers, you have had two choices, high prices or praying that your machine could take high rank count DIMMs. MetaRAM, a startup, has just come out with a thing it calls MetaSDRAM.The problem is pretty basic, volume drives down cost in the PC space, and that means the popular parts are cheap, higher or lower end parts get very expensive. Right now, 1GB density DDR2 chips are somewhat in vogue, so they are cheap. Higher, and to an extent lower, density chips do not have the volume, and are inordinately expensive.I you want to make high memory size DIMMs, say 4GB and above, you have two options, use a lot of RAM chips or use fewer higher density chips. The higher density chips are what you want to do, but that gets really expensive.Lower density chips have a lot more problems, some of which can get expensive as well. You need lots of them, so you have to make a very tall DIMM that won't fit in a lot of places or use exotic (read: expensive) packaging technologies. Either way, there are a lot of them on the DIMM, so the rank count increases. Since the number of ranks is basically what limits how many DIMMs a controller can see, you are stuck with a very small number of DIMMs of high DRAM count. This tends to be no cheaper than a bunch of lower density DIMMs in practice.What you need is a high density DIMM with a low rank count that uses commodity DRAMs. With the new MetaSDRAM products, that is exactly what MetaRAM is providing. It offers eight and 16GB DDR2 DIMMs in a two rank configuration at speeds up to DDR2/667. Actually, it doesn't make DIMMs, just two chips that allow you to make them. The end result looks like this.MetaRAM 8GB DIMMWhat you see here is a DIMM made with the MetaRAM MR08G2 chipset. This consists of two types of chips, the big square one in the center is the AM150 Access Manager coupled to five, four visible, rectangular FC540 Flow Controllers. They make an 8GB 2 rank DIMM out of 1Gb chips. There is also a 16GB DIMM kit called the MR16G2, it consists of two AM160 Access Managers and nine FC540s.The way the chips work their magic is to emulate a 4Gb DRAM with 4 x 1Gb DRAMs. It sounds simple enough, but the trick is in the doing. Sending someone to the moon is quite simple to describe as well, but harder to do in practice.The problem is pretty basic, timing. DRAM has two types of signalling, address and data, and both are independent. When you read an address, the memory controller sends an address to the DIMM and then a few cycles later, gets the resultant data back. That is simple. On a write, you have an address sent and the data along with it, put data X in location Y.The MetaRAM chips add two cycles of latency, but speed up other things, so the end result is CAS goes from 5 on standard rDDR2 DIMMs to 6. Back to the problem, you have to juggle a ton of signals, and do so with high accuracy. MetaRAM claims that their solution is provably correct, and that it is plug and play compatible with any lower density registered ECC DDR2 DIMM.What you end up with is a bit slower memory, probably masked by cache coherency latency on Operons, less so on San Clemente Core Number Numerals, that takes a few percent performance off the top. If you consider how much a disk swap costs, a few percent is a tiny price to pay if you can avoid that fate on a server.In the end, the question you want to ask is how much this costs. MetaRAM sells the 8GB kit for $200 in 1000 unit quantities and the 16GB kit for $450. The eight is available in products now, the 16 is qualified for production, so think for a few months.You can get MetaRAM modules now from Hynix and very soon from Smart Modular (SMT), with SMT qualification parts costing $1500 for an 8G. Appro, Colfax, Rackable and Verari will all sell you servers with 8GB DIMMs on board now as well.If you stop and think about the memory density this offers, it is nothing short of amazing. On a 4S Opteron box, you can stick 64 DIMMs in for 256GB of memory, 512 when the 16GB sticks come out. On an 8S box, you are talking a TB of main memory! If you do the sums, a 4S 1U box with 32 DIMMs will have 144 DRAMs in it. 144 * 32 is 4608, over four thousand DRAMs in a pizza box. Wow.A quick trip to Colfax allows you to config a 4S Opteron box with the highest speed MP chip they offer, the 8222, and 256G of memory for a hair under $50K, ($49,986 with a single 500G SATA drive). If that doesn't impress you, I am not sure anything will.The MetaRAM thing is basically the end of the road for DDR2, the spec only allows for 4G density DIMMs. Since MetaSDRAM can emulate that now with commodity 1Gb RAM chips, there is nowhere left to go but DDR3. They claim to have a DDR3 solution in the works but would not comment on it.The last question is about buyers. Who is going to pay for these things? There are a lot of markets where RAM is the only limiting factor, VMWare for example. VMs tend to be limited by main memory but almost never by CPU power. If you wanted twice the VMs, you can get double the boxes or double the memory.Most VMWare deployments are installed with all memory slots full, so doubling the memory is right out. This means you double the box count, double the CPU count, double the power usage, and double the management overhead.With MetaRAM, you can double, or possibly quadruple, the memory count, leading to huge savings in secondary costs. Will it be enough to pay for the MetaRAM chip costs? Almost assuredly. For markets like CAD, visualization and digital content creation, the results may be far less clear cut.In the end, MetaRAM has a saleable product that does what it says, makes high density, low rank count DIMMs. A single 8GB MetaRAM DIMM costs more than 2 4G DIMMs but fits in a single slot. It also costs much less than a similar solution made from high density DRAMs. All in all, you take a slight speed hit, but other than that, there are almost no down sides.
