Nvidia's bad bump misery deepensBumpgate Insurers bite backBy Charlie DemerjianTuesday, 19 May 2009, 03:59A DOCUMENT HAS COME TO LIGHT that details the lengths to which Nvidia has gone to cover up the problems it has been having with its graphics chips.The most recent lawsuit (PDF) against it by the National Union Fire Insurance Company (NUFI) claims the company has withheld information on the nature of its bad bumps. The very same information it has withheld from us or any other nosy hack or awkward analysts.The story was broken by a certain Mike Magee at TG Daily on Friday, and it has a lot of juicy bits. The short story is that the list of defective chips shipped by Nvidia goes back to the NV4x generation, and the list of OEMs affected counts ten and basically includes every Nvidia customer.NUFI complains bitterly that Nvidia has been covering up essential information it is entitled to receive as Nvidia's insurer by refusing to disclose even the most basic facts about the company's GPU chip failures.We had the same complaint. Let's go back over what happened so you can see the depths of this debacle.On July 2, 2008, Nvidia issued an 8-K report that essentially said: "We have a big problem." But then it got really vague. Its 8-K mentioned devices "in certain versions of our previous generation MCP and GPU products used in notebook systems." Nvidia also claimed that it didn't know why the problems were happening, saying, "We have not been able to determine a root cause for these failures." But was sure that it had fixed them. No, really, it said so quite explicitly. "All newly manufactured products and all products currently shipping in volume have a different and more robust material set," it stated.We said at the time that this was cobblers. Nvidia was blaming everyone but itself. Nvidia was not saying which OEMs were affected, and privately it was telling financial analysts that there was only one OEM, HP, that had problems. We said then that it was a design problem and unlike Nvidia's crack team (pun fully intended) of failure finders, we told you precisely what it was, that is, high lead "bad" bumps cracking between the die and the substrate.We also told you that there was more than one OEM was affected, starting with Dell, then HP, then Apple, and lastly Toshiba. If you read the Apple link, we mentioned it, but it looks like Toshiba never got the guts to do right by its end-users. The ten OEMs listed by NUFI in its complaint are Dell, Toshiba, Apple, HP, Quanta, Wistron, Compal, Asus, Samsung, and Fujitsu-Siemens, "and others." That list adds up to at least ten times the number of affected OEMs that Nvidia had been originally spinning.The claim all along from the company has been that it still doesn't know what is wrong, and it has no way of figuring anything out. That didn't prevent Nvidia from coming up with the dollar amount it would cost to fix the problem, with great precision, but it couldn't figure the problem out.So now, the defective chip list has expanded to the G86, G86A2, G84, C51, G72, G72M, G73, G72A3, MCP67 and NV42 officially, but those are only the ones that are named in the lawsuit. There could be others that haven't cropped up yet, or that had problems that Nvidia managed to suppress early on.We were first to name the G84 and G86 directly, and then pointed out that others were bad too. Nvidia denied it, loudly. We said there were chipset problems, but Nvidia denied that. We said the 9x line was affected. Nvidia denied, but Apple however confirmed. Nvidia claimed it was only laptops that were affected, in an SEC filing no less. We pointed out that this wasn't the case. Nvidia still denies it.Apple has also confirmed that, and the parts listed in the above links include, desktops. NUFI also names the exact chipset that we said was bad in our article as well. Nvidia would say we must have just gotten lucky and guessed right.Everything we claimed and more has been confirmed, and more models have been added. We also claimed that desktop cards, and newer 55nm models were defective too, but those don't show up on the list. We guessthat's just a matter of time.Nvidia's chip design problem is a long-term one, and it takes months or years to start showing up. The technical explanation is in a three-part article, here, here and here. It may be long, but it is the only place you find an expanation. Meanwhile, Nvidia still claims it doesn't understand it.We find this doubtful, mainly because of two things. First, Nvidia claimed to have fixed the problem in the 8-K filing mentioned above, and identified a specific fix. "All newly manufactured products and all products currently shipping in volume have a different and more robust material set." That sounds like it was pretty conclusive. The problem is that Nvidia's claim in that SEC filing just wasn't an accurate reflection of reality.If you look at the three PCNs we were shown, written up here, here and here, they clearly show the first ship dates for the supposedly 'fixed' products as July 25, August 17 and August 17. Those dates are all after the SEC filing, and after the claim was made. Maybe there is a way to spin "first ship date", but it seems pretty clear damning, and it does not seem to align with the statement, "All products currently shipping in volume". Maybe the SEC just didn't care, or was overworked, or something more sinister.Back to the story at hand, the insurance lawsuit. Nvidia's prognostications have been comprehensively debunked, its claims have been shown to be untrue, and it still denies the problem. It refuses to come clean, say what chips are affected, what models of computers they were in, and who they were sold to.When we first asked it, Nvidia claimed that it wanted to protect its customers, that is, the big OEMs, so it wouldn't say a thing. The people who buy Nvidia cards? Well, that was their own problem, as Nvidia was more worried about OEM feedback than end-user harm.This tale really starts to spin down the rabbit hole when you consider that Nvidia has been using the same line on the insurance company. According to the complaint, Nvidia has been asking NUFI to pay up, but has not provided it with any information. You really have to read this to believe it, from the suit (Page 2, lines 12-19):"Most importantly, NVIDIA has failed and even refused to provide material information about the Chip Claims to National Union, despite repeated and specific requests by National Union for that information. Instead, NVIDIA has provided substantial information about the GPUs themselves, held meetings to discuss the GPUs, and flooded National Union with technical data. That information, however, does not contain basic information about the Chip Claims that would allow National Union to evaluate whether any settlements are reasonable, whether coverage exists for those settlements, whether other parties may be at fault, or even what precise injuries the settlements are compensating the Chip Claimants."Other than the part about flooding with technical data, this is exactly what Nvidia did to us humble hacks here at The INQUIRER. Lots of words, much chest thumping about its greatness and infallibility, but never answering the question.The questions that Nvidia will not answer are not at all out of line. To quote Page 12 Para 68:"The information that NVIDIA has failed and/or refused to provide to National Union includes, but is not limited to, the following Material Information:- Records showing the dates of manufacture of all affected notebook computers;- Records showing the dates notebooks were shipped to end users;- Records showing field failures to date, and the specific dates of those failures;- Records showing the specific dates of repair of the affected notebooks;- Records showing what component parts were replaced and when and why;- Records showing any injury to component parts other than NVIDIA chips, including descriptions and dates of injuriers; and- Documentation of settlement discussions, NVIDIA's estimation of claim exposure, and supporting documentation of any estimate."Do these sound like things that are reasonable to ask before an insurance company pays out?Reading the fine print in the suit, you can see there are several good reasons why Nvidia is playing dumb. The first is the civil lawsuits, if Nvidia admits anything to anyone, it will surely have a harder time defending against those civil lawsuits. You could say that what is going on is obvious to even a slow monkey, and we knew over a year before that some Nvidia GPU parts that were out were problematic.Luckily for Nvidia, it is still in the dark, claiming that the science behind the cracking bumps is not understood. Somehow, every packaging expert talked to by The INQUIRER understood it, was able to explain it, and the explanations all matched up. Nvidia might want to hire one of these guys in the future to fill the glaring hole in its corporate semiconductor knowledge base.Secondly, NUFI is claiming that Nvidia did not involve it in the negotiations for which it is being asked to pay Nvidia's settlements. Nvidia's insurance policy states that NUFI must be involved in any such negotiations, or at least that it must be given the opportunity to decline to do so.It looks like Nvidia didn't provide NUFI with details, then sent it a bill with a big number on it. NUFI wisely declined to pay.Things get even thornier on the next bit, on Page 25, Paragraph m(1). That says that property damage from "A defect, deficiency, inadequacy or dangerous condition in "your product" or "your work"" does not qualify for coverage under the policy. That sounds like defective chips might not be covered because they are defective, not because they broke for some unknown mysterious reason that Nvidia won't say. In the end, the whole sad story backs up what we have been saying since the early days of this whole Nvidia "bad bumps" defective chips fiasco.Nvidia is knowingly covering things up, denying information to the people who need it, and stonewalling everything. The list of affected parts has grown longer and longer, the list of affected OEMs by now basically comprises the list of all of Nvidia's OEMs, and now it looks like the company's hope of an insurance payout are under threat.The only common thread is that Nvidia simply won't tell the truth no matter what. Every public statement it has made, from sworn SEC documents to off -the-record claims, has been disproven, but still it carries on with its version of events, almost as though its reality trumps all others by force of will.While this story is fairly well fleshed out now, the ability of Nvidia to dig its own grave deeper seems to be never ending. The allegations in the NUFI lawsuit are so farcical at times they stretch the ability to believe, unless you have been dealing with Nvidia before for a while. Then you'll recognise it as business as usual. µNote NUFI/AIG was contacted twice for comments on this story. We have not heard back from them either time, but will update or follow up should they respond.
Nvidia is out at DellBumpgate Bumps and paybackBy Charlie DemerjianTuesday, 26 May 2009, 05:02FOR A LONG TIME we have wondered when Nvidia's abject stupidity would have a price. The answer, at least at Dell, is now.If you go over the Dell desktop lineup and look through the customization options, you will see that, with a few minor exceptions, there are no more Nvidia cards being offered in Dell systems. Whoopsie. I guess the $10M that Nvidia paid Dell wasn't enough to make amends.The lone remaining desktop Dells with Nvidia graphics are in the high end XPS line and some All-In-One (AIW) models. The AIWs and laptops have a lot in common. They have longer design cycles, and Dell can't just swap out graphics parts when things go pear shaped with a particular supplier. What the systems were designed for years earlier is what they have to be shipped with, and what those models will stay with until they are replaced.On the non-AIW desktop side, there isn't a single Nvidia card in a Dell machine that I could find other than in the XPS line. Gone, poof, almost overnight. The high-end gaming boxes are where Nvidia theoretically should shine. Its image hasn't been as tarnished among the fanboi set to the degree it has elsewhere.That said, there are three lines of XPS machines, the 625, the 630, and the 730, each with four sub-models. The 625 line is entirely devoid of Nvidia cards, but Dell offers multiple ATI graphics card configurations, including several versions of CrossfireX. One of the four 630 models has no Nvidia graphics option, two of the remaining ones have more ATI choices than Nvidia options, and the fourth has two from each vendor.On the top of the heap in the 730 models, where Nvidia has traditionally dominated, one 730 model has no Nvidia option at all while the others offer cards from both graphics vendors about equally. Nvidia is hanging on by its fingernails on the high end, likely because of gaming fanbois more than anything else.This is nothing less than a sea change at Dell. Nvidia has basically been shown the door by Dell in a most unceremonious fashion. Nvidia either decided to stop buying market share, or Dell just got fed up with it, but don't preclude both.From what we hear, Dell's laptops and AIWs aren't far behind either. The decisions at Dell and elsewhere are all starting to sound like a broken record. Look for other OEMs to follow suit shortly. The decisions for back to school models get made in the next four to six weeks, and after that point it is basically game over for the year.Nvidia is looking at openly hostile OEM customers and trying to woo them without any DX11 parts. ATI will have DX11 parts in the pipeline very shortly, and it is being greeted with open arms. Payback is a bitch. µ
Nvidia offers low-end laptops as replacements for Bumpgate victimsCries of bait and switch ring outBy Lawrence LatifTue May 03 2011, 16:20CHIP DESIGNER Nvidia has managed to get away with giving Compaq CQ-56 laptops as replacements to those bumpgate victims involved in the class action lawsuit against it.Last year Nvidia came to a settlement in which it agreed that it would repair or replace computers that were affected by bumpgate, where some of its GPUs overheated and failed due to semiconductor packaging defects.However, later the firm found that it was unable to repair some of the machines. In those cases it offered a replacement laptop, but not any laptop, a gleaming Compaq CQ-56 for each of those machines it couldn't repair.Compaq's CQ-56 laptop comes in a number of configurations but high-end it isn't, not by far. One specification of the machine retails at Comet for less than £280.Not surprisingly some of those involved in the class action settlement complained that Nvidia had essentially pulled the old bait and switch trick on them. Sadly for them, US District Court Judge James Ware overruled their objections, writing that they were "without merit".Outlining the reasons for his ruling, Judge Ware wrote, "the [Compaq] CQ-56 meets or exceeds nearly all of the specifications of the original computers", adding that "it comes with an advanced operating system, new warranty and other programs". As for missing peripherals, Judge Ware said, "the Court finds that they can be easily and inexpensively added".Remember that this is the laptop that will be offered to users including those with Apple Macbook Pros. It's not hard to see why those who had shelled out the best part of £2,000 on an Apple notebook might be somewhat upset to end up receiving a low-priced Compaq laptop, even though the judge apparently thinks it "meets or exceeds the specifications of the original computers".Then there is the question whether Apple Macbook Pro owners should be forced to accept a 'similar specification machine' that can't, legally or operationally, run the same operating system as their original computer.It's a poor show from Nvidia, which could have scored some points if it had offered those affected a decent replacement, perhaps with some games thrown in for good will and to show off the graphics capabilities of the machines. Instead, this whole sad story makes the company look like it is short-changing customers who ended up with its faulty equipment.If Nvidia had hoped that its legal settlement of the bumpgate fiasco would be the end of the story, its decision not to offer like-for-like replacements could end up costing it further damage to its already bruised reputation. µRead more: http://www.theinquirer.net/inquirer/news/2047089/nvidia-offers-low-laptop-replacement-bumpgate-victims#ixzz1LJPjitWiThe Inquirer - Computer hardware news and downloads. Visit the download store today.
