April 22, 2005The Truth about Linux and WindowsSteve HammI've got a bone to pick with the never-ending stream of studies by tech research outfits comparing Linux to Windows. For starters, it seems like about half of them are paid for by one camp or another. Even when analysts aren't on the payroll, this is really complex stuff—and useful facts are hard come by. And, beyond complexity, some studies just make me scratch my head. For example: a recent one put out by the Yankee Group. I just don’t trust its conclusions.My beef with Yankee started April 1. That’s when I got an advance look at the press release announcing the study, and a phone interview with Laura DiDio, the analyst who wrote the report. My thought was I’d use this report to really delve down into the way such research is done and figure out how credible it is. So I not only questioned DiDio about her methods and her conclusions, but I also asked for the data upon which here conclusions were drawn. The headline on the press release is “Yankee Group Survey Reveals Majority of Enterprises rate Quality and Performance of Windows Equal to or Better than Linux.” DiDio described the survey as “totally independent, web-based, and self selecting.” She went down through her findings and conclusions.Two things bothered me: She had surveyed enterprise and small-and-medium businesses in equal portions and presented the results combined. This struck me as odd, since enterprises use Linux heavily, and SMBs barely touch the stuff. It seems to me that the average of the two isn’t very meaningful. Second thing: The release said the respondents had deployed Windows Server 2003. That left out the opinions of people who had not installed that particular piece of software. Hmmm. Seems like that would tilt things in Microsoft’s favor.DiDio is one of the analysts who seems to have taken sides in the holy war over Windows and Linux. She published a “White Paper” on the topic last year that was sponsored by Microsoft and posted prominently on the software giant’s Web site. Also, DiDio got caught up in the famous SCO case. When SCO first made its claims that IBM had misappropriated some of its code and handed it over to the Linux community, SCO showed samples to several analysts to prove its copyrights were being infringed. DiDio, a former journalist and not a programmer, was one of them. She reported that SCO’s claims seemed justified. She told me: “It appeared to be a direct cut and paste right down to the developers’ notes.” A couple of months ago, the judge in the case wrote that he had seen “an astonishing lack of evidence” backing up SCO’s claims. On the phone, I asked DiDio’s reaction to the judge’s statement. She said: “I can’t reconcile it. I want to see what’s presented in court.”Back to her new survey: The press release and survey were scheduled to be released April 4. In preparation, I talked to marketing professors and other tech market researchers to ask what they thought about Web-based, self-selecting surveys. Their answer: they stink. You can’t post a survey on your Web site and let all comers decide if they want to fill it out. When I confronted DiDio with these conclusions on April 4, and told her I was skeptical of her results, she explained that the survey really wasn’t as she had described at all. It was done by a third-party survey outfit who solicited responses from a pool of qualified business computer users. “It really isn’t self-selecting,” she said.

Problem solved? Not so fast. Yankee Group sent me a set of PowerPoint slides, rather than the raw data that I had asked for. The e-mail arrived on April 5, 24 hours after the release went out. It made it difficult for me to write a timely story about the survey. (In fact, this blog is the first I have written about it.) I didn’t think much of the slide show either. One slide said “Half of Users Say Linux Deployment Is Cheaper than Windows.” You might draw the conclusion that the other half say Windows is cheaper than Linux. But you’d be wrong. The bar chart on the slide showed that 34% of the respondents have not deployed a Linux server, so have no grounds for an opinion, and only 9% said their Linux deployments were more expensive than Windows deployments.I asked to speak to the third-party firm that had done the survey, but DiDio instead referred me to Yankee’s CEO, Brian Adamik who referred me to Phil Fersht, a vice-president of research. He called so I asked him all of my skeptical questions. He basically said there was nothing wrong with the way the survey was done and the analysis was presented. He invited me to ask detailed questions about the data, and sent some info in answer to my follow-up e-mail. One factoid: The survey had not, in fact, asked screening questions about what network operating systems people had installed. So, I guess, the press release was wrong. Also, he said, the PowerPoint slides I had questioned were unedited, and had since been improved. He also declined to show me the raw data or tell me who the third-party surveyor was.I know being an industry analyst is a hard job. In this case, the work seems sloppy, and, to my mind, not reliable. Yankee Group once had a reputation for being a researcher for hire—distributing sponsored reports to the public. Fersht says that practice ended a year ago. Good. And good riddance. Hopefully, Yankee will work hard on improving the way it communicates with the press, too.07:15 PM

没跳行了, 这真是很抱歉. 这是OSDL上面的文章.