InfoWorld, Windows 7 and benchmarks: a trio of trouble



I wasn’t going to call attention to the lengthy InfoWorld article that benchmarks Windows 7 Milestone 3, but after further thought, I’d be remiss if I let this slide. After all, more viewpoints are generally better than fewer ones. However, it exemplifies why we try not to put too much stock in benchmarks, although as a courtesy for those that want them, we often do run benchmark tests on devices. I really don’t have a problem with anyone looking under the hood at Windows 7; in fact, I’m doing that myself on both my MSI Wind and my Samsung Q1 Ultra Premium UMPC. What I’m not doing however, is making grand sweeping statements of doom and gloom about Windows 7 based a pre-beta, test release. Unfortunately, and dare I say, irresponsbily, I feel that’s exactly what’s being done at Info World.

The fact is: the version of Windows 7 provided to Microsoft PDC attendees, which is the same version I downloaded as part of the Windows 7 beta program, is not the latest and greatest version of Windows 7. It was meant to give people an idea of where Windows 7 was headed, not to provide the final optimized, de facto look at it from a performance standpoint.

Although the InfoWorld benchmarks show little or no benefit over Vista, my own personal experience is far better. Can I provide detailed numbers to back that up? No, because the version isn’t intended for that purpose. I did offer a few high-level bits of data such as processes upon startup and the Windows Experience Index, but that’s generally as far as I went and that’s as far as I’m going from an empirical standpoint.

In some ways that’s more than I wanted to say on the topic… and in some ways not enough. Suffice it say: I’m disappointed in the article and I hope I’ve explained why. I’ve certainly made my share of writing mistakes as well, and I’m glad that readers have called me out on them when they feel I’ve done so. It makes be a better person and writer. Hopefully, the author of the ill-conceived article at InfoWorld takes my thoughts in the same vein.


Dave Lindhout

@B. Factor: The article you link to was written by Randall Kennedy from InfoWorld, saying PC World has substantiated InforWorld’s claims is a bit miss leading. I found value in Randall’s, however.


I meant to say “I really doubt it will be faster THAN XP on a MSI Wind.” on my previous comment erh!


I am sure it’s faster than Vista but I really doubt it will be faster on a MSI Wind. Since I don’t see any blogger bother to do the comparo with a XP.

And waiting for apps like keytweat running smoothly on W7 you know will take a long time.


James, no business is going to run an alpha build of an OS on production machines so any article that benchmarks that build is worthless. Surely you can appreciate the futility of an article like this.

Even if businesses are deliberately avoiding Vista as you suggest, they will not be evaluating Windows 7 based on an alpha build.

Anyway, I didn’t think anyone bothered with Infoworld after that stupidly pointless save XP campaign they ran.


People should remember InfoWorld was the only place that gave Win7 negative comments a couple of weeks back. I seriously doubt those people’s intention.


Win 7 on my creaky old 1.2Gz TC1100 is unquestionably faster than Vista on same, even vLite’d and tweaked for speed.

The vanilla 10GB Ultimate Win 7 uses less CPU, had 40 processes running and was speedier across the board.

CrystalMark 04 gave Vista 30,000 and Win 7 just 22,000. Using THAT as reference would color real experience.

James Kendrick

I see both Kevin’s and the author of the article’s points and they’re both right. I viewed the InfoWorld article as pointing out valid points for Microsoft. Enterprises have in large part avoided Vista and MS better demonstrate that Win 7 is different than Vista in the areas that the enterprise need it to be. The author was pointing out that in his opinion the current Win 7 build does not do that and that as such the enterprise will have no compelling reason to adopt it.


I simply think Microsoft is unable to build an OS that can work both as a mobile system and an all-in-one desktop mega OS. I remember how they said Vista would boot up as fast as a television set. Look how that turned out.

Someday, when notebooks turn on like our cellphones do, we’ll look back at Windows and LAUGH. I don’t have much hope for WIn7 being a truly mobile OS.


I don’t think the article is wrong in any respect. Kennedy does not SAY that Windows 7 will be a failure. He just points out that based on his evaluation of the software… is not much different from Vista. Then he posits whether this is going to be good enough for those that have already passed on Vista.

Kevin C. Tofel

Tom, I see your point better now. Since there were no restrictions on this pre-beta, test build in terms of NDA status, it’s fair game for commentary; good and bad. I can agree with that bit.



We can agree to disagree on this, but to answer your question about Mac OS X 10.5.6, if Apple made a beta available without restrictions on comment (I’m assuming none of the comments about Win7 break any NDA), then OF COURSE you could publish whatever you wanted. Why wouldn’t you? And that’s my point.

But Apple hasn’t released 10.5.6 in such a fashion. MS, however, DID release Win7 that way, and they must take the bad with the good. That’s all I’m saying.

I’m sure many people will look through the InfoWorld piece as being premature, but unless they broke some sort of agreement or NDA, I’m not inclined to think it was irresponsible. Certainly no more irresponsible than, say, Thurrott’s ramblings about how Win7 is The Second Coming.


Well, I share Kevin´s concerns. But at the same time I posted what I experienced at my site, and it´s a better performance in a UMPC than when using Vista in that same device. And so far I have other friends of mine testing it in a even slower device using a VIA processor and he shares the same experiences than me. It feels more responsive than Vista.

I hope Betas wont becomes worse than current prebeta. I hope so.

B. Factor


I think you misread and misrepresented the Infoworld article. They said by every technical way they had of instrumenting the Windows 7 operating system, it was nearly identical to Vista. It had nearly the same number of kernel threads, they used nearly the same amount of resources, etc etc etc. So Windows 7 is nothing more than Vista RC2 which really shouldn’t surprise anyone given that MSFT is planning to release it in 2009 AND any marketing person would tell MSFT that they have to rename their operating system because so many people hate Vista.

Kevin C. Tofel

Tom, I’m not suggesting we should be drooling over it either but to make broad leaps about how miserable the performance will be on a pre-beta, test build that’s not optimized for performance isn’t responsible in my opinion. I don’t mind that people criticize a product when it’s a final, readily available product.

Put another way: would it be appropriate for me to get the first seeded test build of OS X 10.5.6, find performance flaws and then claim with near certainty that 10.5.6 will peform miserably when it becomes available in final form? I don’t think that would be any different nor would I be correct for doing so. Just my opinion, of course.



There has been a lot of raving over Win7 the last few weeks. Heck, Paul Thurrott actually thanked Microsoft for it! It’s ridiculous.

If I misunderstood you then correct me, but you seem to be saying that people can drool all over it but not criticize it since it’s beta. I don’t agree. That sword cuts both ways, and who knows what features or promises that people look forward to may get removed?

I believe MS has to take the bad with the good. You release it — even as beta — and it’s going to looked at “as is”. It appears the “risk” MS took was worth it, as the press is generally favorable.

I’m waiting for something much more real. After 20 years, MS promises are pretty much worthless to me. Still, it’s just as fair to criticize any aspect of this release as it is to praise it.

Comments are closed.