Windows 7: How We Test
Thursday, October 1, 2009; 12:18 AM
WorldBench 6 Testing
For our "Windows 7 Performance Tests" article, we ran our WorldBench 6 test suite on a set of five different PCs: two desktop systems (the high-end E&C Black Mamba PC and the mainstream HP Pavilion a6710t), two laptop systems (the budget Gateway T-6815 and the mainstream Lenovo IdeaPad Y530), and one netbook (the Lenovo IdeaPad S10-2). On the two desktops and two laptops, we compared Windows 7 against Windows Vista.
Unlike many benchmark tools, our WorldBench 6 test suite is a real-world benchmark. We test systems by running actual, commonly used applications, not synthetic, arbitrary benchmarks, to give you a better idea of how well a machine will perform in daily, regular use.
To test laptop battery life, we alternate between 15 minutes of typing and 15 minutes of full-screen video playback. We repeat the process until the battery dies. We run the test twice and then average the scores. If the scores differ by more than 10 percent, we run the test a third time and take the closest two scores for the average.
For boot-up and shutdown times, a member of the PC World Test Center hand-times our speed tests. To ensure consistent results in these tests, we repeat each test ten times and then calculate the average for each. We measure startup times from when we press the computer's power button until the Windows desktop loads (defined for these purposes as when the mouse pointer and the desktop background appear). We measure shutdown times from when we click the Start menu's Shut Down button until the PC completely powers down and the power lights (if present) go dark.
Application Launch Times
To test application launch times for our "Windows 7 Performance Tests" article, we used Microsoft Word 2007 and Excel 2007, plus Adobe Photoshop CS4. Photoshop CS4 opened in 32-bit mode on 32-bit systems, and in 64-bit mode on 64-bit PCs. For Word, we launched the program and then opened a two-page document. We used a similar process for Excel and Photoshop, except that we opened a 10-page document in Excel and we opened both a PDF file and a JPEG image in Photoshop CS4. We ran these tests three times per application to ensure consistent results.
When measuring app launch times, we began timing from when we clicked the app or document's icon to when the program was fully loaded and usable.
On the Lenovo netbook, we compared Windows 7 with Windows XP, since XP is the operating system that most netbook models ship with today. For more on the performance results we achieved with those two OSs, see "Windows 7: Can Your Netbook Handle It?"
For more information about Windows 7, sign up for PC World's Windows News and Tips newsletter.