 |
PDAs |
 |
|
PDAs are becoming so fast and feature-rich that they're performing many tasks that used to belong in the exclusive domain of the PC. A PDA's feature set is probably the most important factor when it comes to deciding which one is right for you, but how well a PDA performs is an important variable that should not be neglected. Performance dictates everything from how fast a PDA opens a file to how long the device's battery lasts. Our test suites utilize real-world applications that test a PDA's various subsystems, such as its CPU, file system, and battery life.
| Pocket PC test environment |
|
All tests are run with a PDA's CPU set to highest speed and with its wireless networking (if supported) disabled. We connect the PDA to a desktop system using the fastest interface that the PDA supports--typically, USB 2.0, USB 1.1, or serial. We test using Microsoft ActiveSync 4.0. The desktop system is configured with a 3.4GHz Pentium 4 550 processor, 1GB of DDR2 SDRAM running at 533MHz, an Nvidia GeForce 6600 PCI-Express graphics card with 256MB of memory, a 74GB Western Digital WD740 Raptor hard drive, and Windows XP Professional SP2. All tests are repeated a minimum of three times.
| Pocket PC performance tests |
|
CNET Labs uses the Spb Benchmark utility to measure the performance of Pocket PC-based PDAs. The test produces three scores: an ActiveSync index, the Spb index, and battery-test score.
)
|
The ActiveSync index measures the performance of a Pocket PC-based device communicating with a desktop computer by downloading and uploading files from the Pocket PC device to the desktop via ActiveSync.
)
|
The Spb index is a single, overall system-
performance score derived from these four
subsystem indexes:
- CPU index: This test measures the performance of a Pocket PC-based device's processor and memory subsystems by performing both integer and floating-point calculations.
- File-system index: This test measures the performance of a Pocket PC-based device's internal file system. Neither the amount of onboard memory nor the presence of storage cards impact this test.
- Graphics index: This test measures the performance of a Pocket PC-based device's graphics subsystem.
- Arkaball frames-per-second test: This test loads and runs the Spb Arkaball game and measures the screen-redraw rate. This test utilizes a Pocket PC-based device's processor and graphics subsystems.
)
|
To test a Pocket PC-based device's battery life, we start with a fully charged battery. We set the device's CPU to Auto (if available), Mid, or Normal level and the backlight brightness to medium. We don't allow the device to enter Suspend mode, and we disable the device's "Receive all incoming beams" setting.
The test repeatedly plays a video clip using the device's Windows Media Player application. For devices that contain backup batteries, we allow the video clip to keep playing until the main battery dies. For those devices that don't have backup batteries, we stop the test when the battery is down to 10 percent. If we were to allow a device without a backup cell to continue until the battery died completely, all of the user data would be lost. Battery life is reported in minutes.
| Palm OS performance tests |
|
While we await the finalization of a more formalized benchmark for Palm OS-based PDAs, we currently test these PDAs for CPU/video subsystems performance and battery life.
)
|
These tests are designed to detect dropped or stuttering frames in full-motion video and games. Dropped frames or sluggish response is indicative of poor screen-response time and/or an underpowered CPU. Our tests include playing condensed QuickTime movie trailers in Kinoma Player 2, as well as such games as TableTennis3D, Kickoo's Breakout, and SimCity from MDM's Game Essentials.
Battery tests
To test a Palm OS PDA's battery life, we start with a fully charged cell. We set the device's screen brightness and volume settings to half, and we don't allow the device to enter Suspend mode. The battery-life test repeatedly plays a video clip using the Kinoma Player until the battery dies. Another test consists of playing an MP3 audio file using the PDA's default audio-player app; the MP3 file is repeatedly played in a loop, with the screen allowed to go into standby. If the PDA includes built-in wireless networking, the initial battery tests are conducted with the wireless networking disabled, then repeated with them activated. Battery life is reported in minutes.
