Testing Mobile-Optimized Websites on Many Mobile Devices
As mobile devices proliferate, you'll want to ensure that your mobile-optimized website performs properly on the many various mobile OS/browser combinations, and that it displays properly on as many different screen sizes as possible. Given the unlikely possibility that you can obtain one device of every shape and size, here are some recommendations for conducting comprehensive cross-device QA testing.
Simultaneous testing of six mobile devices using Adobe Shadow
Test on as many different actual devices as you can
Mobile device emulators (see below) are good for getting a preliminary sense for how your mobile website will display on various devices--but given the margin of error provided by the stack of technologies involved (e.g., an Android or iPhone emulator running on Windows)--you really need to do your final testing on real devices.
Hopefully your organization has some mobile devices retained specifically for this purpose.
People often hang onto their old devices, and (even without a phone plan) those devices are still fully capable of connecting to the Internet through your local wireless network. Poll your co-workers and friends to see who can lend you old smart phones or iPod Touch.
If you need to beg your co-workers to lend you a device/OS combination not represented in your collection, swallow your pride and do it. You'll be glad you did.
You should assume that even the old devices are all still in use somewhere, so you'll want to find out if any of them have issues with your website.
Test on multiple devices simultaneously with Adobe Shadow
Adobe Shadow is a very cool free mechanism for testing multiple mobile devices simultaneously. You install the Adobe Shadow "master" (my term) as an add-on to Chrome on your computer. Your computer must be accessible by the mobile devices via a local wireless network. Obtain the Shadow mobile app from the Apple App Store or the Google Play Store, and install it on as many mobile devices as it will run. It it does not run on iOS 4x, and you'll have to jump through extra hoops to get it installed a Kindle. There were so many hoops to get an app onto a Nook, it's easier just to test that device manually.
When you start the Shadow app on each mobile device it will look for the Shadow master (in Chrome on your PC or Mac) and will identify itself to the master. Enter the password provided by the app into the master console to complete the connection.
With Shadow running in Chrome, a URL you enter into the Chrome address bar gets transmitted to the browser on each mobile device. Similarly the URL behind each click in Chrome is transmitted to each mobile device. Effectively you are driving multiple devices with one installation of the Chrome browser.
Note that the current version of Shadow does not transfer text input from the Chrome master to the mobile apps—a future enhancement, I hope. If the mobile site requires a login you’ll have to manually enter the username and password on each mobile device. Speaking from experience, keep your username and password short!
Use device emulators
Using device emulators on your PC or Mac will give you a preliminary sense for how your mobile site looks on each type of device. I would not rely exclusively on emulators since, obviously, they are not the real thing and can bring their own bugs or idiosyncracies to the table, but they definitely provide value.
For Android, you’ll need to install the Android SDK to get the emulator. The Android package will, however, let you create smart phone and tablet emulators for all Google versions of the Android OS.
Mobilizer by Springbox provides a software package that emulates iPhone 4, HTC Evo, Blackberry Storm and Palm Pre. The version I installed was a little buggy, but overall it did the trick of emulating the iPhone experience for me.