Speed Test: iPhone 3GS Even Faster than Apple Claims
June 24th, 2009 by Theo SkyeApple has made claims that the iPhone OS 3.0 yields significant performance gains on the 3G model, and that the new 3GS can accomplish the same tasks up to twice as fast as its predecessors. Anecdotally, the new 3GS definitely “feels” faster under certain conditions. But how do Apple’s devices and OS versions really compare to one another? And perhaps of even greater interest, how does the latest hardware from Cupertino compare to smart phones recently released from other vendors?
“Objavectaweb-C” OS?
One of the key challenges in conducting an objective evaluation of software performance across devices that utilize different operating systems lies in accounting for the fundamental differences in the various OSs. While the iPhone 3G and 3GS could potentially run the same app on the same Objective-C-based operating system (making direct comparisons relatively straightforward), Android apps are Java-based, and the Palm Pre runs the entirely new Web OS. Given these divergent OS implementations, is there anything that come close to a standard unit of measure for judging performance of this growing breed of “superphones?”
Finding Common Ground
The common thread between these three OS’s is JavaScript execution in WebKit—the open source project that, in varying degrees, powers web browsing technology for these three disparate operating systems. With the exception of certain browser plugins (e.g., Flash), web rendering technology installed on today’s premiere mobile devices makes almost all—and in some cases even more—features of their ubiquitous desktop web browser counterparts available. Therefore, given the global commonality of JavaScript and WebKit-based web browsers, it becomes possible to compare the performance of these “pocket computers that make phone calls” to the performance of desktop machines.
The Yardstick
The WebKit Open Source Project provides a JavaScript test Suite dubbed SunSpider. According to the description on the SunSpider home page, “this benchmark tests the core JavaScript language only, not the DOM or other browser APIs. It is designed to compare different versions of the same browser, and different browsers to each other.” We at Medialets have found it to be one of the best attempts to measure real world JavaScript performance in a balanced and statistically sound way.
Medialets ran the SunSpider test suite in the following environments:
- Safari 4.0.1 on a 2.0 GHz Intel Core 2 Duo White MacBook.
The MacBook results were used as a baseline for relative comparisons. - Mobile Safari on the iPhone 3G with iPhone OS v2.2.1
- Mobile Safari on the iPhone 3G with iPhone OS v3.0
- Mobile Safari on the iPhone 3GS with iPhone OS v3.0
- The “Browser” app on the T-Mobile G1 with Android OS v1.5 (Cupcake)
- The “Web” app on the Palm Pre with Web OS v1.0.2
Each device was fully restored and rebooted immediately before running the test suite. Every attempt was made to assure that no atypical background tasks were executing while the tests were running. The SunSpider tests automatically run five times sequentially and the mean average from all five tests are reported. Network speed and latency have no effect on the results of the test.
Disclaimer: Before considering the results of the tests, it is important to note that each OS likely has certain advantages and features that probably make it inherently well suited for some tasks more than others. The main purpose of these comparisons is merely to compare JavaScript performance within each environment. It should not be misconstrued as indicative of which device or OS is inherently “better” than any other.
Results
The results of the iPhone-based tests alone are rather astonishing and seem to indicate that many of Apple’s claims about the performance gains of their 3.0 OS and the iPhone 3GS may hold some water. Using OS 3.0 on the same iPhone 3G yields nearly 3X the JavaScript performance in Mobile Safari vs. using iPhone OS 2.2.1. The iPhone 3GS ups the ante by another factor of 3, bringing JavaScript performance on the iPhone 3GS to just 12X that of a full-powered desktop machine that has well over four times the raw processing muscle alone. The T-Mobile G1 running the “Cupcake” version of the Android OS completed the test suite in about 91 seconds. This makes it about a third faster than the iPhone 3G running Apple’s previous OS (2.2.1). The Palm Pre came storming out of the gate with speeds that closely rival the iPhone 3G running Apple’s latest iPhone OS.
Do any of these numbers really indicate which phone might be the best choice for a given individual? Absolutely not. At Medialets we use all of these devices, and love each one for many reasons. The fact that these tests can even be performed across this many device/OS combinations is a testament to how far mobile technology has come in such a relatively short time. We are looking forward to seeing an even greater variety of advanced mobile devices and OS revisions enter the market and we’ll keep you posted as we test more devices in our lab. Subscribe to our feed, leave a comment below, or reach out to us directly if you have any questions.
Tags: android, benchmarks, G1, iphone, iphone3g, iphone3gs, palm, pre
June 24th, 2009 at 3:12 pm
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 24th, 2009 at 3:15 pm
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 24th, 2009 at 3:17 pm
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 24th, 2009 at 3:39 pm
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 24th, 2009 at 3:47 pm
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 24th, 2009 at 4:00 pm
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 24th, 2009 at 4:02 pm
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ tests, the [...]
June 24th, 2009 at 4:03 pm
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ tests, the [...]
June 24th, 2009 at 4:04 pm
[...] Sobhany of Medialets has posted on Sun Spider benchmarks of the iPhone 3GS as well as other devices which shows off the performance angle of 3GS: The WebKit Open Source [...]
June 24th, 2009 at 4:04 pm
[...] that take a bunch of variables into detail. Mobile analytics and advertising company Medialets has released numbers for a experiment that it considers to be the most direct line of comparison for the iPhone 3GS [...]
June 24th, 2009 at 4:05 pm
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ tests, the [...]
June 24th, 2009 at 4:08 pm
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ tests, the [...]
June 24th, 2009 at 4:18 pm
[...] I don’t have a Pre yet, but take a look at the SunSpider JavaScript Benchmark results performed by Medialets (this is originally from an article about how the iPhone 3GS is much faster than the 3G). [...]
June 24th, 2009 at 5:35 pm
[...] navegadores, se comparados com o novo celular da Apple? A fim de traçá-lo, a Medialets fez um estudo do desempenho dos principais navegadores móveis do mercado (baseados em WebKit), usando como base [...]
June 24th, 2009 at 5:38 pm
[...] validating Dieter’s iPhone 3GS vs. Palm Pre web rendering smackdown, MacRumors reports on Medialets‘ latest Sunspider Javascript tests pitting the iPhone 3GS against the iPhone 3G (both on 3.0 [...]
June 24th, 2009 at 5:43 pm
[...] validating Dieter’s iPhone 3GS vs. Palm Pre web rendering smackdown, MacRumors reports on Medialets‘ latest Sunspider Javascript tests pitting the iPhone 3GS against the iPhone 3G (both on 3.0 [...]
June 24th, 2009 at 6:37 pm
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 24th, 2009 at 7:12 pm
[...] Speed Test: iPhone 3GS Even Faster than Apple Claims – Medialets: Apple has made claims that the iPhone OS 3.0 yields significant performance gains on the 3G model, and that the new 3GS can accomplish the same tasks up to twice as fast as its predecessors. Anecdotally, the new 3GS definitely “feels” faster under certain conditions. But how do Apple’s devices and OS versions really compare to one another? And perhaps of even greater interest, how does the latest hardware from Cupertino compare to smart phones recently released from other vendors? [...]
June 24th, 2009 at 7:38 pm
[...] For full details of the benchmark pop quiz, see Medialets’ summary of results. [...]
June 24th, 2009 at 7:49 pm
[...] For full details of the benchmark tryout, see Medialets’ summary of results. [...]
June 24th, 2009 at 7:59 pm
[...] For full details of the benchmark pop quiz, see Medialets’ summary of results. [...]
June 24th, 2009 at 8:02 pm
[...] For full details of the benchmark tryout, see Medialets’ summary of results. [...]
June 24th, 2009 at 8:17 pm
[...] тестов нового iPhone 3GS сегодня пополнилась результатами проведенного компанией Medialets сравнительного [...]
June 24th, 2009 at 8:18 pm
[...] For full details of the benchmark experiment, see Medialets’ summary of results. [...]
June 24th, 2009 at 8:18 pm
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 24th, 2009 at 8:27 pm
[...] For full details of the benchmark tryout, see Medialets’ summary of results. [...]
June 24th, 2009 at 8:28 pm
[...] slower at browsing than the Iphone? Medialets
June 24th, 2009 at 8:29 pm
[...] For full details of the benchmark analysis, see Medialets’ summary of results. [...]
June 24th, 2009 at 8:33 pm
The reason for the huge speedup in iPhone OS 3.0 on the 3G is that the new OS uses SquirrelFish, a highly optimized new JavaScript engine Apple’s developed. SquirrelFish is also in Safari 4 — you didn’t specify which version of Safari is installed on the Mac you tested with.
From the numbers, I would guess that the Palm Pre also uses the same version of WebKit with SquirrelFish, and that Android does not. (Google has its own accelerated JavaScript engine, V8, but it might not yet be ported to the G1’s CPU — V8 translates JS to machine code, so it’s very CPU-dependent.)
June 24th, 2009 at 8:34 pm
[...] full details of the benchmark analysis, see Medialets’ summary of results. cellphone accessories cellphone camera apple iphone iphone application mobile reviews mobile [...]
June 24th, 2009 at 8:46 pm
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 24th, 2009 at 9:31 pm
[...] Medialets Ads by GoogleWap Mobi Android – Mobile development – m.micromediausa.com SPONSORED [...]
June 24th, 2009 at 10:38 pm
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ [...]
June 24th, 2009 at 11:20 pm
[...] For full sum of the benchmark test, see Medialets’ outline of results. [...]
June 25th, 2009 at 12:02 am
[...] Medialets » Blog Archive » Speed Test: iPhone 3GS Even Faster than Apple Claims [...]
June 25th, 2009 at 1:02 am
[...] addirittura aver sottostimato l’aumento di velocità ottenuto in questo tipo di processo. Medialets ha testato diversi dispositivi con il noto benchmark SunSpider e ha raccolto i dati nel grafico che [...]
June 25th, 2009 at 1:13 am
[...] some of the speed gains it advertised. Medialets, a mobile advertising and analytics company, ran the SunSpider JavaScript benchmark on the iPhone 3G with the old and new OS versions, as well as on the 3GS. In Medialets’ [...]
June 25th, 2009 at 2:05 am
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 25th, 2009 at 2:25 am
[...] of iPhone 3G and 3GS, the T-Mobile G1 running Android, and the Palm Pre running webOS by Medialets. And who else by iPhone 3GS outperformed the Palm Pre by a factor of three and the T-Mobile G1 by [...]
June 25th, 2009 at 3:28 am
[...] 3G和3GS上运用SunSpider JavaScript进行了基准测试。在测试中，iPhone [...]
June 25th, 2009 at 4:17 am
[...] that take a bunch of variables into account. Mobile analytics and advertising company Medialets has released numbers for a test that it considers to be the most direct line of comparison for the iPhone 3GS against [...]
June 25th, 2009 at 4:47 am
[...] Medialets tweetmeme_url = [...]
June 25th, 2009 at 5:05 am
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 25th, 2009 at 5:31 am
[...] mobile analytics outfit Medialets reckons it’s found a common ground for speed comparisons in the form of JavaScript benchmark [...]
June 25th, 2009 at 6:20 am
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 25th, 2009 at 7:30 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 7:55 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 7:58 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a racetrack. [...]
June 25th, 2009 at 8:15 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:16 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:17 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:17 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:27 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:28 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 8:53 am
[...] again validating Dieter’s iPhone 3GS vs. Palm Pre web rendering smackdown, MacRumors reports on Medialets‘ latest Sunspider Javascript tests pitting the iPhone 3GS against the iPhone 3G (both on 3.0 and [...]
June 25th, 2009 at 9:04 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 9:47 am
[...] mobile analytics outfit Medialets reckons it’s found a common ground for speed comparisons in the form of JavaScript benchmark [...]
June 25th, 2009 at 10:00 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 10:21 am
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 25th, 2009 at 10:29 am
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 25th, 2009 at 11:15 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 11:32 am
[...] Medialets – Javascript speed test for iphone 3g, 3gs, android, and palm pre. the 3gs won by a mile. interestingly, the 3g has a massive improvement from the 2.0 to 3.0 OS. [...]
June 25th, 2009 at 12:35 pm
Thanks for sharing the additional information, Jens.
Also, I just wanted to make sure you saw within the article (however, not on the chart) that we used Safari v4.0.1 on the 2.0 GHz Intel Core 2 Duo White MacBook.
June 25th, 2009 at 12:46 pm
[...] Medialets: “The results of the iPhone-based tests alone are rather astonishing and seem to indicate that many of Apple’s claims about the performance gains of their 3.0 OS and the iPhone 3GS may hold some water.” [...]
June 25th, 2009 at 1:15 pm
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 25th, 2009 at 1:33 pm
[...] For full details of the benchmark test, see Medialets’ summary of results. [...]
June 25th, 2009 at 2:05 pm
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a racetrack. [...]
June 25th, 2009 at 4:22 pm
[...] zitiert die Smartphone-Analytik-Plattform Medialets mit der Aussage, dass 3GS-Modelle bei der Aufgabe, rein JavaScript-basierende Webseiten aufzurufen, [...]
June 25th, 2009 at 5:05 pm
[...] een nieuwe test gedaan door Medialets komt de iPhone 3GS bijzonder goed naar voren. De test heeft een andere insteek dan overige [...]
June 25th, 2009 at 5:54 pm
[...] Test Sonuclari Medialets [...]
June 25th, 2009 at 6:00 pm
[...] test showing the Palm Pre beating the iPhone 3GS in two out of three speed tests. Now there’s a new speed test from Medialets that does a fairly rigorous comparison of the JavaScript performance of iPhone 3GS, Palm Pre, [...]
June 25th, 2009 at 8:43 pm
[...] Speed Test: iPhone 3GS Even Faster than Apple Claims. I wonder if JavaScript on the 3GS is faster than Java on the [...]
June 25th, 2009 at 8:50 pm
[...] Medialets shows the iPhone 3G S blows the competition away on their benchmark Test Suite. The iPhone 3G S completes it in 16.5 seconds compared to the Palm Pre’s 48.6 seconds. Googles G1 toped a minute and a half with 91.1 seconds. This also shows the substantial increase of the 3G S compared to the 3G. [...]
June 26th, 2009 at 1:55 am
[...] discovering, you are in possession of one sweet handset. And by the way, the speed being offered is actually faster than even Apple is laying claim to. Lucky [...]
June 26th, 2009 at 8:30 am
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 26th, 2009 at 2:26 pm
[...] Damn, that’s a fast 3GS [...]
June 26th, 2009 at 2:45 pm
[...] about which device is faster than which, without resolution or relief. Fortunately for the weary, Medialets has found common ground between a few of the most popular smart phones to use as a [...]
June 27th, 2009 at 2:13 am
[...] has performed a JavaScript based benchmark tests using the SunSpider test suite on the iPhone 3G and 3GS, the T-Mobile G1 running Android, and the [...]
June 27th, 2009 at 9:47 am
[...] Sobhany of Medialets has posted on Sun Spider benchmarks of the iPhone 3GS as well as other devices which shows off the performance angle of 3GS: The WebKit Open Source [...]
June 28th, 2009 at 2:10 pm
[...] iPhone 3GS web browsing — faster than iPhone 3G,G1, and the Palm Pre [...]
June 28th, 2009 at 9:58 pm
[...] Uncategorized | Monday June 29 2009 1:54 am | Tags: 3g, app, apple, iPhone, iPod, touch Medialets released the results of JavaScript benchmark tests with the SunSpider test suite on the iPhone 3G, [...]
July 2nd, 2009 at 8:39 am
[...] new speed test made by medialets shows super speed by the iPhone 3GS and a very improved speed by the iPhone 3G when using the new [...]