Blog Post

iPhone 3G and iOS 4: Benchmarking the 4.1 Update

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

When iOS 4 was released, many iPhone 3G owners felt the pains associated with having a slower device due to an OS that was more processor intensive. Apple (s aapl) stated it would be looking into the performance issues associated with running iOS 4 on older 3G iPhones, and thankfully, at the latest media event, a solution was confirmed to be ready.

So has the recently released 4.1 update finally closed the door on the issue of iOS 4 running sluggishly on the iPhone 3G?

iOS 4.1 Update Tested

To test how the iOS 4.1 affected performance, I took a pair of iPhone 3Gs and ran them through a series of performance tests. I used GeekBench 2 ($1.99), Gauge Mathematical Tool ($1.99), BenchTest (99 cents), SunSpider (free) and V8 (free) to test the performance of each OS version on the 3G.

I started out with one iPhone 3G running 3.1.3, and the second running 4.0.2. I must admit that both 3Gs were dumbed down quite a bit, and were not being utilized as iPhones, but more like iPod touches. The iPhones used were purchased at the same time, shipped in the same shipment, and have very similar Serial, IMEI, Wi-Fi and Bluetooth numbers assigned to them. The following results are only a subset of the data collected, and focus on the data points that changed the most from version to version:

Results for iPhone 3G: 3.1.3 vs 4.0.2

It’s not what the primary results did say, it’s what they did not say. As I executed the tests side by side, I noticed that even when similar end results were returned by the benchmark tests, the UI of each iPhone would behave differently. The 3G running 3.1.3 would behave as expected, but the 3G running 4.0.2 did not. Progress bars wouldn’t update smoothly on 4.0.2, but would jump to 100 percent at the end of the test. As tests were executing, it was as if the user interface on 4.0.2 was somehow sporadically frozen during the tests. This behavior is consistent with other video reports online that iOS 4 is slow on 3G devices. It also explains why certain activities, namely games that don’t utilize Apple’s stock UI components, run pretty much the same on each OS version. If you now use your 3G primarily as a game device as I do, you may not have noticed all the differences in performance.

If, on the other hand, you’ve continued to use the device as a smart phone, you’ve most likely noticed the degradation in performance. What was surprising at first was that the JavaScript results were much faster on iOS 4. This makes some sense, since the version of Safari on iOS 4 most likely has the latest JavaScript engine under the hood. So while some UI elements may be slower, and most hardware-based benchmarks remained relatively constant, the JavaScript results actually improved from 3.1.3 to 4.0.2.

Results for iPhone 3G: The 4.1 Update

I then upgraded the 3G running iOS 4.0.2 to the new iOS 4.1. At this point, I had one iPhone 3G running iOS 3.1.3, and the other running iOS 4.1. This time around, the UI did appear to behave more as one would expect on 4.1, with smoother progress bars and a zippier response from the UI. It felt faster, but was it as fast as 3.1.3? No, not really.

While certain numbers did seem to bounce back, namely the performance of the  Stdlib Allocate test within the GeekBench test suite, most numbers remained the same as in 4.0.2. The BenchTest results for Drawing Into View and File to Filesystem did seem to bounce back a little as well. It was only the JavaScript results (as per the SunSpider test) that really seemed to improve even further. However, it’s important to note that the iPhone 3G (on both versions of the OS) never passed the V8 test. Safari always crashed before completing the test.

One More Test: iPhone 4 on 4.0.2 vs. 4.1

So before drawing any conclusions, I wanted to see if the subtle performance increases noticed between 4.0.2 and 4.1 on the 3G were the same performance increases noticed on the new iPhone 4. Finding the same performance increases on the iPhone 4 as compared to the iPhone 3G would point to an overall performance boost based on OS updates that weren’t specific to any one hardware platform. Luckily, I happen to have two iPhone 4s in the house as well. This time, I performed benchmarks on the exact same hardware before and after the upgrade. I was very surprised with the results of the performance tests on 4.0.2 compared to 4.1 on the iPhone 4.


It’s still very likely that Apple did focus on performance issues related to each one of its own internal apps that are distributed only via OS updates. Apart from the JavaScript benchmark results — which prove that Safari on iOS 4 outperforms Safari on iPhone OS 3 — the boost to overall performance based solely on the test results listed above is marginal.

These tests, however, are looking at the OS itself, not individual app performance improvements. Surprisingly, there does appear to be a platform-specific focus on performance issues related to the iPhone 3G, as performance does appear to have degraded on the iPhone 4 with the 4.1 update. I’d been focusing heavily on the iPhone 3G, and had run several iterations of tests comparing 3.1.3 to 4.0.2, and was prepared to collect the same amount of data following the 4.1 update on the iPhone 3G. I was much more meticulous with the details on executing the tests.

The tests against the iPhone 4 were an afterthought, and would require a more through examination before claiming that the iOS 4.1 update did indeed degrade the performance of the iPhone 4. Just as we struggled through the iPhone OS 3 updates on the iPhone 3G last year, there will likely be more updates to iOS 4 in the months to come. While I still don’t feel iOS 4.1 is back to the same performance levels of iPhone OS 3.1.3, there do appear to be performance gains in 4.1 that prove that Apple is serious about supporting the iPhone 3G on the iOS 4 platform.

Online BenchMark Results

20 Responses to “iPhone 3G and iOS 4: Benchmarking the 4.1 Update”

  1. 4.1 upgrade, 3G, WiFi will not stay connected.
    Right when I upgraded to 4.1 my WiFi started going On/Off/On/Off over and over.
    the phone itself worked smoother, but at the Expense of my WiFi..
    1st I Reset all my Network settings=no help
    2nd I JUST Reset my Whole Iphone= No Help
    .. I guess we ALL need to email apple and get them to fix it in the next year or so..
    … I so wish they’d FIX their new iphone4 antenna issue, then I’d give up my 3G.
    Otherwise I’ll have to wait for the newer iphone
    ( and NO, their bumper guard on their screwed up iphone4 is No Fix )

  2. I bought a refurbished 3G a week ago and I upgraded straight to 4.1 because all application were claiming incompatibility. From a subjective point of view, iPhone takes its time to load things, above all searches, but it’s nothing you couldn’t live with. All animations are smooth, though.

  3. Thanks for the article, really informative.

    My question to you; if you were running 3.1.3 on a 3G, would you upgrade to 4.1 for the extra features? Is the performance different a reasonable payoff? Thanks!

    • For myself, I have two 3Gs, one is still running 3.1.3, and the other is running 4.1. Neither are being used as phones. On an app by app basis, most games appear to be just as responsive on both. Movies and music play back the same on both. So as a media device, 4.1 is just fine for my needs. If you are using the phone, sms, contacts, calendar, and mail (core apps), I will have to defer to those that are using the 3G in a similar manner. It appears that many that are still using the 3G as their primary phone have opted to stay on 3.1.3 a little while longer. If you have 4.0.2, on a 3G at least, I would recommend updating to 4.1. 4.1 does appear to perform better than 4.0.2 on the iPhone 3G.

  4. On my iPhone 3G, the 4.1 iOS is a big improvement on previous 4x versions. It feels stable and zippy. The battery performance is better on 3x but it doesnt seem as bad as in 4.02. The wifi performance and mail box improvements are worth the battery performance drop in my opinion. Overall, a great improvement on 4.02 for 3G owners.

    • I agree as well. Individual Apps authored by Apple all feel snappier. But unfortunately that is a subjective measurement. It is how I feel about the 4.1 update. I have seen several videos comparing one to the other online as well. The intent of this article was to look for objective measurements that would prove the same thing. My personal subjective perception of the 4.1 update is that is is a great improvement. The afore mentioned objective data does not support the same opinion to the same degree of satisfaction.

      • Amish thakrar

        Is anyone having a wifi issue after 4.1 update on iphone4, I’m having trouble connecting to my home and office wifi modem, it will only get connected after trying 3-4 times.

      • Amish Thakrar.. Right when I upgraded to 4.1 my WiFi started going On/Off/On/Off over and over.
        the phone itself worked smoother, but at the Expense of my WiFi..
        1st I Reset all my Network settings=no help
        2nd I JUST Reset my Whole Iphone= No Help
        .. I guess we ALL need to email apple and get them to fix it in the next year or so..
        … I so wish they’d FIX their new iphone4 antenna issue, then I’d give up my 3G.
        Otherwise I’ll have to wait for the newer iphone
        ( and NO, their bumper guard on their screwed up iphone4 is No Fix )

  5. nytroutbum

    Nice Job. Finally, some controlled tests with hard numbers. This has been sorely needed. I don’t have an iPhone, but have been struggling with iOS 4 on my 3g 32G Touch. Thousands of folks have complained about fast battery draining, overheating, wifi issues, and many incompatibilities. Apple has not even acknowledged there’s any problems at all. Of course, there’s no way back to 3.1.3. From your results, I think I’ll wait on Apple some more (nearly three months and counting). I’m on 4.02 and don’t want any further degradation of performance. As it is, I’ve had to disable all sorts of useful functions and charge throughout the day. A permanent loss of battery capacity is now noticeable as well. Any thoughts?

    • For me 4.1 on my old 3g is a big improvement. It feels faster, switching from home screen to home screen is more smooth and responsive. Some apps seem to load faster and backup/synch is much faster.

      It’s nowhere as fast or responsive as my iPhone 4, but still a noticeable improvement over 4.0.

    • I have been updating both my iPhone and iTunes (running on a Mac Pro) on a regular basis. Sometime earlier this year, with either an iTunes update, or when I replaced my 3G with a new iPhone 4, I noticed a significant improvement in backup times. My backups are not nearly as painful as they use to be.

    • Backing up regularly keeps your backup times shorter. However, when I went to backup to upgrade to iOS 4, I experienced the same problem with because I hadn’t backed up my 3g for a few months. It was taking over 2 hrs just to get about half way through the backup process.

      What I ended up doing was just resetting to factory defaults, then upgrading. That speed up the backup and synch process considerably.

      The drawback was that I had to manually add my apps and media (music, vids, ..) on my next synch. If you play games, you will also lose some data (such as levels reached …). You can also try restoring from a previous backup.

      If you have alot of new photos on the phone and you use a mac, use iPhoto to back them up/add them to your computer before you reset or restore.