More often than not, they’re conducted on highly optimized systems running workloads that don’t necessarily mirror what anyone actually runs in production. If the results in question come from vendors rather than third parties, there’s a good chance they’ve only been published because the vendor was able to achieve the desired result. Thinking your experience will be equally as fast is like watching a fishing show on television and then hitting the water expecting bite after bite.

However, cloud computing and the advent of popular open source software such as Hadoop and NoSQL databases could change the way we do benchmarks. With relatively little cost and effort, anyone can conduct their own tests to see how their specific applications and configurations run on their specific infrastructure. Throw in a platform to share these results, and you have crowdsourced performance benchmarks free from vendor hype and the vacuum-like conditions of standardized tests.

Ideally, it ends working a lot like the crowdsourced medical platforms I’ve come across lately, PatientsLikeMe and the forthcoming Lucine Biotechnology. Rather than rely on claims from drug companies or even doctors whose knowledge is limited to published research, users share their own real-world experiences with drugs, symptoms and side effects, and learn from others like them what they might expect.

It’s might get worse before it gets better

However, with so many Hadoop distributions in the market now, and so much money at play, I’d prepare to hear a lot more chest-beating in the months to come about whose implementation is actually fastest. It actually has been going on for a while – in December, I detailed a series of purported and contested records from SGI, MapR and HPCC Systems on the Terasort benchmark — and not all the voices have yet to be heard. Like SGI before them, hardware partners such as Dell, HP and Cisco probably want to prove their reference architectures are the best.

And VMware has already published a study claiming that Hadoop actually performs faster on its vSphere hypervisor than on bare metal. If Hadoop workloads really do move to virtual machines, Hadoop vendors are going to have to prove themselves there, too. VMware’s study ran CDH3 (Cloudera’s third-generation distribution), but Hortonworks has been working closely with VMware lately and might have something to say. Of course, EMC Greenplum is actually under the same corporate umbrella as VMware and can’t afford to be seen as slower than the competition on virtualized servers.

In cloud computing, too, providers have spent the past few years arguing against the idea of cloud servers as commodities by claiming their systems offer the best performance. There have been plenty of boasts (sometimes, perhaps, misleading) and quite a few attempts to benchmark cloud system and network performance (GigaOM Pro subscription req’d) With that much of the cloud market still up for grabs, we’re not yet done hearing about whose cloud is the biggest and the fastest (see, for example, Google’s emphasis on performance when it launched Compute Engine last month).

But it should get better

Despite all the effort by vendors and cloud providers to claim superiority, though, the truth is that it’s easier than ever for users of next-generation software and services to run their own tests. Hadoop or NoSQL databases aren’t expensive Oracle software that needs to run on scale-up, big-iron systems; they’re all free to download and can run on small clusters of commodity boxes. For applications that are going to run in the cloud, renting a few instances from a cloud provider might only cost a few bucks.

Sure, it might take a little time to configure everything (although software vendors might be willing to help), but isn’t that effort worth it in the end?

Use cases and test results that demonstrate the value of crowdsourcing in-the-wild performance metrics are everywhere on corporate technology blogs across the web. Earlier this month, for example, Medialets discussed how it tested its Hadoop workload (on a cluster of rented physical machines) and found that Cloudera actually outperformed the supposedly faster MapR for that job. This week, video-transcoding service Zencoder shared some interesting (if not always surprising) results when it compared Amazon Web Services’ highest-powered cloud instances to Google Compute Engine’s best.

Aggregated and indexed on a single platform, these types of experiences could help quiet the boasts from vendors and industry organizations touting their latest benchmark results. I’d argue it matters a lot more to a systems architect to know the production throughput of someone running a similar application on similar resources than to know how fast a generic workload ran in a lab on a setup he doesn’t have. Add some analytics to this information, and ideal configurations for different application types and data volumes might begin to emerge.

Cloud computing and open source software have freed IT practitioners from so much legacy vendor baggage over the past few years. Isn’t it time to free them from inane benchmark boasting, too?

Feature image courtesy of Shutterstock user Suzanne Tucker.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Benchmarking the Cloud: Your Mileage May Vary
• Infrastructure Q1: Cloud and big data woo enterprises
• A near-term outlook for big data

iOS 5 update tested

Once again, I used GeekBench 2 ($0.99), Gauge Mathematical Tool (free), BenchTest ($0.99), SunSpider (free) and V8 (free) to test the performance of each OS version on each device. I tested the original iPad 1 Wi-Fi, the iPhone 4 (GSM), the iPod touch 4th Gen, and the iPad 2 Wi-Fi+3G (GSM).

Conclusions

If you use the web more than anything else on your iOS device, then you should be eager to apply each and every update that Apple puts out, since they all improve Safari’s performance every time. iOS 5 is no exception in that regard.

Considering all of the new features you get with iOS 5.0, the hit on performance in other areas is in most cases negligible.  In fact, with the exception of the original iPad (the older device that was tested), all devices actually gained in performance in most areas. That is simply amazing for the first release of a major OS update. This is nothing like the 4.0 release, where we saw incremental performance gains over the lifetime of 4.0 to restore what was lost to older device; iOS 5.0 is a solid release for all where performance is concerned.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How scribbling on an iPad makes your work life easier
• Tablet market to hit over 377 million units by 2016
• 2012: Data, spectrum and the race to LTE

iOS 4.3 Update Tested

The only concern I had was that the benchmark apps were all iPhone apps, not true iPad apps, but it shouldn’t affect the overall results.  I used GeekBench 2 ($0.99), Gauge Mathematical Tool (free), BenchTest ($0.99), SunSpider (free) and V8 (free) to test the performance of each OS version on the iPad. I performed all the tests on the same device, both before and after the upgrade.

Results for the Wi-Fi iPad

For the most part, the test results showed that performance was slightly better on iOS 4.2.1 than on iOS 4.3 on the iPad.  But overall, the results were close enough to consider the upgrade to iOS 4.3 on the iPad simply worthwhile.  That is, until you look at the JavaScript test results. Mobile Safari on the iPad was significantly faster on iOS 4.3.  The only problem I ran into was that the Google V8 JavaScript tests would not complete on the iPad.  The test would either hang or crash mobile Safari.  I had a similar issue on the iPhone 3G when trying to execute the same tests.

Results for the iPhone 4

While I couldn’t upgrade my iPhone 3G, I could apply the latest update to my iPhone 4 for comparison to prior results and see how it was affected by by the upgrade.  Unlike on the iPad, I’ve been maintaining benchmark results on my iPhone 4 for some time now, so I have some historical data to compare the latest tests to:

Over time, you can see where the performance of iOS on the iPhone 4 has slowly degraded with each release. 4.3 is no exception to this trend.  Keep in mind that this performance degradation is not nearly as dramatic as that which occurred with the iPhone 3G.  Note also the continual increase in performance noted in the JavaScript performance with each release.  The 4.3 update is the most significant yet in that regard on the iPhone 4.

Conclusions

For what you get in the form of features withe the iOS 4.3 update, and for the almost insignificant general performance hit on the iPad, the upgrade is overall a great one.  Any time you get more features for little or no degradation in performance is a good thing.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A look back at mobile in Q1
• Flash analysis: Steve Jobs
• Mobile Q2: Smartphone growth surges; iPad’s rule continues

CNET ran the iPad 2 through the SunSpider benchmarking tool, which measures Javascript performance. It found that the iPad 2 had the best overall scores of devices it tested, with a score of 2097. The original iPad running iOS 4.2 scored 8321, and lower scores are better when testing with SunSpider. The iPhone 4 on iOS 4.2 scored 10414, and the Nexus S and Galaxy Tab had scores of 6128 and 7066 respectively.

In addition to the iPad 2′s dual-core A5 processor, a lot of the increase in performance can be attributed to changes made to the iPad’s operating system in iOS 4.3. CNET also ran the SunSpider test on an original iPad and iPhone 4 running iOS 4.3′s latest beta, and those numbers showed huge improvements. The original iPad on iOS 4.3 scored 3261, and the iPhone 4 scored 4151. That’s great news for users who aren’t planning to upgrade, since you should experience much snappier browsing even with just a software update.

Android fans shouldn’t feel too badly about the poor performance of the Galaxy Tab and Nexus S, since SunSpider tests performed using the upcoming Galaxy Tab 10.1 found that it scored 2256, which is nearly on par with the iPad 2.

Bottom line, Apple mobile device users are about to get yet another improvement to the overall iOS experience, with or without the hefty price tag of buying new hardware.

Related content from GigaOM Pro (sub req’d):

• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• App Developers: Are You Ready for HTML5 and Metered Data?
• How to Market Your iPhone App: A Developer’s Guide

Geekbench 2, the latest version of Primate’s testing software, only measures processor and memory performance, and doesn’t take into account video card performance. Results are drawn from the Geekbench user community, and higher scores are better. Partial results are shown below; click on the image to see the full results.

As you can see in the full results, the new MacBook Pros outperform the previous models by a wide margin, which is to be expected when jumping from a dual- to a quad-core configuration. Note that even the 13-inch MacBook Pro outperforms the previous 15- and 17-inch Pro powerhouses released in early 2010.

The new notebooks even outperform some current Mac Pros, according to Primate’s general Mac benchmarks: definitely an impressive display of raw muscle from a portable computer.

That said, the ultimate test of any computing device is whether or not it meets your actual usage needs. Accordingly, I’m still inclined to wait until the next MacBook update rolls around. Do these performance scores influence your decision one way or another?

Related content from GigaOM Pro (sub req’d):

• Transient Apps: The Consumer Influence on Enterprise Mobility, Part 2
• Why Apple Hasn’t Sewn Up the Tablet Market — Yet
• Rogue Devices: The Consumer Influence on Enterprise Mobility, Part 1

iOS 4.2.1 Update Tested

To test how iOS 4.2 affected performance, I took an iPhone 3G and ran it through the same series of performance tests I did when iOS 4.1 was released. Just like last time, I used GeekBench 2 ($1.99), Gauge Mathematical Tool ($1.99), BenchTest (99 cents), SunSpider (free) and V8 (free) to test the performance of each OS version on the 3G.

Also like last time, the 3G I was testing was still running with a lot of features disabled, and was being used more as an iPod touch than as an iPhone. I am reusing the same data collected last time for comparison. The following results are only a subset of the data collected, and focus on the data points that changed the most from version to version:

Results for iPhone 3G: The 4.2.1 Update

While some things seem to have improved slightly, like JavaScript and drawing new content when you switch views in an app, other tests have dropped off considerably. Memory performance as measured by GeekBench, as well as the executions of Gague’s Prime Number test are especially bad. These would affect apps that are doing much more than just displaying or editing a simple screen, like an RSS reader, for instance.  It would have a greater impact on apps performing more advanced computational functions, such as Documents to Go.

Just informally trying out several core apps like Apple’s Notes, Safari, Calendar and Contacts revealed more than the benchmarking tests indicate, however. Scrolling speed and data entry appear to be much improved. It seemed as though my performance tests were contradicting my real world experience.

One More iPhone 4 Test: The 4.2.1 Update

Testing on the iPhone 4 was the best way to find out if the performance degradation I was seeing on the iPhone 3G in the tests was really something to worry about. I reused the same benchmarking data collected in earlier tests for comparison. I was looking for the same degradation in performance in the same areas.

Since both the GeekBench Stdlib Allocate test as well as the Gague Prime Number test degrade on the iPhone 4 as they did on the iPhone 3G, I had to conclude that it was an issue with how the test was seeing changes made to the iOS firmware in version 4.2.1, and not necessarily a reflection of actual device performance.

Conclusions

While data entry, scrolling, pinching and zooming have improved slightly, the 3G is still not up to par with its newer siblings. Where we see the most gains on both the iPhone 3G and the iPhone 4 is in the JavaScript results.  Whatever the Webkit/Safari team is doing, it is having a positive effect with each release.

Bottom line: If you’re still using iOS 3.1.3 and use sophisticated apps that are heavily computational, then it’s probably best to avoid 4.2 (that’s easier if the apps you’re using don’t require 4.0 or higher). If you’re content with the apps Apple provides and mobile Safari, you should be fine. If you need the more demanding apps, and those apps need iOS 4.0 or higher, it might be time to consider that hardware upgrade.

Online Benchmark Results

• GeekBench 2 – iPhone 3G running iOS 4.2.1
• SunSpider – iPhone 3G running iOS 4.2.1

Related content from GigaOM Pro (sub req’d):

• App Developers: Are You Ready for HTML5 and Metered Data?
• HTML5’s a Game-Changer for Web Apps
• How to Market Your iPhone App: A Developer’s Guide

Thinner, yes. But better?

I am one of the few, proud MacBook Air (MBA) owners in the world. Every article you’ve seen from me in the past was written on the petite beauty. In one last month, I wrote:

I’m only here to make a case that anyone that hasn’t used one should give it a shot and that Apple needs to throw some marketing dollars behind it so everyone else knows what we MacBook Air owners have known for a long time: It’s the best notebook Apple makes and it also happens to be the smallest.

This is exactly what Apple did yesterday, and yet, I was severely disappointed. It seems as if Apple is finally throwing marketing dollars behind the MBA, and I watched my Twitter stream as thousands of geeks’ heads exploded at how thin it is. But I felt a little weird reading this on my MacBook Air and thinking out loud, “I know. It’s been that way for years.” It’s thinner, yes, but not significantly. That’s the whole story in a nutshell.

Pricing & Marketing

On the whole, the reason Apple’s MacBook Air wasn’t a hit before now was price. Imagine the iPad started at $999. Fewer people would own one, and those that did and sang its praises it to friends would be immediately be greeted with the response that it was simply too expensive for a tablet. Basically, Apple just dropped the price. The MacBook Air of today is largely unchanged from what we had before, and that’s disappointing.

11.6″ Model Versus the Sony TZ (circa 2008)

Consider Engadget’s live-blog from Macworld 2008, when the Macbook Air was introduced. Steve initially compared the MacBook Air to Sony’s TZ series. His biggest gripe was that the processor speed maxed out at 1.2GHz Core 2 Duo, and the display was only 11 inches.

The 11.6-inch MacBook Air is “too cramped” and “too slow” according to the Steve of 2008. To be fair, the Air isn’t the same as the TZ in every way. I haven’t used the new keyboard, but Apple claims it’s full-size and the body is still a bit thinner.

Let’s Talk Speed

Current MacBook Air owners won’t see a significant speed bump if they own the Revision C model (1.8/2.13GHz) released last year. In fact, not much has changed about the machine at all.

MacBook Air Revision C (Jun. 2009)

• 1.86 ($1499) or 2.13GHz ($1799)
• 3.0 Pounds
• 6MB of L2 Cache w/ 1066Mhz Frontside Bus
• 120GB HDD or 128GB SSD
• 2GB RAM (DDR3-8500 at 1066Mhz)
• 13.3-inch screen w/ 1280×800 resolution
• NVIDIA GeForce 9400 w/ 256Mb of VRAM (shared with main memory)

MacBook Air Revision D (October,2010)

• 1.86Ghz ($1,299) or 2.13Ghz ($1699)
• 2.9 pounds
• 6MB of L2 Cache w/ 1066Mhz Frontside Bus
• 128GB SSD or 256GB SSD
• 2GB/4GB RAM (DDR3-8500 at 1066MHz)
• 13.13″ screen w/ 1440×900 resolution
• NVIDIA GeForce 320m w/ 256Mb of VRAM (shared with main memory)

Of course, I’m only comparing the 13-inch models, but you’re getting a machine that’s basically the same speed with an option to add more ram and more storage,  plus a high-resolution screen which isn’t something I’d recommend to everyone. Some (especially those over 50) will prefer a lower resolution. Even I had trouble using a 17-inch MacBook Pro and eventually sold it.

What you are getting is a drop in price. Last year, a 2.13GHz machine with 2GB of RAM and a 128GB SSD would cost $1799. Today, that same machine costs $1699 with double the storage, and it’s only $100 to upgrade to 4GB of RAM.

Real World Performance

Will HD Flash movies on YouTube perform better? No. Will Photoshop CS5 filters apply faster? Maybe with a RAM upgrade. Nothing about this upgrade is impressive if you’re a current MacBook Air owner looking for more speed.

Why can’t Apple squeeze more juice out of this machine? Apple didn’t talk speed at all. They didn’t throw their famous “2x Faster” graphic up on the “buy now” page. Apple does tout the 2.9x faster graphics card, but I can guarantee you the Air released yesterday is only marginally faster than last year’s model, despite the introduction of the 320M, because that’s a video card using shared RAM, meaning the graphics performance goes down as you open more applications.

I was underwhelmed by today’s announcement, but the MBA was already a good notebook. I was hoping Apple would make it even better. It didn’t. “One more thing” was simply a minor evolutionary product refresh, and that’s too bad.

Related content from GigaOM Pro (sub req’d):

• Mobile Operators’ Strategies for Connected Devicess
• Strategies for the Future of Digital Content Storage
• Company Profile: Apple

So has the recently released 4.1 update finally closed the door on the issue of iOS 4 running sluggishly on the iPhone 3G?

iOS 4.1 Update Tested

To test how the iOS 4.1 affected performance, I took a pair of iPhone 3Gs and ran them through a series of performance tests. I used GeekBench 2 ($1.99), Gauge Mathematical Tool ($1.99), BenchTest (99 cents), SunSpider (free) and V8 (free) to test the performance of each OS version on the 3G.

I started out with one iPhone 3G running 3.1.3, and the second running 4.0.2. I must admit that both 3Gs were dumbed down quite a bit, and were not being utilized as iPhones, but more like iPod touches. The iPhones used were purchased at the same time, shipped in the same shipment, and have very similar Serial, IMEI, Wi-Fi and Bluetooth numbers assigned to them. The following results are only a subset of the data collected, and focus on the data points that changed the most from version to version:

Results for iPhone 3G: 3.1.3 vs 4.0.2

It’s not what the primary results did say, it’s what they did not say. As I executed the tests side by side, I noticed that even when similar end results were returned by the benchmark tests, the UI of each iPhone would behave differently. The 3G running 3.1.3 would behave as expected, but the 3G running 4.0.2 did not. Progress bars wouldn’t update smoothly on 4.0.2, but would jump to 100 percent at the end of the test. As tests were executing, it was as if the user interface on 4.0.2 was somehow sporadically frozen during the tests. This behavior is consistent with other video reports online that iOS 4 is slow on 3G devices. It also explains why certain activities, namely games that don’t utilize Apple’s stock UI components, run pretty much the same on each OS version. If you now use your 3G primarily as a game device as I do, you may not have noticed all the differences in performance.

If, on the other hand, you’ve continued to use the device as a smart phone, you’ve most likely noticed the degradation in performance. What was surprising at first was that the JavaScript results were much faster on iOS 4. This makes some sense, since the version of Safari on iOS 4 most likely has the latest JavaScript engine under the hood. So while some UI elements may be slower, and most hardware-based benchmarks remained relatively constant, the JavaScript results actually improved from 3.1.3 to 4.0.2.

Results for iPhone 3G: The 4.1 Update

I then upgraded the 3G running iOS 4.0.2 to the new iOS 4.1. At this point, I had one iPhone 3G running iOS 3.1.3, and the other running iOS 4.1. This time around, the UI did appear to behave more as one would expect on 4.1, with smoother progress bars and a zippier response from the UI. It felt faster, but was it as fast as 3.1.3? No, not really.

While certain numbers did seem to bounce back, namely the performance of the  Stdlib Allocate test within the GeekBench test suite, most numbers remained the same as in 4.0.2. The BenchTest results for Drawing Into View and File to Filesystem did seem to bounce back a little as well. It was only the JavaScript results (as per the SunSpider test) that really seemed to improve even further. However, it’s important to note that the iPhone 3G (on both versions of the OS) never passed the V8 test. Safari always crashed before completing the test.

One More Test: iPhone 4 on 4.0.2 vs. 4.1

So before drawing any conclusions, I wanted to see if the subtle performance increases noticed between 4.0.2 and 4.1 on the 3G were the same performance increases noticed on the new iPhone 4. Finding the same performance increases on the iPhone 4 as compared to the iPhone 3G would point to an overall performance boost based on OS updates that weren’t specific to any one hardware platform. Luckily, I happen to have two iPhone 4s in the house as well. This time, I performed benchmarks on the exact same hardware before and after the upgrade. I was very surprised with the results of the performance tests on 4.0.2 compared to 4.1 on the iPhone 4.

Conclusions

It’s still very likely that Apple did focus on performance issues related to each one of its own internal apps that are distributed only via OS updates. Apart from the JavaScript benchmark results — which prove that Safari on iOS 4 outperforms Safari on iPhone OS 3 — the boost to overall performance based solely on the test results listed above is marginal.

These tests, however, are looking at the OS itself, not individual app performance improvements. Surprisingly, there does appear to be a platform-specific focus on performance issues related to the iPhone 3G, as performance does appear to have degraded on the iPhone 4 with the 4.1 update. I’d been focusing heavily on the iPhone 3G, and had run several iterations of tests comparing 3.1.3 to 4.0.2, and was prepared to collect the same amount of data following the 4.1 update on the iPhone 3G. I was much more meticulous with the details on executing the tests.

The tests against the iPhone 4 were an afterthought, and would require a more through examination before claiming that the iOS 4.1 update did indeed degrade the performance of the iPhone 4. Just as we struggled through the iPhone OS 3 updates on the iPhone 3G last year, there will likely be more updates to iOS 4 in the months to come. While I still don’t feel iOS 4.1 is back to the same performance levels of iPhone OS 3.1.3, there do appear to be performance gains in 4.1 that prove that Apple is serious about supporting the iPhone 3G on the iOS 4 platform.

Online BenchMark Results

• GeekBench 2 – iPhone 3G running iPhone OS 3.1.3
• GeekBench 2 – iPhone 3G running iOS 4.0.2
• GeekBench 2 – iPhone 3G running iOS 4.1
• SunSpider – iPhone 3G running iPhone OS 3.1.3
• SunSpider – iPhone 3G running iOS 4.0.2
• SunSpider – iPhone 3G running iOS 4.1

Next Gen Customers on Previous Gen Devices

The most likely recipients of older iPhone 3G devices will be the siblings of the owners of the devices which have replaced them with the new iPhone 4. This is a very important class of user that Apple cares a lot about. The new owners of these older iPhone 3G devices will be potential Apple customers throughout their academic life. And it appears that students are serious about Apple. Providing a less than awesome experience now could jeopardize their loyalties later.

One iOS to Rule Them All

The current burden on developers targeting Apple’s mobile devices is that there are too many software versions to manage; 3.1, 3.2, and 4.0, to name a few. The number of devices on each of these three major platforms is still significant. With the upcoming iOS 4.1 update, Apple has the opportunity to converge more of these devices to just one OS platform. That is, if they can convince iPhone 3G owners that performance is no longer an issue.

Focusing on multiple platforms does take time away from adding features and releasing new titles. Giving developers the tools they need is only part of the equation. Switching between simulators and managing several different devices takes time. Cutting down on the number of device and OS combinations that developers need to support will only strengthen the quality of the apps for devs that don’t test across the matrix of combinations, and give back some development time to those that do.

Engineering Pride

Apple has been quite public about the engineering capabilities of other software development shops. Apple was also adamant all along that iPhone 3G devices would be supported on iOS 4, minus some features. Nonetheless, Apple proclaimed support for these devices. Taking on its own challenge, stepping up to the plate and delivering an awesome experience to its next generation of customers is what Apple’s core values are all about.

Related GigaOM Pro Research: Rogue Devices: The Consumer Influence on Enterprise Mobility, Part 1

The architecture of the new iPhone 4 places the antenna structure within the stainless steel band that wraps the new phone. And the problem many users have noted is that applying normal pressure along specific points of the phone causes the signal quality indicator to drop. Seems like just a hardware issue, right? Well before you go old-school and wrap your phone in aluminum foil in a desperate attempt to boost the signal strength, there’s more to the story.

The issue of signal strength has been picked apart across the Internet; some people can reproduce the problem, while others cannot. A few days after these issues first came to light, Steve Jobs said in response to an email:

Gripping any phone will result in some attenuation of its antenna performance, with certain places being worse than others depending on the placement of the antennas. This is a fact of life for every wireless phone. If you ever experience this on your iPhone 4, avoid gripping it in the lower left corner in a way that covers both sides of the black strip in the metal band, or simply use one of many available cases.

It’s an interesting response, but it seems to speak to Apple’s industrial design decisions, since such sensitive hardware is in a region of the phone that’s naturally likely to be handled. Regardless, since Jobs’ comments, there have been daily rumors concerning Apple’s solution to the issue, whether it’s a software update to “fix” the problem or providing customers with an iPhone 4 bumper case that avoids placing pressure on the antenna. Most recently, fabricated emails have come to light suggesting that Jobs told a customer “calm down” and that it’s “not worth it.” While that turned out to be a fake email according to Apple, the company finally released a statement regarding the issue altogether.

According to the statement, the algorithm for calculating the signal strength has just been inaccurate, showing more bars than it should in some cases. In an example provided by Apple, what is displayed on your iPhone could be two bars higher than the actual signal strength. The statement also suggests your real signal strength never changes, so when you see the lower signal strength as a result of placing pressure along one of the antennas, you’re really seeing a more accurate representation of the signal.

Okay, so it’s a software issue. Or is it?

Apple says this problem of inaccurate signal strength has been present in every iPhone since launch. (Thanks, Apple!)

So if it’s an issue of inaccurate signal strength, how does that affect performance? Look at the video that Cameron Hunt posted to Vimeo that shows how Safari simply stops loading when he touches one of these antenna points along the device. If you apply Apple’s logic to his scenario, when you watch the bars begin to drop, his actual signal strength shouldn’t be dropping. Yet it does, because Safari cannot finish downloading the page. Clearly, there’s still some degree of a hardware issue involved.

Apple says a free software update for iPhone 4, iPhone 3G and 3GS users will be available in the next few weeks to address the problem, and will cause your iPhone to display a more accurate signal strength. Additionally, the update will make the first three signal bars “a bit taller” and “easier to see.” Just remember, they’re only taller to make them more visible, not because the signal strength is any better.

For the technically minded out there, it’s been mentioned that the field test mode in iPhone 4 has disappeared. That’s too bad, as it would have been a great way to see what’s really happening to the signal strength. Does anybody know how to access it on the new iPhone 4?

Do you really believe Apple’s response that the issue is software-related and the reality is that the network reception is actually much lower than what your iPhone displays? Is there really anything wrong? Or is nothing wrong? Or is there something wrong but the problem is just normal of cell phones and it’s just time for us Apple users to drink the Kool-Aid again? Share your thoughts in the comments.