Apple Utilities: Your First Step

Regardless of what problem you’re having, your first step should be to try one of Apple’s own utilities. Apple provides several free Mac utilities in the OS. Apple Hardware Test (AHT) is included with every new Mac. The AHT boot CD bypasses your operating system and tests basic hardware such as your hard drive, memory, processing, and video. The tests are very basic and a passing  grade doesn’t always mean all is well. Apple sees AHT as a quick test to rule out obvious problems. If your Mac is just acting weird, AHT is a quick and easy place to start your troubleshooting journey.

Another Apple utility is Apple’s Disk Utility which can be found inside the Utilities folder (which is in your Mac’s Applications folder). The app is also on your OS X installation disc. Disk Utility’s is primarily for screening drives for serious problems like disk file structure errors (things are out of place), SMART failures (early warning signs of a problem), or permissions issues. For some repairs, you’ll need to boot from your installation disc. Apple’s provides extensive instructions on how to use Disk Utility on its support website.

General Disk Maintenance and Basic Repair

For basic hard drive checks and repair, the closest equivalent today to the power and simplicity of the original Norton Disk Doctor is Alsoft’s Disk Warrior ($99.95). Disk Warrior primarily does one thing and does it very well: it repairs disk directories.

All the cryptic and confusing errors Apple’s Disk Utility has problems with, Disk Warrior fixes with flying colors. Your disk’s directory is similar to a table of contents crossed with an index, and when it gets confused (usually due to program crashes or improper machine shutdowns), Disk Warrior rebuilds (instead of repairs) the directory, allowing the computer to operate normally.

Disk Warrior can’t repair the drive it’s installed on, so you’ll have to run it off of the installation DVD or another volume if you want to repair your boot drive.

Disk Warrior should be a part of any basic troubleshooting routine and I always run it before any system updates. It’s my most trusted and relied upon Mac Utility.

Data Recovery

At the first sign of disk failure, you need to consider whether or not you’ll be sending the drive off for forensic-style data recovery from a company like DriveSavers.  If you think you’ll be using professionals, then stop all recovery efforts and call them right away. Although these services often cost thousands of dollars, sometimes it could be covered by insurance.

If you want to go the do-it-yourself route, my first choice is Stellar Phoenix 4.1 ($79). The program is extremely powerful and deals best with volumes that won’t mount on your Mac. It pushes right through read errors and recovers data that other programs can’t find. Some recovery applications may give up if they encounter excessive read errors, which are common on dying hard drives, but Stellar Phoenix will keep trying until it recovers whatever data it can. Unfortunately, the interface is confusing and unintuitive like. In spite of the bad UI, however, the program itself is very reliable.

Hard Drive Optimization

Drive Genius 3($99) by Prosoft Engineering, has a comprehensive suite of hard drive utilities attractively packaged with a easy-to-use interface that lets you tweak every possible byte of performance from your storage device.

Drive Genius includes not only those functions covered by Disk Utility such as secure erasure, initialization and partitioning, but also adds a slew of maintenance and optimization routines. For those using a smaller hard drive, their proprietary DriveSlim algorithms removes unnecessary files to add space.

Although not necessary for the average user, Drive Genius also includes a defragmentation option, along with advanced functions for sector editing, benchmarking, cloning and an early warning system for potential problems called “Drive Pulse.” Drive Genius is used by Apple’s own Genius staff for yearly checkups. I recommend Drive Genius for more experienced users and geeks who want to really explore their hard drive.

Hardware Problems

TechTool Pro 5 ($99) by Micromat goes way beyond the AHT, doing more extensive testing of your memory, hard drive, audio and video. Apple includes a limited version of TechTool called TechTool Deluxe with the purchase of AppleCare that provides a limited subset of the full version’s hardware tests.

A unique part of TechTool is the “eDrive” feature, which creates a bootable partition on your current hard drive in order to run its utilities in case of emergency. More technical users probably would be more comfortable using an external hard drive, but rest of us will find this eDrive very useful for basic maintenance and testing. eDrive and most of TechTool’s other hard drive tools aren’t included with TechTool Deluxe, representing the major difference between it and the full Pro product.

Which should you buy?

Overall, every Mac user would benefit from owning a copy of DiskWarrior for periodic maintenance. If you aren’t fastidious about backing up (or work with others who aren’t), Stellar Phoenix can really save your data and I suggest it as your primary recovery tool, but you might be able to wait until something actually goes wrong. For the more technically inclined, I’d recommend TechTool Pro over Drive Genius because it includes both optimization and hardware testing. If all you want to do is occasionally optimize your hard drive and save space, owning just Drive Genius is a great choice.

While one program doesn’t do it all like in the days of good ol’ Norton, this list of programs will provide you the key to solving some of your own computer problems.

Related content from GigaOM Pro (sub req’d):

• Privacy: How to Avoid the Third Rail of Online Services
• The Internet of Things: What It Is, Why It Matters
• Are the Stars Aligning for an Amazon PaaS Offering?

Today it’s the foundation for all modern pre-installed Windows systems. It replaced the aging FAT (File Allocation Table) file system with a leaner, meaner standard. It’s more secure, more flexible, and has more built-in gadgets than a Swiss Army Knife. It’s NTFS, of course — the New Technology File System.

I grew up with NTFS. I was there when it was introduced with Windows NT, and I insisted on reformatting every Windows machine I ever got my hands on — wiping out stuffy old FAT32 — with a file system so much more capable and pleasing. Yeah, OK, I was total nerd about the whole thing, I know.

So it came as something of a surprise to me that I didn’t think about file systems when I was switching to the Mac. And I didn’t think about file systems when I took a terabyte hard drive from my old PC and put it in my Mac Pro. And I didn’t think about file systems when I first clicked around that drive’s contents in Finder.

I did think about file systems when I tried to save changes I made to an old document that migrated to my Mac along with the drive. I was rewarded for my Saving efforts with an unwelcome dialogue telling me I couldn’t write to the drive. And then the penny dropped. My big, new shiny Mac Pro was using HFS+ while that second HDD was formatted with NTFS. And then I had to go and have a lie down for a bit due to an acronym-induced headache.

You’re unlikely to have experienced this access problem unless you have an external drive you like to use for both a Windows machine and a Mac.

Until Apple gets around to building full read/write compatibility into its operating system (and let’s not forget Snow Leopard is right around the corner) we have to find third-party solutions that cajole our hard drives into getting along with each other. One popular (and free) solution is NTFS-3G: a reasonably simple, though somewhat fiddly, application that requires users mount their NTFS drives/partitions in order to have full read/write access to the data stored therein. This can become cumbersome if an external drive is improperly unmounted on another machine. It’s not impossible to fix, mind you, but not effortless by any means.

If you need a simpler solution that offers transparent, and deceptively native, interoperability with NTFS formatted drives, you could try Paragon’s venerable (though horribly named) NTFS for Mac OS X 7.0 (or NTFSMOSX7 as I shall now refer to it because that string of characters is marginally less cumbersome than the full name).

NTFSMOSX7 is a small download, at a little under 3 megabytes and, once installed, ticks away in the background. There’s no user interface or settings to tweak. This really is nothing more than a driver.

If you buy the full version of the software, the download will undoubtedly be larger, since the paid-for app includes Paragon’s MacBrowser software for Windows. I might have tested MacBrowser for the purpose of this review, but Paragon never responded to my emails requesting more information, so if you have used it, and have an experience to share, please add a comment at the end of the article.

The only real problem I had once the hard drive was once again mine to use and abuse, was a conspicuous lack of Spotlight functionality. This is because, by default, Spotlight doesn’t index NTFS volumes. It’s not impossible to do, but it’s also not easy to achieve. The Spotlight preferences panel is typically spartan, so it’s necessary to bypass the Spotlight settings GUI and get your hands dirty in a Terminal session.

For those of you interested, by the way, the syntax for manually adding an NTFS volume (or any volume, for that matter) to Spotlight’s index via Terminal goes like this:

sudo mdutil -i on /Volume/VolumeName

Once entered, Spotlight will begin indexing your volume, which means it’s time for you to go watch some “Galactica” or “Big Bang Theory” while you wait.

A lot has been made of the apparent (and the not-even-remotely-important) differences between NTFS and HFS+. While HFS+ was ahead of the game at a time when too many Windows users were still mired in FAT (no jokes, please), NTFS was, and remains, a sophisticated file system. In fact, it bests HFS+ in many ways. Overall, though, is it better? I don’t know for sure, and I suspect for 99 percent of everyday users the question “who cares?” springs to mind.

Sidenote: Exuberant Apple supporters — aka Wailing Fanboys — muddy the waters with their auto-hate for anything not intrinsically Apple, claiming time and time again on discussion boards that “NTFS sux.” Occasionally, a particularly tech-savvy zealot will talk about comparing streams and counting data forks and probably even get into a debate over which Enterprise Captain is the best.

None of that matters so long as you can access your data quickly and painlessly, and, at least on the Mac, NTFSMOSX7 makes that possible. If you have to use NTFS volumes regularly but can’t reformat them to use HFS+, the $40 asking price for NTFSMOSX7 is perhaps not so steep in return for the convenience of seamless data interoperability. If, on the other hand, you want occasional quick-and-dirty access to an old NTFS volume and the ability to write data to it is not a serious concern, $40 is much too expensive for what amounts to a simple driver, and I’d suggest you stick with a free alternative.

You can try NTFSMOSX7 free for 10 days by downloading it from the Paragon web site. You can see which Enterprise captain is superior by clicking here.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The Near-Term Evolution of Social Commerce
• A 2011 Connected Consumer Forecast
• A 2011 NewNet Forecast

In early December of last year, the folks at Western Digital released some new USB/Firewire “Turbo” drivers for their My Book line of external storage devices. There was no mention of this news, nor was there a press release issued from the company.

For my own storage purposes, I own a My Book Pro II (1TB) drive. I use it to backup my audio (iTunes) and video (Sony HD Camcorder) content. This drive is connected to my iMac via FireWire 800.

When I stumbled upon the Turbo Drivers a month ago, I decided to take the risk and install them. There is no readme included with the installer, although from the download page, here is the information Western Digital provides:

“This installer – WDMacTurboDrivers.dmg, contained within the WDMacTurboDrivers.zip file installs custom Western Digital performance based Turbo drivers and drive icons for WD USB and FireWire enabled external hard drives under Macintosh computers.”

So what happened after I installed the drivers? Other than the icon changing (see below), I can’t see any performance benefits. After recently performing a search on the Western Digital support site, here is what I found:

Question:

Why is there no performance gain using the WD Macintosh Turbo Drivers for a My Book Premium II, Pro II, or Studio II while connected by FireWire?

Cause:

The My Book Premium II, Pro II, and Studio II Edition hard drives use a Macintosh based performance driver already included in the Macintosh Operating System. This Macintosh based performance driver has the same performance as the WD Macintosh Turbo Drivers.

Resolution:

There is no resolution for this as there is no performance gain or loss due to the usage of either the Macintosh based performance drivers or the WD Macintosh Turbo Drivers.

OK, who isn’t confused by this explanation from Western Digital? What is the value of creating these custom “turbo” drivers, other than changing the icon? Is it just for branding? Why did Western Digital spend the software development cost to implement a useless driver? In these economic times, I would think that Western Digital would be more cost conscious.

By the way, here is the visual change for the disk icon:

Before:

After: 

Impressed by the visual change? I would hope not, but then, we all have different aesthetic values.

Overall, I am pretty disappointed by this offering from Western Digital. My expectations were that I would see improved read/write performance from this driver. Instead, and clearly stated from Western Digital, there is none.

For what it’s worth, I didn’t do any formal speed tests comparing the standard Apple FireWire driver vs. the new Western Digital Turbo driver. I mistakenly fell into the marketing trap of “ooh-faster-from-company-must-be-better-than-Apple-nonsense.”

Thankfully, I have neither seen any impact to my overall system stability, nor any negative impact to the My Book Pro II drive itself. If anyone else has tried these drivers, I would love to hear your feedback and see if you have received any benefit from installing them. Otherwise, you can thank me for being your guinea pig.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce

You’re running out of disk space? You want to clean your disk to free up some space and get rid of the clutter? Sponge is a tool to keep your Mac clean and tidy.

Features

Sponge combines three different techniques to clean up your Mac: removing applications, locating the large files on your drive and finding duplicates.

• Removing applications: Sponge generates a list of all the applications installed on your system, how much space these applications occupy, and their associated files and folders. Sponge then provides the facility to remove applications safely from the system.
• Sweeping disk hogs: Sponge will show which files and directories take up the most space, giving you a starting point for cleaning some of the clutter from your Mac. Once you’ve tracked them down, Sponge can conveniently swipe them off your disk.
• Finding duplicates: With all the attachments, memos, letters and other documents with which we are flooded in today’s world, it’s easy to lose track and accidentally save duplicates of files. Sponge makes finding these multiple copies easy.

Likes and Dislikes

The aspect of Sponge I’m most impressed with is that the creators have thought to put these three tools together in one piece of software. I commonly use similar software to sweep my disk and remove applications, and have never thought to perform an in-depth search for duplicate files before now. Finding duplicates works well and has a simple interface for determining which file is the most up to date.

Unfortunately, I haven’t completely fallen for the application uninstall feature. One main issue is that Sponge generates a list of all your installed applications — this takes time, and limits the choice of apps to uninstall. I’d prefer to be able to point to a file/application and have Sponge find all the associated resources to remove.

Finally, the ‘Sweep Disk Hogs’ tool impressed me with its speed. Previous applications I’ve used for this functionality have scanned the entire hard drive. Sponge allows you to drill down folder-by-folder, making it considerably quicker. It isn’t as easy to get a picture of your whole drive but is still straight forward to find large, cumbersome files.

Competing Utilities

Some other utilities which can perform similar functions to Sponge include AppZapper, a software uninstallation tool, and DiskInventory X, capable of showing the biggest files on your hard drive. From experience of using all three, I am of the opinion that AppZapper and DiskInventory perform a better overall job in their respective categories. However, Sponge seems unique in offering all three of these facilities in one package. If you’re looking for a single tool to help clean up the files on your system, Sponge could be a good option. It costs $19.95, but a full 15 day trial is available.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A 2011 Connected Consumer Forecast
• Connected Consumer Q4: New Platforms and OTT’s Dynamic Duo Dominated
• Big Data, ARM and Legal Troubles Transformed Infrastructure in Q4

My enthusiastic use and promotion of Growl should come as no surprise to regular readers of The Apple Blog. While some dismiss it as an annoyance reminiscent of the Windows notification popups, others — like myself – view it as a way be informed without stopping what I am currently doing (and that is definitely not the case with its Windows pseudo-counterpart).

Even though many of the applications and utilities that help me with my daily workflows have embedded Growl support in some fashion, there is one use of Growl that may help convert even the most stalwart Growl skeptic: Hardware Growler.

Setting Up Hardware Growler

You’ll find this handy utility in the “Extras” folder on your Growl installer disk image. It’s main purpose is to fire off a Growl notification each time devices are connected or disconnected from your Mac, and includes support for FireWire, USB and filesystem volumes (i.e. disks) as well as Bluetooth devices and network interfaces. Unfortunately, to make the best of use of Hardware Growler, you will have to do some work to get it installed properly.

First, copy the “Hardware Growler” application to either “/Applications” or “/Applications/Utilities” on your Mac. The application itself is nothing to write home about. Double-click on it to see a very sparse menu and an innocuous Dock icon. The “Preferences” window is even sparse, only providing a means to enable or disable a notification about what devices are currently connected upon startup of Hardware Growler (not very useful, at least for me).

You can leave Hardware Growler running this way and it will go about it’s job quite nicely. However, since it is not really an interactive program, it would be best to “make it go away”, especially if you happen to miss-hit a Cmd-Q and accicentally cancel out of it. The easiest way to accomplish this is to modify one of the Hardware Growler configuration files.

Depending on where you decided to install Hardware Growler, either open /Applications/HardwareGrowler.app/Contents/Info.plist or /Applications/Utilities/HardwareGrowler.app/Contents/Info.plist add the following two lines after the <dict> line:

   <dict>
      <key>LSUIElement</key>
   <true/>

You will need to quit and re-start Hardware Grolwer for this setting to take effect. (CocoaDev has more good info on the LSUIelement propery.)

Next, to make it even easier to ensure Hardware Growler is running after you login, go to System Preferences > Accounts and add it as a “Login Item” and check the “Hide” checkbox just for good measure.

Using Hardware Growler to Spy on Your System & Applications

Because Hardware Growler hooks itself into the device events on your system, you now have the opportunity to see some of what happens underneath the pretty OS X GUI. While you should be aware of when you connect FireWire and USB devices, Hardware Growler still notifies you when those devices are inserted or removed and having a notification of that may help troubleshoot issues.

For me, it is very handy to be notified when my Option Express 3G card suddenly decides to disconnect itself from the card slot and it has saved me a ton of grief. Expanding on that particular example, you will see that many PCI ExpressCard adapters show up as USB devices. This is actually the most common way for vendors to utilize the ExpressCard slot and you really are not getting much benefit out of using the card versus an external adapter, apart from freeing up an external USB slot for other uses.

One activity that is especially interesting to have a view into is when your applications that are set to auto-update are downloading and mounting disk images. I managed to “spy” on the Google updater (no screen capture, though) doing it’s work just this past week (I had actually forgotten I let it auto-update).

Gaining a view into network-related events is also pretty handy. Not only will you see which adapter is connecting, you will also see the current MAC (unique hardware) address and all changes of IP address. This can be especially handy when you are attempting to diagnose connection issues.

If you do decide to give Hardware Growler a try, let it run in the background for a week as you go about your workflow. I think you’ll find the experience illuminating and you may just gain more insight into how your system and many of your applications work.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce
• Content Farms: The Players, The Benefits, The Risks

Despite being an avid OS X user, there are deficiencies in this great OS of ours and many of the ones I focus on center — unsurprisingly — around security.

In the plethora of accurate claims of superiority in Apple’s “I’m a Mac” ads, one counter-example is the ability within Windows to encrypt individual folders. While Microsoft’s EFS is no panacea of security and usability, it does work and there has been no practical parallel yet within OS X. Until now.

A Twitter post early Thursday morning from the legendary Matt Gemmell quietly announced Espionage from Tao Effect software (Greg Slepak & John Ashenden). This $14.95 utility (for OS X 10.5+) uses some interesting tricks to bring folder-level encryption and/or privacy to your workstation. Read on to see what’s going on under the covers and to find out if Espionage is the right solution for you.

Encryption Choices on OS X

Without bringing in additional tools, such as TrueCrypt into the mix, Apple offers two ways to secure your information. The first is with FileVault (which has some security and usability issues of it’s own) where you can choose to encrypt your entire home folder — but only your home folder — to keep prying eyes away.

The second is to use Disk Utility to create an encrypted disk image and then mount that whenever you need to store or retrieve data. This is a cumbersome, but effective, process and is ultimately what FileVault is doing under the covers to work it’s magic.

If only there was a way to associate these secure disk images with folders and have the mounting be handled automatically…

A Peek Behind the Curtain

Normally, the inner- and inter-workings of an application are either too-intricate (e.g. Photoshop) or too mundane (e.g. TextEdit) to cover during an app-review. However, when it comes to security, very few details are insignificant and one of the prime uses of Espionage is to secure your data and control the access to it.

Espionage has two basic features, enabling general encrypted folders (using the same “trick” as FileVault) and providing a way to “lock” folders and require a password to access them.

It performs the latter through a kernel extension named “iSpy” that is installed upon first run of the application and can be seen by dropping into the Terminal and issuing the following command:

$ kextfind -case-insensitive -bundle-id -substring 'com.taoeffect.' -print
/System/Library/Extensions/iSpy.kext

“Protected” folders show the typical “restricted access” icon when locked:

And prompt you for an access password (which you create when “securing” the folder):

Because it operates at such a low-level, this “protection” exists even when using command-line utilities to access files in the folder. That is, even attempting an “ls” from the Terminal will bring up the access prompt (provided you have not already unlocked the folder). This “protection” only works on the system the folder was “protected” on and requires the kernel extension to be running. If you disable/unload the extension or just boot in target disk mode, you will be able to access the data. The Tao Effect developers make no claims of security with this method of protection and even go out of their way to warn you.

But, What About Encrypted Folders?!

Ah, yes. The main reason you will want to use Espionage is to take advantage of the encrypted folders. As I have indicated, they use the same slight-of-hand that FileVault uses and create a hidden, encrypted sparse disk image that then is mounted and linked with the folder you specify. For existing folders, it creates this disk image, copies the files and folders from your target selection into the new disk image and sets up the linkage behind the scenes after deleting your old files. I should warn you that it did not do a secure delete of the “expenses” directory and I was able to find it and the contents therein in the “Trash”. This could easily be recovered and is a pretty serious oversight in an attempt to make your digital life more secure.

As part of the magic, you will see that there is a new folder in your “Volumes” directory (this is where all mounted disks get placed by default) where Espionage keeps mount points for all these sparse images.

And, you can also see just where Espionage stores these sparse disk images via the Terminal or through Disk Utility.

Since it is just a disk image “hack”, Espionage also provides a way to specify the default size and filesystem type:

So, What’s The Verdict?

Espionage does have some very interesting capabilities and I was impressed that the installer (which puts the kernel extension into place) includes full details as to what it is doing.

The application also includes other niceties such as support for Growl notifications and the ability to always enable or block application access to a particular folder under the watch of iSpy — and, you will need to make use this feature if you plan on utilizing any type of automated backup solution that will include that folder in the source path list.

However, due to the deficiencies with the way it initially creates encrypted folders and also some quirks during the operation – especially when performing multiple operations on the test “expenses” folder — I, personally, will have to continue to use my existing methods of securing data. As you saw from the FileVault screen capture, I do not use FileVault, but I do use secure disk images locally, on USB sticks, fileshares and when I am backing up sensitive data to my offsite provider. I also use TrueCrypt when I need to ensure my disks are fully protected.

I strongly suggest, however, that you do watch for future updates to Espionage as the developers will no doubt work the kinks out of this initial release and provide a very solid solution to fill the gap left by Apple. Since I am not aware of any features of Snow Leopard that will obsolete the functionality of Espionage, it should continue to fill this gap through the next release of Apple’s desktop operating system.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce
• Content Farms: The Players, The Benefits, The Risks