iTunes database issues

I love iTunes, and I love that it provides me with convenient access to all my music. But I’m beginning to get frustrated with some of the interesting DB issues present in the way iTunes was developed.

I have a large library of music and audiobooks running at just under 31,000 items using 205GB of storage. I have everything on here from Tomita to Eminem and Janet Jackson to Rufus Wainwright. All of the music is (currently) stored on a single volume on the main file server (which itself runs OS X Server).

iTunes is not, and never was, a music manager – it’s a player/organizer, but there are some aspects of the database that iTunes uses that make it frustrating to use.

Database size and updates

One of the things about iTunes I love is the ability to identify songs you haven’t played recently. But for this to work, the database has to be updated each time you play a song. And that’s my current gripe.

You see, to update the database with the play count and ‘last time played’ information iTunes has to save the in-memory version of the database to disk. The iTunes database is stored in the iTunes library file in the Music folder in your home folder. It’s usually many megabytes – there’s a lot of information in there – and it obviously increases in size significantly as the size of your music library incrases. For me, the iTunes database file is just shy of 70MB in size.

So, when updating the information for one song (something iTunes does each time you play a song), iTunes has to save a file that is 70MB in size. On my PowerBook, where I store the main version of my iTunes library, it takes 3-4 seconds, and sometimes 10 seconds, for iTunes to load, modify and resave this file. For those precious seconds, iTunes locks up – it still plays music, but the interface to iTunes freezes completely until the DB update is complete. Sometimes, it even locks up the rest of the OS – annoying if I happen to be typing and then have to wait a few seconds while the system sorts itself out.

Even worse, iTunes also generates a XML file version of the database each time it updates the song playcount too. For my library that means an additional 52MB, plus the overhead for generating the XML content in the first place.

To summarize, each time I play a song – any song – iTunes has to write out 122MB of data.

It begs the question of why on earth Apple chose to go with a single database for this – I can’t be unusual in using iTunes to manage a vast library of music. I can also foresee that as we use iTunes to access videos the problems are likely to increase as we put more and more pressure on iTunes to store the information.

Using a monolithic single file is just slowing the application down and it really wouldn’t be hard to split the file up into manageable chunks, or to use one of the embedded database libraries to access a more structured and easier to update version of the information. At a push, I don’t think it unreasonable to at least be allowed the option of using MySQL (an included component of OS X) to store the data, although I realize for many that would be extreme. There’s no reason thought that iTunes couldn’t support both a file and SQL db model.

Now I wouldn’t mind the use of a single file, if it wasn’t for the fact that it caused such a hiccup while using the application. I don’t want iTunes, or the system on which I’m running it, to lock up for a few seconds while it manipulates a huge file for the sake of updating a few bytes.

It just leaves a bad taste in the mouth.

Database storage

The other issue with the monolithic file style is that backups identify a change in that file as a requirement to backup. That means I get 70MB of backup each day just for listening to my music. That’s not backing up the music I listen to, just the file that holds the database that iTunes uses to store information about what I’ve listened.

To put it into perspective, for all my other material (the articles, books, code, applications, screenshots and diagrams) that I might create in a day, the average amount to backup is just a few megs.

So, just listening to content causes larger quantities of material to be backed up than the amount I generate. :(

XML

I love the fact that I can export my database and playlists into XML. It provides a great way for me to take my music and update an SQL database with the information. It also means I can easily share the data with other people.

So why does iTunes create an XML file every time it also saves the database?

Yes, I know the XML file is a backup, but why create it for every update to the database? Why not just each time you quit iTunes?

Those operations (sharing the DB data through XML) are steps I would elect to do when I needed to do them, not done automatically each time I update the database. Why isn’t the generation of an XML version of the database an operation I can switch off in the preferences?

99% of the time – particularly when simply playing music – I don’t need the XML file regenerated. For some users, they never need the XML file generated.

iTunes, however, thinks differently.

Using a database that links to music

One of the other oddities of the iTunes database is the way it refers to and finds the music you want to play. The server holding my music has changed IP address a couple of times in the years I’ve been using it, although the name of the machine and the name of the volume (imaginatively, Music) on which the files reside has not changed in that time.

If I forget to mount the shared Music volume before I start to play a tune, iTunes can lock up while it works out where the file is. For music added recently, recent versions of iTunes have fixed the ability to automatically mount the volume where the music is located. No problems there.

If the music is old though, then iTunes starts looking for the IP address (not the name) of the server on which the volume used to reside. Now I can, kind of, understand why it does this, but why does iTunes wait around for a minute expecting this machine to answer?

Ignoring this annoying aspect, if iTunes retains this specific information about where the file is located, how is it able to find the file when I mount a volume with the same name? If iTunes wants to be so specific about where the file is located, why does it ignore that information when it first looks for the file?

Since iTunes does find the music at /Volumes/Music/… why does it even bother recording the rest of the information? It obviously doesn’t need it to play the file.

Of course, there’s no way of block updating the information so that it always looks at the new location. Nor does iTunes intelligently update it’s database when it plays the file and discovers it on a different server (but same apparent volume) than that registered in the database.

Oh no. Instead, iTunes makes you wait a minute while iTunes times out looking for a server it shouldn’t be accessing any more anyway.

Summing up

None of these problems are show stoppers. They are, however, all annoying and show some annoying lack of thought that we are normally not subjected to by Apple engineers.

In some cases we’re talking about adding a option in the preferences and a simple check during the save process – that would eliminate the need to save the XML each time the DB was updated and save both disk space and CPU cycles most users just dont need.

In others, all that’s needed is a minor rethink on the way the database and the information it stores are handled. Splitting out the DB by the first letter of the album name would go a long way to alleviating the monolithic file problem.

Despite this, I still love iTunes and, frustrating though these issues are they are not going to stop me from using it.

loading

Comments have been disabled for this post