Blog Post

iWork hates Subversion

Stay on Top of Emerging Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Or does Subversion hate iWork? I was just hit with the iWork/SVN bug that seems to be infamous among those using the two products concurrently. The problem, essentially, is the way that iWork—Pages, Numbers, and Keynote—and Subversion store their files.

It’s well-known that, for some time, Apple has chosen to package certain files as bundles; they appear in the Finder as single files, but from the Terminal and the file system proper, they are directories. Subversion (SVN, the popular version control system), like CVS (Concurrent Versioning System—another, earlier versioning system), stores its state files containing information about the current revision, the main repository, and so forth, within special directories inside each directory under Subversion control.

I have been using SVN to manage revisions of my writing—a prudent thing, you might agree, as it is both easier and more robust than the “track changes” feature of Pages (or Microsoft Word, for that matter). However, the problem lies in those .svn subdirectories that are part of every directory under Subversion control.

When Pages (or Numbers or Keynote) saves a file, it appears to do so by completely overwriting what is already present: the .svn directory is erased in the process. Attempting to commit those changes to the central repository for safekeeping illustrates the problem:

SVN failure

Files (directories? bundles?) are therefore left high and dry, and the entire point of revision control becomes moot. Since iWork does not support built-in revision control (unlike Microsoft Office), utilities such as CVS or SVN would be the next best option—if they weren’t cut down by the file bundle notion.

Ideally, Apple would decide whether file bundles—files such as applications, plugins, and iWork files—were actually files or directories and treat them as one or the other in all cases. It seems that file bundles are here to stay in their present state, at least for the foreseeable future, however.

While there are several workarounds [1][2], perhaps the most logical solution is to migrate from Subversion to a different revision control system that does not suffer this problem, such as Mercurial, Git, or GNU arch. None of these three keeps local state information within subdirectories of the working copy.

It is unknown whether other bundle-type files are susceptible to SVN and CVS impotence.

21 Responses to “iWork hates Subversion”

  1. Symonty Gresham

    Most Version control requirements are more complex than simply backup with ability to going back to previous versions, this include :-

    Tagging, releasing, comparative differential of versions and collaborative features of SVN or CVS.

    This is what most need from version control…

    While timemachine is an amazingly useful , yet simple, backup system that snapshots, this only has value for single user environments ( like my laptop !! ).

    For example jump into timemachine now and find the pages document before you removed the word “elephant” on line 3 of paragraph 4.

    Maybe I am missing something but I view timemachine as a backup system with file recovery, not a versioning system per say.

  2. Vermilion

    It sounds like Time Machine is the perfect solution for Mac OS X compatible version control. If you’ve got Leopard, it’s worth a look.

  3. algal — I was mistaken how the addremove and commit -a functions would actually work. After testing this with Mercurial’s addremove, I agree that this is definitely the way to go. (Initially, I thought that the “remove” part of addremove would cause Mercurial to forget deleted files in its version history; this is not the case.)

    I’m not a Fink or MacPorts user, but I believe either should install hg or git for you easily enough. As for GUIs, the best I can offer is the discussion of TextMate support for both hg and git; BBEdit or TextWranger may also support one or both as well. Finally, git does ship with its own GUI, of sorts, called gitk (git + Tk) that may be of help to you.

    I’m glad you’ve found this post and ensuing discussion beneficial! That’s what we strive for at The Apple Blog.

  4. Billy:

    After thinking about it for a minute, I think Mercurial’s addremove command is exactly the right behavior for OS X bundles.

    Consider. If Keynote adds or removes some graphic asset from inside a file.key bundle, then I would want that asset to be automatically added or removed from version control. As long as version control lets me go back and reconstruct an earlier version of the file.key bundle, including that asset, then I’m fine. In fact, I don’t want the version control system to keep resist Keynote’s actions on files within file.key bundles — just to track them. So addremove is perfect.

    I checked the man page and git’s “commit -a” has the same effect, automatically adding new files and automatically removing deleted files. So that’s good news too.

    FYI, I use fink and fink was able to install both mercurial and git from source without a hitch. (Fink does not have a current version of bazaar, tho.) I’d rather use a good GUI than the command line, but I’ve never seen a revision control GUI that really earned its keep.

    This has been a useful discussion! I know this issue is biting a lot of OS X users, but I’ve never seen anyone really hash out the details. The answer seems to be addremove for hg, and “commit -a” for git.

  5. algal — I see the problem: Keynote does not Gzip its assets together. commit -a is a Git command (not Mercurial). With Mercurial, there are two things you could do: First, there’s a command called addremove that will add all files not under control and remove all files under control that have been deleted. Unfortunately, if you remove a graphic from a presentation, you lose it in your version history — no good! The other option is a two-step process:

    hg add -I '*'
    hg commit

    The first line will add all files to Mercurial. (The -I specifies an include pattern.) The second, of course, performs the commit. This has the effect of adding any new files that show up without removing the old ones that might have been deleted in the process of revising your presentations.

    Although this is a two-step process, you might find Mercurial a little more friendly if you aren’t a Terminal sort. Git requires building from source while Mercurial has binary packages available for OS X.

  6. Billy:

    Thanks for the info. That’s good to know.

    My main problem is really Keynote’s .key files. My presentations can easily contain over 100+ graphic assets (equations rendered as pdfs), and I imagine every one of them will appear as a new file inside the file bundle. So I am worried about having to stay on top of the internals of every bundle.

    The “commit -a” options sounds like a good solution for when new files appear inside the bundle. Do you happen to know if git has a similar shorthand for when files disappear ? — that is, a commit command which instructs it to assume that all files deleted from the working tree should be deleted from version control as well?

  7. @algal —

    As long as one is diligent to do commits and status checks, there seems to be no problem with either Git or Mercurial. (I’ve now tried both.)

    As it appears, the x.pages/ bundle is fairly predictable with an index.xml.gz file, a Contents subdirectory with a PkgInfo file, and a QuickLook subdirectory with Preview.pdf and Thumbnail.jpg. While Apple doesn’t actually “bundle” their bundles (say, with 0-compression ZIP), they do at least Gzip the supporting files that make up the document.

    Both Git and Mercurial (and even CVS and SVN) will point out any files inside the working copy that are not under their control, making it easy enough to add them without digging for them. In addition, Git (I haven’t looked into this with Mercurial) supports a “commit -a” mode, which will both add to version control and commit it at the same time, so if a new file does magically pop up, running “commit -a” will add it and commit it without even having to look for it.

    At any rate, it isn’t a problem for either Git or Mercurial, provided that you keep on top of any potential “gotchas”. It’s a good idea to do a git status or hg status before running a commit. It works fine. The only complaint is that there is of course too much granularity. I don’t particularly need to keep revision control on index.xml.gz as much as I do x.pages. There’s no way around that (that I’m aware of) without writing a new OS X-centric version control system.

  8. Hi,

    I am facing the same problem and here is my question: does switching to git or mercurial really solve your problem with bundled directories?

    Neither git nor mercurial dumps spurious files and directories within the bundles. But they both still require command-line intervention for adding new files and removing old files from revision control, right? So if Pages adds a new “file” deep within a .pages bundle, then aren’t you still required to go burrowing into the bundle to add or remove the file in git?

  9. @HG — Thanks for the summary. I noticed the overabundance of git-* scripts in /usr/local/bin and was a bit taken aback, but if it works, hey, it’s an improvement over Subversion.

    Perhaps I’ll work on a TextMate plugin for Git. Had I even thought to check TM support, though, I probably would have chosen Mercurial, even though I usually manage my revision control from the terminal anyway.

  10. Forget Subversion. Use Git or Mercurial instead. They’re much better and DVCSs are the future anyway.

    @Billy Halsey

    Regarding comparing Git and Mercurial.

    For me, their differences conceptually and performance-wise are insignificant. If you learn one you can apply your knowledge easily to the other.

    Mercurial is fast, but Linus claims Git is faster. That would make sense since Mercurial is written in Python and Git is mostly written in C. But the speed difference is insignificant to me and the size of projects that I work on. Mercurial is fast enough.

    There is a Git-SVN tools which lets you translate between Git and Subversion. I’ve read developers needing to commit to Subversion but wanting to use a DVCS really like this tool. So Git converts some developers for this reason.

    On the other hand, Mercurial converts Windows developers more easily, because Mercurial works with little hassle on Windows and Git doesn’t.

    I don’t like the fact that Git installs 146 scripts and executable binaries in your system. That’s way too many, but I’ve read that they’re addressing this issue.

    I know Git will get more attention from the Linux community and so the Mac OS X community (which I work in) will be better served if there’s competition. For example, Mercurial can plug-in to TextMate which I use for my Rails development. A Git plug-in for TextMate doesn’t exist yet.

    2008 will be an interesting year to see where DVCSs advance. It’s too early to say which, if any, will win, but I hope they tie.

  11. @Brian and Chad — git was very easy to compile and install. (A dream, in fact, compared to some packages I’ve pulled out my hair over.) I usually like to compile all my own stuff, and I don’t use MacPorts or Fink, but git gave me no problems at all.

    Thanks, Brian, for filling in the blank on other file bundles. I wasn’t sure about those.

    If anyone is interested in a precompiled version, I can tar/bz2 my compiled version of git (for Intel, built with gcc 4.2.2, optimized for Core 2 Duo).

    @jojo — What do you like specifically about Mercurial? I chose git primarily because it was originally the work of Linus Torvalds and easy enough to migrate from SVN.

  12. Try using another version control system, that doesn’t pollute your repository by scattering its meta information across the whole directory tree: Mercurial for example (which has lots of other benefits over svn BTW… :-)

  13. @Twist — That’s a great idea. I’ll work on that.

    @Scott and Nigel — I saw that svk would overcome the problem (after writing this article), but I’ve actually already migrated to Git. As I’ll point out in the article I put together at Twist’s suggestion, Git actually works better for my needs anyway because there is no central repository.

  14. You could use svk, which builds on subversion (so can use your existing subversion repos and has a familar command set). It also gives you offline VCS functionality which is really useful to me.

    As a side effect it does not put any additional data into the working directory.

    Mac installers are available, or it can be installed using the MacPorts port command.

  15. Try svk. It’s an alternative front end to Subversion that adds distributed operation and a few other cool things. It doesn’t create .svn directories in its trees, so it should be more friendly with iWork, and it’ll work with your current SVN server.

  16. I have been looking for a solution to help keep me and other members of a creative project in sync and safely backed up. Subversion might be a decent solution for this, but I really have no idea how to setup either the client or the server. Sounds like you have experience with it though so maybe you should put together a how-to article covering Subversion from the perspective of a Mac user wanting to use it to do versioning and syncing of general files (mostly RTF, JPEG, and PSD in our case). Most of the stuff I have seen mostly covers setting it up to work with stuff like XCode.