Category Archives: Technical Notes

The Fall of iPhoto, Part II

2014-02-16 11.33.33-2

Getting free from iPhoto has taken me years. But I’m finally free.

The eventual solution I hit upon is the one you see above, sketched out on a trusty Field Notes notebook while on vacation earlier this year. instead of losing myself in the minutiae of sand and sea, I was thinking about file transfers and backups.

Side note, I’m absolutely horrible at taking vacations and need at least three days before I start relaxing. Sometimes four or five.

Dropbox is the key, sadly. Dropbox’s camera import function is 1) available on practically everything now, 2) renames pictures uniformly and sensibly, and 3) frees me from the tyranny of wires without subjecting me to iPhoto or its slightly less obnoxious offshoot, Photo Streams. The downside is serious privacy concerns with Dropbox, but I’m feeling security fatigue these days and cant be arsed to care.

Is that bad? It’s probably bad, but it’s honest.

Dropbox slurps up the photos off devices and pops them onto my Mac Mini Server. That server (named Sirion, if you’ve been keeping track at home) puts them in the normal Dropbox place – ~/Dropbox/Camera Uploads/. Thats where the backing up starts.

  • First, Crashplan senses the changes on Sirion and triggers a backup every 15 minutes. So the first copy is on Dropbox (in at least three locations), and now theres an actual backup in the cloud.
  • Second, I have a cronjob running which looks for media files in certain locations, identifies their modified-on date, and copies or moves them into sorted directories on my main external drive. The directory format is Media/YYYY-MM/, with movies, screenshots, gifs and photos all treated separately. This is the main archive where I do basic editing.
  • Third, Crashplan runs again and backs up my external hard drive. Now the photo can be removed from Dropbox with impunity, which is good because  I have a script clean out Dropbox/Camera Uploads and Dropbox/Screenshots of files older than 90 or 15 days, respectively.
  • Fourth, I do a brute force rsync between my primary external drive and an identical secondary local drive once a week to snapshot everything and guard against disk failure.

Clunky? A little Rube Goldberg-like? Yes. But effective.

Living in the Finder has some disadvantages – mostly because I havent gotten out any Christmas photobooks in 2 years – but it has a lot of advantages, too. Speed, ability to arrange photo collections as desired, editing on iOS devices at will – did I mention I can actually page through photos again?

Well, I can. It’s pretty dang neat.

 

2014-10-04 18.10.52

Goodbye, iPhoto. I don’t miss you at all.

The Fall of iPhoto

20131029-222010.jpg

iPhoto is not up to the demands of a modern photographic library.

When I returned to a Mac in 2005 I had a lot of hope for the little photo database who could; it was shiny, easy to use, with neat features to help me organize my small but rapidly growing photo collection. My first child was born that year and I had a digital camera.

Surprising no one, I was unprepared for the sheer volume of photos I would generate. Not only did I start taking exponentially more pictures, but they got bigger all the time. Then my wife got a camera, adding more photos to the input. Then my in-laws. I moved from a Linux-based hodgepodge to a monolithic silo called iPhoto, and that silo filled up. Every year or two I would upgrade, and every year I would curse at it as it slowed down more and more.

20131029-214559.jpg

Looking back, I think the 10,000 photo mark was where I stopped having acceptable performance. Up to that point iPhoto performed pretty well within an overly-complex photo workflow; once I started going past that number it slowed down past acceptable levels. I struggled with it for a few years, before finally giving up on the idea of publishing photos online entirely. This wasn’t entirely iPhoto’s fault – Flickr was slowly becoming moribund during the end of the last decade, Zooomr (remember them, the first with geotagging?) collapsed – and I certainly went through a period of dissatisfaction that’s another story entirely. But it’s important, when looking at why you stopped doing something, to inspect your tools and ask: did these make me happy or not?

iPhoto didn’t make me happy.

I kept it around out of a sense of obligation and convenience. Every year my wife and I would publish a little softcover book out of it for Christmas with pictures of the kids for our families, and those pocketbooks have been great. Fantastic. I’m glad I kept that tradition going, even with a second kid and serious illnesses hitting us several winters running.

But the photos kept piling up in iPhoto. 20k. 30k. 40k. 45k.

I stopped using it as anything other than photo storage. It took minutes to open no matter how much database optimization I tried. It pegged my CPU, even with 16 GB of RAM and nothing else running. It was untenable, but I didn’t have the time to fix it, nor did I really have a better option. And then two things happened.

I met an entrepreneur from Holland, and I developed carpel tunnel syndrome.

20131029-215221.jpg

Steven is a bright, charismatic guy. He and his savvy tech team came to visit during a deployment of a startup he’d gotten funding for back about a year ago now. In the way of such things, we chatted about all kinds of different technical things, and one of the things he really was passionate about was Dropbox. Dude lived in his Dropbox to such an extent it was inspiring. Paper would come in, get scanned on the spot: boom, Dropboxed. Important tech stuff? Right into Dropbox. Pictures?

Ah, pictures. In the span of thirty seconds – no longer than an elevator pitch – Steven sold me on returning to the basics of storing your photos on the Finder again – using Dropbox, naturally. “Everything goes into one big folder and I can find it by date. Or name, if I want. It scales naturally, no application overhead, and easy to see where it all is.” He was so passionate about this simple system he used! I think he sold me more on returning to the Finder than he did on his business. I thought a lot about how iPhoto had become an obstacle to my photo collection because of its inability to handle large collections.

Then, about six months later, my hands started to hurt. First a little, then a lot, then it became clear I needed to stop all non-essential activities on the computer. I stopped all gaming, all blogging, even Twitter and limited email for a while.

What this injury gave me, though, was time to poke at my photo collection. To pick up a camera again, firstly. Then to poke at that iPhoto library, slowly. I couldn’t be active but photo processing and sorting with an overburdened, underperforming database is very mow impact. You can only do so many actions every minute – 1 if I was lucky.

I wasn’t usually lucky.

But the poking at it had a purpose – this idea of returning to the Finder percolated through the long months with painkillers. I made copies of the photos, tried different export routines, slowly cobbled together a plan for escaping the iPhoto trap. Every time I got thwarted by iPhoto’s limits (whhhhyyyyy won’t you let me drag and drop more than a few photos at a time? Why do you change dates when I do?) the incentive to free my photos grew. I just wanted to be able to look at them again. I wanted to have a sane organization, something I didn’t have to wait 5 minutes to launch.

My injury gave me the time to realize that continuing to use iPhoto as my picture repository was dumb, and the longer I waited the worse it got, and since I couldn’t do much else – at least I could fix my photos.

This is not to pick on iPhoto, to say it has no value. It’s great at putting together small slide shows, print books, etc.. For a small, well-curated collection it’s probably great. But it doesn’t scale. It really doesn’t.

20131029-212536.jpg

It took ten days to code and run my first pass through escaping iPhoto, and I netted 73GB of duplicate photos that went into the Trash. Given how scattered and disorganized I’ve become, and how many backups and copies I’ve made over the years, there are several hundred GBs to be reclaimed off my hard drives.

To start off my return to blogging here on this site, I’ll walk through what I’m doing to better handle the media deluge I create for myself and see if Steven was right – maybe letting the file system handle your files really is the best option.