Brian Johnson

Google Contacts and Calendar backup solution

For awhile now I've been using Google as the primary store of my contacts and calendar data.  It automatically syncs with my Android phone, with Thunderbird which I usually use for email, as well as with the Exchange calendar at work so that I can share calendar info with my coworkers.

All of this works quite well, except when there's a problem.  I upgraded my phone, and in the process it managed to sync a lot of bad contact data up to Google, as well as deleted a lot of contacts.  I was able to recover from this with my backups of my Thunderbird address book, but not easily.

Solution:

I've created a git repository that holds backups of my contacts and calendar.  This way I can go back and restore data from a previous date.

This gave me the beginnings of my backup script:

#!/bin/sh
cd `dirname "$0"`
wget -qO- https://www.google.com/calendar/ical/BLAHBLAH/basic.ics | grep -v "^DTSTAMP:" > BrianJohnson.ics
git add BrianJohnson.ics
git commit -m 'daily backup'

Note: I'm stripping DTSTAMP from the ical file.  This is a "required" field according to the ical standard, and so this is probably not the best thing. The reason I did it was that it was changing with every download of the file, and I wanted to have clean diffs between versions.  I did test re-uploading one of these backups and Google accepted it, so it should be sufficient. If it does cause problems in the future I can add a dummy DTSTAMP field based on the file date/time back in. You may decide you'd rather have valid files then clean diffs, if so then use "wget -qO Calendar.ics https://www.google.com/calendar/ical/BLAHBLAH/basic.ics" instead.

So, I did that for each calendar I wanted to backup, and that works quite well.  Next problem, contacts.

Contacts can easily be exported as a .csv manually by clicking on the export link on the contacts page, but I wanted something daily and automated so I started looking at Google's gdata APIs.  Eventually I found that someone already wrote the script I'm looking for http://www.neomantic.com/software/ruby-google-contacts/.

So I downloaded "downloadGoogleContacts.rb" and added it to my git repo. It's pretty straightforward to use with a few commandline parameters. However, again I was worried about what the diffs will look like I wanted to reformat the xml, so I'm taking the output of downloadGoogleContacts.rb and passing it to xmllint before commiting it.

My final script (run nightly from cron) looks like:

#!/bin/sh

cd `dirname "$0"`

wget -qO- https://www.google.com/calendar/ical/BLAHBLAH/basic.ics | grep -v "^DTSTAMP:" > BrianJohnson.ics
git add BrianJohnson.ics

wget -qO- https://www.google.com/calendar/ical/BLAHBLAH/basic.ics | grep -v "^DTSTAMP:" > EventsAndBirthdays.ics
git add EventsAndBirthdays.ics

ruby downloadGoogleContacts.rb -e example@gmail.com -p password -t ~/gdata-token -o temp-contacts.atom -m 10000
xmllint --format  temp-contacts.atom  > Contacts.atom
rm temp-contacts.atom
git add Contacts.atom

git commit -m 'daily backup'

There is still a missing step here.  I've got the backups part taken care of, and restores for the calendars are straightforward enough, but I still need to test restoring contacts.  More to come on that.