Sunday, March 23, 2014

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.


I discovered vcsh when I had its creator, Richard Hartmann guesting on GitMinutes, and I'm still surprised how little known this great tool is. Partly responsible for this I guess is that it's a bit hard to understand how vcsh works, and the README only does a half-good job at explaining it.

So let me try with my own words:

vcsh is a sort of a Git wrapper that let's you track files in Git repositories, while the actual files (work-tree) are in your home directory. 

For example, my .gitconfig is tracked in this repository: 

~./config/mr/repo.d/config-git.git (see it on GitHub)

But vcsh sets the working directory to be ~, so when I check out the contents of the repository, it lands directly in my home directory.

vcsh works pretty much like Git, except for most commands you tell it which repository to work on:

vcsh init git # creates a bare repository by with the name 'git' under ~./config/mr/repo.d/
vcsh git add .gitconfig # adds the file to the 'git' repository
vcsh git commit -m "My initial .gitconfig"
vcsh git diff # shows any changes you have made to your .gitconfig since

If you know about Git's ability to keep the GIT_DIR separate from the work-tree, this should be pretty easy to grasp.

Enter myrepos

If you're happy to keep all your dot-files in one repository, you'd be well off using vcsh by itself. But if you prefer having several repositories (perhaps you don't want all dot-files on a particular computer), or if you have any other Git repositories you always need checked out on your computer, myrepos is the tool for syncing a bunch of repositories in one go.

Imagine myrepos like this: It's a file with a list of repositories to should be managed on your computer:

[$HOME/projects/foo]
checkout = git clone git@github.com:tfnico/foo.git

[$HOME/projects/bar]
checkout = git clone git@github.com:tfnico/bar.git

Lucky for us, it also does the right thing with vcsh repositories:

[$HOME/.config/vcsh/repo.d/various.git]
checkout = vcsh clone git@github.com:tfnico/config-various.git various

That's pretty straight forward. Now you'll soon realize that myrepos and vcsh overlap in the sense that they both can do Git operations, and this can be a bit confusing. I'll try postulating a bit on which is for what:
  • myrepos is for checking out or operating a bunch of repositories at the same time. This includes your vcsh repositories.
  • vcsh is for managing files and directories in your home directory. The contents of a vcsh repository is all relative to home.
  • The myrepos configuration is kept in a vcsh repository. So in order to bootstrap, doing a vcsh clone of our mr repository is the first thing we do (as we did near the top of this blog post).
  • As myrepos also operates your vcsh repositories, it also manages its own configuration. As an effect, if you make any changes to your myrepos config on one computer, you'll have to run `mr update` twice in order to see the changes on another computer.
  • If you're making changes on one repository, like your vim-config, use vcsh to diff and commit the changes.
  • If you want to push or pull all the latest changes in all your repositories, use myrepos.
Some food for thought: Consider my vcsh repo for myrepos - I could have kept the .mrconfig file by itself in a vcsh repository, and then have mr check out the ~/.conf/mr/ stuff as a normal mr.git repo inside of ~/.conf/. This hypothetical aside, it makes more sense to keep files for one purpose in one repo.

I can understand after writing this that it's really hard to come up with a good README for vcsh. But I like to consider how much time I've spent on dealing with my dot-files, and how much more time I spent not managing them. In light of that, spending half a day on getting vcsh and mr set up right doesn't seem too bad. They are simply the best tools for the job (and other jobs), and the only draw-back is a little learning curve.

Friday, March 21, 2014

Automating Computer Setup with Boxen

I just finished setting up a new laptop at work, and in doing so I revamped my personal computer automation quite a bit. I set up Boxen for installing software, and I improved my handling of dot-files using vcsh, which I'll cover in the next blog-post after this one.

Since it's a Mac, it doesn't come with any reasonable package manager built in. A lot of people get along with a combination of homebrew or MacPorts plus manual installs, but this time I took it a step further and decided to install all the "desktop" tools like VLC and Spotify using GitHub's Boxen:

  include vlc
  include cyberduck
  include pgadmin3
  include spotify
  include jumpcut
  include googledrive
  include virtualbox

If the above excerpt looks like Puppet to you, it's because it is. The nice thing about this is that I can apply the same puppet scripts on my Ubuntu machines as well. Boxen is Mac-specific, Puppet is not.

It was a little weird to get started with Boxen, as you're offered a download, but you are supposed to fork their our-boxen repo and take it from there. Here's a short recap:

  1. Create /opt/boxen/ and give yourself permissions to write in it
  2. Clone your our-boxen fork into /opt/boxen/repo
  3. cd into repo and run ./script/boxen
  4. Add further modules (recipes) in repo/Puppetfile and include them in repo/manifests/site.pp
  5. Run boxen again (should be automatically added to your global PATH by now).
  6. Commit and push your changes so you can apply the same to other machines, or share with colleagues.

I found lots of good modules by just scrolling through all the repos in the Boxen organization. You can lock versions for the modules, and do some special configuration in some cases, just look in each module repo's README if you need something special.

Note that you're bound to run into a bunch of hiccups along the way. Most of these have known solutions, so just search through the module's issues. In a few cases I ended up doing brew install <module> directly to get one working, which is OK since Boxen uses Homebrew as package provider.

Expect to fiddle with this a couple of hours the first time you add all your favorite modules. When downloading a lot of large packages, Boxen will break down with long stack traces because of network timeouts. Just keep restarting it and keep it up until it's done.

Here's my boxen repo. The interesting files there are Puppetfile and manifests/site.pp. You can also put some personal stuff (a Boxen repo is supposed to be used across a whole team) in a personal config, but I didn't really get into this yet.

Sunday, March 09, 2014

Calling All Programmer Podcasts

One of the reasons why I started podcasting, is that I listen to a lot of podcasts.

It took me a long while to build up my podcatching portfolio. For half a year, I listened to mostly gaming podcasts because the only programmer podcasts I knew about was Hanselminutes and Java Posse. I simply didn't know what programmer podcasts were out there.

Podcast discovery is about as well established as it was 10 years ago, meaning iTunes. Of course you can blindly google for "<topic> podcast", or you might start off with some recommendations from friends, but there still is no established way of discovering more podcasts of the kind you'd like (1).


Another problem is that I see very little cross-pollination between the programmer podcasts. Even though they intersect just the right amount, I never heard JavaScript Jabber mentioned on TheChangelog, for example (2).

To help remedy this I've thrown together all the currently active, English-speaking programmer-oriented podcasts that I'm aware of. This is just a quick stab at helping you discover what's out there in our genre. Kind of like a good old webring, or a Yahoo directory.

I'd love to make the collection a bit more interactive, perhaps pulling in some RSS data to present the latest release date, amount of episodes and stuff like that.

But for starters, it'll just be a place you can point programmers who have just figured out podcasts. Let me know if there are any you think should be added via Twitter or Google+, or just comment on this post. In the long term, perhaps I can move the list to more of a community wiki thing.

(1) Stitcher shows some promise here, seeking to be the Pandora of podcasts, but I don't use it as I prefer my selected listening software (BeyondPod), and neither do most podcast-listening programmers out there, I suspect. 

I don't use iTunes either, and I don't like their obscure algorithms of deciding what is "new or noteworthy". 

(2) Exception-wise, I did get an initial boost in listenership from +Randal L. Schwartz mentioning GitMinutes on FLOSS Weekly (twice I think).