Git, the command-line version control system, was initially released on April 7, 2005. Github launched to the public on April 10, 2008 and was the killer application that promoted Git beyond its initial audience of systems programmers. By 2010 it was clear to me that the industry was moving away from Subversion and Git was the new standard in version control. I took it upon myself to git with the times and learn it for myself. My learning project whose initial commit was on September 13, 2010 was a repository for my dotfiles and I've been using the same repo since then to manage my dotfiles. It was my first attempt at using Git and my first attempt at managing configuration in a rigorous way. Over the years I've hit several problems with my initial design and I've also learned a lot about configuration management and what is now called devops - a term whose first appearance on Hacker News was on September 3, 2010! I've wanted to replace my dotfiles setup for some time but have never really put it into action. Recently I discovered deadc0de6's dotdrop which seemed to be more or less what I was looking for: source-controlled, templated management of dotfiles. As I dug into the project I found several serious shortcomings with the code which I will explain in this blog post. The next blog post will describe the design of a not-yet-written system that incorporates what I feel are best practices for dotfile management that I hope to implement some day.
First, I want to applaud deadc0de6 for what they did right. Using a human-readable, declarative configuration to orchestrate the dotfile setup across all hosts is a good practice. YAML is a good fit for the heavily hierarchical configuration inherent to this domain and probably a better fit for the problem than the trendier TOML. Jinja2 is an excellent templating system flexible enough to find use outside of its original HTML templating purpose. Dotdrop's dotfile orchestration uses configuration profiles and inheritance to eliminate the repetitive configuration similarities between machines which is a good fit for the domain. I also like the author's suggestion to put dotdrop's upstream code directly in your dotfile repository as a git submodule. This is a sane way to pin the version of dotdrop and install it correctly and easily when setting up a new machine. It's also nice that dotdrop uses your machine's hostname as the default profile name although it would also be a nice enhancement to search for profiles that match your user and hostname.
There are also several design decisions that I don't agree with but I think that nothing in here is too serious that deadc0de6 coudn't release a workaround, even if it needs a compatibility-breaking dotdrop2 release. if these changes are made I would consider using dotdrop but otherwise I'm still planning on writing my own system at some point.
If you install dotdrop as a git submodule you have to use a dotdrop.sh driver script to set up the Python environment correctly. The driver script is a necessary workaround for the situation but the .sh suffix (to keep it from conflicting with the dotdrop submodule directory) seems like a hack. Couldn't you bundle a ~/dotfiles/bin or ~/bin along with your dotfiles and put the driver script in there along with any other shell scripts you might tote around with you? It would make sense to promote the practice of maintaining your own personal ~/bin in dotdrop's documentation.
The Python community has pushed virtualenvs for almost a decade now but the dotdrop documentation does not encourage their use with either of its installation methods. This is a big missed opportunity because dotdrop already requires a working directory to hold your dotfiles. It's the perfect place to stash a virtualenv and keep your dependencies isolated from the system Python. The current installation methods are not good Python practice.
Dotdrop lets you install dotfiles by placing a symlink in your home directory that points back to the dotfile kept under source control. I think this is a good practice: it's a good, easy-to-check way to see if a dotfile is managed by dotdrop. However, if your dotfile is a template the expanded dotfile must be written directly to its destination, there is no way to have dotdrop make a symlink. I think it would make sense to have dotdrop maintain a working directory for dotfiles and symlink into that working directory so all dotfiles, templated or not, can be visibly symlinked. I also think that in addition to using symlinks to identify managed dotfiles that all dotfiles should strongly be encouraged (possibly with a deploy-time check) to have a dotdrop-generated header that reminds readers that the file is managed by dotdrop and maybe lists some statistics like build date, git commit id and commit date. If a user insists on being able to review a dotfile before deploying it to their system there could be an optional way to disable symlinking and the deployment could be split into a second, independently run phase that does the copy of dotfiles out of the working directory.
Users that do not use symlinks in their dotdrop installations can use the dotdrop update command to install their updated dotfiles by copying them out of the dotdrop directory and into their proper location. However, if your dotfile is a template dotdrop will refuse to do this copy and ask you to edit the configuration file by hand. This is the opposite of what I would expect, I would want dotdrop to always copy and overwrite template-generated dotfiles. If dotdrop keeps a working directory for its dotfiles as described above the update command could always overwrite the dotfile as it is under dotdrop's control. Users who use symlinks to point into the working directory would instantly have the newly generated dotfile.
Finally, the template system's variable support is lacking. Variables must be sourced from the environment, there is no way to source them from the YAML configuration. This seems backwards. I would expect to only keep one or two NODE_ENV-esqe variables in an .env file. They would just define the configuration file's location and dotdrop's profile name, two things that likely won't be checked into source control but are easy to set up in a new installation. All of the rest of the configuration variables would be in dotdrop's source-controlled YAML configuration file where you can take advantage of profiles and inheritance. If you have a ~/bin driver shell script it can source this file and any virtualenvs needed to run dotdrop itself. The current design that pushes all variable state to a .env file keeps a large portion of your configuration outside of source control and duplicates work already done by dotdrop's profile system.
There is clearly a lot of effort put into dotdrop and many parts of it are polished. But at the same time I think it is clear that not as much thought has gone into the templating features of dotdrop. Templates are an important enough feature that I am willing to hold off until I can either write my own dotfile management or see if dotdrop will improve its feature set.