Assumptions
GitHub allows you to create a git repository via a Web interface, then interact with the repository using a few protocols. I focused on git+ssh, because it's the only protocol that allows pushing to repositories.
I assume that the Git server is a Linux box, and I tested my work on Ubuntu. The setup will most likely require changes for MacOS. I use the already-available infrastructure, such as git and ssh. On Debian, you should have the git-core and openssh-server packages installed.
I assume you're not willing to change the OpenSSH server configuration, as you want to stick to the secure defaults on your production infrastructure.
My Web application is written in Ruby on Rails, and it uses the grit gem, also written by the GitHub engineers. While the code is specific to this technology, most of the article is relevant outside the Rails world.
To avoid edge cases in putting together shell commands, I assume repositories and Web users have very sane names (only letters, digits, underscores).
The code for this article is available here (GitHub). All the code links in this article reference a particular tree, so that the code would match the writing. Future revisions are available in the same repository, but they may be more optimized and harder to follow.
Big Picture
A GitHub blog post describes their SSH serving architecture (section Tracing a SSH Request). Scaling aside, these are the main components:
- the server has a git user; git pushes and pulls get processed under that user account
- authorized_keys (the sshd configuration) for the git user contains all the public SSH keys in the Web application (GitHub modified openssh to query a database, since their authorized_keys would have been huge)
- authorized_keys also sets up sshd to run a restricted shell instead of the user's normal login shell (git provides git-shell for this purpose), so users can't use the git account to get shell access on the server
- sshd is not pointed to git-shell directly; instead, a proprietary wrapper checks that the SSH key's owner is allowed to access the repository and, if the operation succeeds, flushes or updates any Web application caches associated with the modified repository
I use this script to set up the git user. The script accepts any user name, but to keep it simple, I'll use git in this article. The script's effects are undone by this script, which I won't cover here.
I don't expect that the Web application will run under the git account, so instead I set the git user's primary group to be the same as a group on the Web application's account. I assume that the Web application will run under its own user, and nothing else will use that user and group. It's also possible to create a git group, and add the Web application to it. What matters is that the Web application can write to the git user's home directory.
The git repositories are stored in the repos directory, under git's homedir. The repos directory must be writable by the Web application, so its permission bits are 770 (rwxrwx---).
Recent versions of sshd are very strict with the permissions on authorized_keys. I appease them by setting the git home directory bits to 750 (rwxr-x---), setting .ssh's bits to 700 (rwx------), and setting .ssh/authorized_keys bits to 600 (rw-------). So the Web application will not be able to change authorized_keys directly, which is a problem, since I'd like to manage authorized SSH keys in the Web application.
To compensate for the issue above, I create an install_keys setuid executable in git's homedir that overwrites authorized_keys with the contents of repos/.ssh_keys. Due to the setup above, only the Web application should be able to write to this file. Furthermore, install_keys's bits are 750 (rwxr-x---), so it can only be run by the Web application.
I encountered two more minor but annoying issues. install_keys cannot be a script, because the setuid flag doesn't work for scripts with shebang lines. My setup script writes out a C program and runs gcc to obtain a binary. The setuid bit is lost when a file is chowned, so chmod must be called after chown.
SSH Configuration and Integration
authorized_key has one line for each key. The code that generates the lines is here. I used all the options I could find to lock down git's shell. The command line points to a shell script in my Rails application, and passes the key's ID as an argument, so I can identify the Web application user.
My keys are standard Rails models, and I use the after_save and after_destroy ActiveRecord callbacks to keep authorized_keys in sync with the database. More specifically, for each key addition or modification, I re-generate the contents of authorized_keys, write it to /home/git/repos/.ssh_keys and then run install_keys to get the actual file modified. If response time becomes an issue, I can move this process to a background work queue.
git's restricted shell script (stub here, real code here, test here) is stored in my Rails application, for my development convenience. Once the implementation is crystallized, it could be moved in git's home directory, so the git user doesn't need read access to the Web application's directory. The script performs the following steps:
- parses the command line to extract the git command to be executed, the repository path, and the key ID
- checks that the git command line matches the list of allowed commands, and determines whether the command does a pull or a push
- issues a HTTP request against the Web application to check whether the SSH key's owner has permission to pull or push to the repository
- runs the git command, prefixing the repository path with repos/
- if the command succeeds, pings the Web application so it can update its caches
- relays the git command's exit code
Web Application and Testing Considerations
An application user is modeled by a Profile, and a git repo is represented by a Repository. Both models use ActiveRecord callbacks to synchronize on-disk state: profiles correspond to directories under repos/ (sync code here) and repositories correspond to bare Git repositories created by grit (sync code here). The callbacks are slightly more complicated than for SSH keys, because I need the old name when a profile or repository's name changes, in order to rename the corresponding directory.
Unit tests stub out the methods that compute the path to repos/ (code here and here), so they can run without a full setup. This is desirable so I can get CI running later. I also have a giant integration test (code) that runs all the code described in this article. Since it creates a new user, it requires sudo access, which makes it unlikely that I'll ever be able to set up CI for it.
A particularly challenging for the integration test was pointing git to a SSH key to be used when pushing and pulling. Git doesn't take a command-line argument, and the easiest solution I found was to override the ssh command used by git with a custom wrapper (wrapper-generating code) that contains the options I need to point to a custom key. Overriding is achieved by setting the GIT_SSH environment variable (code). Also, the permission bits on the key file must be set to 600 (rw-------), otherwise ssh will ignore the key.
Motivation
I find GitHub's setup to be awesome, and I always wanted to have my own server implementation to play with. Since I read the team's blog post on how to do it, I wanted to give it a try. When I finally found the time, my experience was rewarding, but also frustrating. POSIX permission bits are limiting, and working around them is non-trivial, at least for me.
Conclusion
I hope you found this post useful, or at least interesting. I look forward to your comments and suggestions for simplifying the setup, or improving its security!
No comments:
Post a Comment