Synchronizing Git repositories without a server

This post describes a method for pushing changes between two repositories without using a server with network connections to both hosts having repositories. The solution is a reasonably straightforward application of local repositories, and I'm providing it to save the time of fellow developers with similar needs.

I first describe a solution using a USB stick to move data, which can be generalized to any other method for transferring files offline. Then I explain how to use this solution to deploy Rails applications using my tool, without a version control server. In case you're curious, I end the post explaining what prompted me to do this. Obviously, you can stop reading after you learned all you needed to know.

Pushing Changes with a USB Stick

I'm assuming your working directory is the place where you have your source repository. If not

cd /path/to/your/repository

Start up by creating a repository on the USB stick.

mkdir /path/to/usb/stick/repository.git
git clone --local --bare . /path/to/usb/stick/repository.git

Then register the repository on the USB stick as a remote repository, and push the desired branch to it (if you don't want to push master, substitute your desired branch).

git remote add usb file:///path/to/usb/stick/repository.git
git push usb master

In the future, you can treat the USB repository as any other remote repository. Just make sure it's mounted :) For instance, the following pushes new changes to the USB repository.

git push usb

On the receiving end, mount the USB stick, and use a file URL for the repository

file:///path/to/usb/stick/repository.git

A few handy commands:

• cloning the repository on the USB stick
  
  
  
  

  git clone file:///path/to/usb/stick/repository.git
  

• updating a repository cloned from the USB stick using the above command
  
  
  
  

  git pull origin

• adding the USB stick repository as a remote for an existing repository
  
  
  

  git remote add usb file:///path/to/usb/stick/repository.git

• updating from a remote repository configured using the above command
  
  
  
  

  git pull usb master

The technique above works for any offline data transfer method, as long as you can copy the git repository from one file-system to the other.

Deploy Rails Applications without a Version Control Server

rpwn assumes that the source code for Rails applications is under version control on a server. But, at least when using git, applications can be deployed from repositories on USB sticks, created as explained in the previous section. Install an application as follows.

sudo rpwn install file:///path/to/usb/stick/repository.git

Keep in mind that rpwn will use the same repository URL to update the application, so you will always have to use a USB stick that mounts to the same location. This limitation can be overcome by adding another indirection layer -- create a symbolic link to the remote repository, and use that symlink when installing.

ln -s /path/to/usb/stick/repository.git ~/victor/repository.git
sudo rpwn install file:///home/victor/repository.git

Note that the symlink preserves the repository name, so rpwn will still use the right name for the application.

Motivation

I wanted to help someone deploy a Rails application in a government setting, where the application server is not connected to the Internet (stupid security rules). Worker's computers are still connected to the Internet, so we wanted to host the Git repository on Github. This way, the application can be developed and tested anywhere.

So, the development machine is connected to the Internet, but the application server is not. I didn't want to push unversioned source code to the application server, and I was hoping I don't have to add extra logic to rpwn to handle this case. Git local repositories turned out to be an awesome, if not well-documented, solution.


I hope you found this post useful, and kindly ask that you share any better methods you may come up with to solve your own problem.

Dismissing the Virtual Keyboard on iPhone

This post is a quick recipe for making the virtual keyboard go away on the iPhone SDK. There are two aspects to the problem: dismissing the keyboard when the user taps Done (easy, most resources describe this right) and dismissing the keyboard when the user taps the screen outside any control that can receive focus.

When the user taps Done
Go to Interface Builder, and connect the edit control's delegate outlet to your view controller (in most cases, that will be the File's Owner). If your delegate is different, follow the steps below making changes when appropriate.

In the view controller, use the following snippet:


That's it, you're done. Run the code in the simulator and confirm it works.

When the user taps outside the field
You might see some more complicated solutions elsewhere (like Apress' book). The solution below works without any UI changes. Add the snippet below in your view controller. The snippet belongs in the view's controller, even if your text field's delegate is another object.


Bonus: reacting to the user leaving the field
If you want to take an action when the user leaves a field (such as issuing a Web service request in the background), the approach below plays nice with the modifications described in this post:


This code belongs in the text field's delegate (most of the time, it is your view controller).

I hope this post makes your life a bit easier. Happy coding!

Date, time, and datetime validation in Rails 2.x

This is a very short post pointing to a good plug-in for validating time stuff in Rails. One-line summary: use validates_timeliness.

I wanted to validate the format of a datetime field in one of my models, so I tried validates_date_time and saw that Rails doesn't have it installed. I googled validates_date_time, and stumbled upon the plug-in with the same name by Jonathan Viney.

I installed the plugin, fixed the unit tests to run on Rails 2.2 with sqlite3, and then proceeded to write my validation, complete with unit tests. All the code is included, for your pleasure. After some booring debugging, I came to the conclusion the plugin can't handle invalid formats on new models, because of how it's using _before_type_cast to determine whether its parser failed or succeeded.

I googled for datetime validation in rails 2.2, hoping that someone fixed the bug. I didn't find a fix, but instead stumbled over the validates_timeliness plugin, written by Adam Meehan. In case you're seeing this in plain text, the link to the plug-in is http://github.com/adzap/validates_timeliness/

I hope Google indexes my post, and that it helps you avoid going through the same trouble that I have.

The code I used for this post:

Auto-rotating Tab Bars on the iPhone

This post is a quick summary of what's needed to get a tab bar (like the one in the Clock application) to switch from portrait to landscape when the user rotates their phone. Accelerometer goodness, yum!

The process is easy, but it has a couple of pitfalls. Here are the steps:

1. You have to override shouldAutorotateToInterfaceOrientation: in the view controllers for all the views in the tab bar. If a view controller's shouldAutorotateToInterfaceOrientation: returns NO, then the tab bar will not rotate, even if the view is hidden at the time of the rotation.
   
2. You should not override the tab bar controller's version of shouldAutorotateToInterfaceOrientation:
   
3. For regular views in the tab bar add the code below to viewDidLoad. If you skip this, your view will not resize when the phone is rotated while it's selected. However, it will resize when the user transitions to it from another view.
   
   
4. Make sure the controls on your regular views respond to changes in view size. If you're using Interface Builder's springs and struts, you can test the views right in IB, using the arrow at the right of the views' title bars.

Acknowledgements: I used the knowledge in some forum posts that Google revealed to me, together with some testing.

I hope you found this useful. If you have more tips, please post them in the comments, and I'll edit the posting accordingly, so others can have an easier time finding this information.

Tips on building a CS paper or thesis

I'll describe some techniques that found useful when writing my own M.Eng. thesis and papers. My focus is on improving the process of building the paper, not writing it. So I'll write about some things you can do to make the process easier. This is not yet another compilation of writing tips.

Latex
Get a complete distribution, and install all of it. Don't try to be smart about packages. 1Gb of your disk space is worth less than the time you'll spend debugging missing dependencies.

I use texlive, because it's available on all platforms:

• MacOS: http://www.tug.org/mactex/
  
• Ubuntu: apt-get install texlive-full
  
• Windows: http://www.tug.org/texlive/ (you'll get the same distribution as above, but you have to install it yourself)
  

Tools
I'm a developer. When I write something that's not an e-mail, I'm in Eclipse. So I got myself an Eclipse plug-in for Latex at http://texlipse.sourceforge.net/. You can plug this link directly in the Update Manager to get the plug-in.

I recommend configuring your texlipse project to use pdflatex. This gets you the PDF that you need, and lets you use .pdf files as figures. Most software used for figures has pdf output.

Version Control
I feel uneasy if my work is not under version control... it's a disaster waiting to happen. Latex is all text, so you can even work on multiple machines, and merge the changes intelligently.

I recommend subversion or git. They both have Eclipse plugins:

• subversion: http://subclipse.tigris.org/
  
• git: http://www.jgit.org/

At the time of this writing, the git plug-in isn't self-sufficient... you'll need to know git's command-line to get stuff done. On the other hand, git's cheap branches might be worth the hassle.

Bibliography
There's no way in hell I'm typing in the data for my 100-200 bibliography references. Here's how I get my .bib entries:

1. Go to Google Scholar Preferences
2. Under Bibliography manager, select Show link to import citations into
3. Make sure you have Bibtex selected as the format, then click OK.
   
4. Find the cited paper / book on Google Scholar: http://scholar.google.com/
5. Click on the Import into Bibtex link. The entry is revealed for your copy-pasting pleasure.

I know this might seem obvious, but it took me a while to try it out. Scholar's advanced search is useful when you're looking for a certain author's work, and that work is referenced a lot by newer papers. If you don't believe me, try finding the paper on Dijkstra's algorithm without advanced search.

Act Like You're Building Software
Writing in latex is more like building software than writing a humanities paper. Once I embraced that, I became less miserable.

Unlike MS Word, latex lets you distribute your work across many files. A good directory / file organization can make re-organizing sections really easy.

Writing in latex is also like building software in that you can use libraries. They call them packages. Here's what used in my thesis:

• listings - code listings
  
• url - lets you add URLs to bibliography entries (why is that not in base, again?)
• graphicx - figures? I think you need it to import PDF files, I'm not sure
• amssymb, latexsym, amsmath - useful stuff for mathy formulas (I include them everywhere, it makes my life easier)
• boxedminipage - nice borders around code listings
• times - I forgot what it does
• clrscode - format your pseudocode CLRS style (I like it)
  

When I need something that seems general, I try to find a package that does it for me. People who write packages tend to write nice documentation on them, and I'd rather read that than fight with latex to figure out how to do something on my own.

I'd like to push the building software similarity even further, but I haven't figured out how to set up a continuous build yet :)

Conclusion
Writing in latex isn't as intuitive as using MS Word. On the other hand, there are some advantages to describing the contents of your paper in code. Knowing and taking advantage of them has made my life easier, and saved me from potential disasters.

I hope you found this useful. If you have more tips, please leave a comment!

Some Tricks for Background Processing in Rails

This post describes a bunch of tricks that I'm using to do background processing in my Rails applications, Movie Nights and MIT's 6.006 Course Site.

If you want the whole picture, read ahead. If you just want to see the tricks, skip the next section. There's also code at the end, for your copy-pasting pleasure. And it all works with the newly released Rails 2.2.

Big Picture
I'm doing my background processing in one or more long-lived processes. I keep their code in script/background, and use simple-daemon to make them... daemons. I use Starling to pass messages between my Rails front-end processes and the background processors. I don't like starting anything by hand, so I use daemonz to start up Starling and my daemons.

Trick 1: ActiveRecord + Long-Running Process
When using ActiveRecord in long-lived processes, you'll see their connections drop. You know you're experiencing this if you see the following in your logs
Error processing task - ActiveRecord::StatementInvalid: Mysql::Error: MySQL server has gone away:

I fixed this problem by having ActiveRecord re-check its connections every time a Starling request is processed:

ActiveRecord::Base.verify_active_connections!

If your background tasks are really long, and you're working with your database (for instance, updating some status when a task completes) you might need to run the line above several times during a task.

Trick 2: ActiveRecord + Fork
I know I'm not supposed to fork. But I need to. My course website runs student-submitted programs, and then processes their stdouts. So my background processor daemon needs to fork/exec to be able to run those programs.

The problem is, forking copies the parent's memory. So ActiveRecord's connection pools will get copied. And when the child exits, ActiveRecord will close the database connections, and screw over the parent process. spawn used to handle this for me, then it stopped working in Rails 2.2. So, after some experimentation, the following seems to be the most concise fix:

ActiveRecord::Base.connection_handler.instance_variable_set :@connection_pools, {}

Keep in mind that my forked children run non-rails code, or exec something else right away. I haven't tried using ActiveRecord in them, and I suspect it would break.

Code
My daemonz configuration (daemonz.yml) for Starling and the background processor is below. I have 4 task processors, because the production server has 4 cores and therefore can run 4 student submissions in parallel.



My background processor boilerplate code is below. It uses two queues, pulls and pushes, and tasks in pulls have priority over tasks in pushes. It also works with daemonz.yml to get multiple instances of the same daemon, which is non-trivial when using simple-daemon.



The code above calls into OfflineTasks, which I define in lib/offline_tasks.rb. The code for my course site is below. Each background task has a method that pushes it into Starling, and a case branch that executes it when it's popped from the Starling queue.



And finally, a snippet of code that I use to fork:



Thanks for reading! I hope you found this useful.

Post-install / post-update scripts for ruby gems

This post outlines a hack that allows a gem to run Ruby code when the gem is installed or updated, which in effect gives post-install / post-update hooks. The method described here works with any reasonable version of ruby and rubygems (I only tested as far back as ruby 1.8.4 and rubygems 0.9.4, and everything looked good.)

Summary
Add a fake extension, use extconf.rb to run your code and then simulate successful compilation. Use the links at the end for example code.


Detailed Description
RubyGems isn't supposed to run arbitrary code during gem installations, but it supports building extensions for gems. This is a really sweet feature, and makes rubygems a nice tool for cross-platform package management. That's good and all, but the part that we care about is that the process of building an extension starts by running extconf.rb in the extension's directory, which is responsible for producing a Make file that will orchestrate the building process.

Knowing this, the first thing that comes to one's mind is - let's add an extension to the gem, and put the hook code in extconf.rb. However, there's one more issue left. If rubygems believes that your gem's extension hasn't been built properly, it will not finish installing the gem, and it will spit out a nasty error message.

In order to work around that, we need to trick rubygems' build process, so we need to bypass 3 checks:

1. a Make file is generated - create an empty Makefile
2. make all and make install run successfully - generate a make file that contains empty all and install targets; create fake make binaries to cover the case when the user doesn't have a build environment (Linux/Mac: an executable make with a /bin/sh shebang should work; Windows: an empty nmake.bat should trick the Windows port of rubygems)
3. an extension binary is generated - create empty files your_extension_name.so and your_extension_name.dll
   

You can implement this yourself, or you can depend on my zerg_support gem, and use the method there, like the example code below does. I promise I won't mind if you copy-paste the code, so you don't have to take an extra dependency :)

I haven't tested out the Windows plan yet, but I have good reasons to believe it should work (I've built gems with real extensions on Windows some time ago).


Code Map
Adding an extension to your gemspec (assumes you're using hoe or echoe):
http://rails-pwnage.rubyforge.org/svn/trunk/zerg/Rakefile

Placing your hook in extconf.rb:
http://rails-pwnage.rubyforge.org/svn/trunk/zerg/ext/zerg_setup_hook/extconf.rb

Tricking rubygems into thinking an extension was built (see emulate_extension_install):
http://rails-pwnage.rubyforge.org/svn/trunk/zerg_support/lib/zerg_support/gems.rb