Fat iPhone Static Libraries: One File = Device + Simulator Code

Abstract
This post explains how to build a static iPhone library that contains ARM code (for iPhones and iPod Touches) and x86 code (for the simulator), in a single file.

This is useful for companies which do not wish to expose their source code when distributing iPhone code, e.g. AdMob. The currently accepted practice is to distribute separate static library files for devices and for the simulator. This means the that the library clients have to setup separate targets for the simulator and device. The separate targets duplicate most information, and only differ in the static libraries included. This violates the DRY principle,

Solution

Open a Terminal, cd into your library's Xcode project directory, and issue the following commands.

xcodebuild -sdk iphoneos3.0 "ARCHS=armv6 armv7" clean build

xcodebuild -sdk iphonesimulator3.0 "ARCHS=i386 x86_64" "VALID_ARCHS=i386 x86_64" clean build

lipo -output build/libClosedLib.a -create build/Release-iphoneos/libClosedLib.a build/Release-iphonesimulator/libClosedLib.a

The commands above assume your library's name is ClosedLib and you're targeting iPhone OS 3.0. It should be straightforward to tweak the commands to accomodate different library names and SDK versions.


Assumptions
I assume that the library is not updated too often, so a couple of manual steps are acceptable. I would like to make add an automatic build feature to my zerg-xcode tool, but there is no ETA for that. I also assume that your project's default target is the static library.

I do not assume that the device and simulator SDKs have similar headers. The beauty of my method is that the simulator code is built with the simulator SDK, and the device code is build with the device SDK.

Explanation
The format used in Mac OS X and iPhone OS libraries and executable files is Mach-O. The format supports "fat" binaries, which are pretty much concatenated Mach-O files for different architectures, with a thin header.

Xcode can build fat binaries. In fact, if you set Architectures to Optimized in your project's build options, you'll get a fat binary containing both ARMv6 (iPhone 2G + 3G, iPod Touch) and ARMv7 (iPhone 3GS, iPod Touch 2G + 3G) code.

Xcode's output is controlled by the options ARCH (Architectures) and VALID_ARCH (Valid architectures). The architectures that actually get built are the intersections of these two options. Due to the different ARM processors in iPhones, device builds have VALID_ARCH set to include ARMv6 and ARMv7. However, simulator builds only target the i386 platform. I want a fat binary for the simulator as well, so I change the VALID_ARCH option to include the AMD64 platform.

The last step in the process is the lipo tool that comes with the Xcode Tools, and manages fat binaries. I use it to create a fat binary containing the union of the architectures in the two static libraries. The device build contributes the ARM architectures, and the simulator build contributes the Intel architectures.

The build process can be tweaked to throw out the AMD64 code, but I wanted to avoid hard-coding processor constants. Most importantly, using a fat library does not translate to larger iPhone applications, because the GNU linker strips out unused architectures when building binaries.

Testing
I tested this method by creating a iPhone static library, and an iPhone Window Application. I built the library using the steps above, and I included the headers and static library in the application. Then I used the debugger to confirm that the library code works both on the simulator and on an iPod Touch 2G.

My solution was only tested with the latest version of Xcode at the time of this writing (Xcode 3.1.2), but it should work on any version of Xcode 3, assuming no bugs come up. I tested on Snow Leopard, but the Leopard version of the iPhone SDK should work as well.

References
Mac Dev Center: Mac OS X ABI Reference
Amit Singh: Mac OS X Internals

iPhone Development and Code Sharing

This post describes an elegant method for sharing code between iPhone applications. According to my knowledge, the techniques in this post are the first major breakthrough over manually configuring a static library, or building a SDK.
The post starts with an explanation of how I package code in an Xcode project, so that the code can be reused. The structure description is followed by a step-by-step guide of importing reusable code in another Xcode project. My process has the following advantages:

• the reusable code is imported in the consumer Xcode project, so it is built together with the consumer code, and the developer has full browsing and debugging support
• the reusable code that was imported into a project can easily be updated to newer versions

Update: I have open-sourced the iPhone toolkit that I used when writing this article. This means you can find an example packaged project structure here, and you can read more about the toolkit here. 

Reusable Xcode Projects

The picture on the right shows an Xcode project wrapping the code to be reused.

First, notice how all the code has been tucked away under a single folder, with the same name as the project, which is ZergSupport for this example. This is because everything in the project will be imported directly into Xcode projects that use the code, and we don't want to litter their directory structure.

The other important feature of this project is its targets. The code is grouped into three targets:

1. All the "production" code that is supposed to end up on customers' iPhones is included into a static library with the same name as the project (ZergSupport).
2. Code intended to help test the production code above is included into another static library. I named the library ZergTestSupport in my example, because I don't know an obvious naming convention. This library includes the main library (ZergSupport).
3. Tests for all the code above are separated into another target, whose name is the project name plus the suffix Tests (in this case, ZergSupportTests). This target is an iPhone application, prepared according to the instructions in the Google Toolbox for Mac.

The first two targets are static libraries. Create a static library target by going to Xcode's menu, selecting Project > New Target. Name your library, then select OK.

In order to add files to the static library, first make it the active target, by selecting it from the top-left drop-down (see picture at the right). Then click on the folder holding all your sources in under Groups & Files (Xcode's project tree) and click the checkboxes next to the files that you want in your static library, as shown in the picture below and to the left.

Having to separate sources for each target can be a pain, but it's well worth it. Here's why:

• Shipping testing code in an iPhone application will increase its size. If the application goes above 10Mb, people cannot download it over a cellular connection, and need either WiFi or a computer. This means less customers. So separate test code from production code.
• Some open-source licenses (e.g. GPL, BSD with advertising clause) tend to have more relaxed requirements if the code is only used in-house. Testing code meets this condition, as long as it doesn't end up in the binary that is sold. So separate test code from production code :)
• People using your reusable code should not have to run your unit tests (and wait for them), if they don't modify your code. So separate test code from test-supporting code.

Last, maintaining target membership can be simplified by using good naming conventions, and a tool I wrote. For instance, suppose you have two targets, CodeTargetName (for the code that ends up in the client) and TestTargetName (for the test cases), and all your test code is in files whose names end in Test (example: RssReader.m, RssReader.h, RssReaderTest.m). The following commands in your project folder will bucket your files correctly between your targets.

$ zerg-xcode retarget . ".*" CodeTargetName
$ zerg-xcode retarget . "Test\.m$" TestTargetName

The commands above use zerg-xcode, a tool that is introduced below.

Importing Reusable Code

Fortunately, using code packaged in the manner described above is much easier than packaging the code.

The instructions below assume that the library project (ZergSupport in my example) and the project that consumes it are in sibling folders. For example, I have all my Xcode projects in ~/xcodes so I would download the reusable project in ~/xcodes/ZergSupport and my application, which wants to use it, would be in ~/xcodes/MyApp. The following commands will perform the import, when issued from the directory of the consumer application (MyApp).

$ sudo gem install zerg-xcode
$ zerg-xcode import ../ZergSupport
$ zerg-xcode addlibrary ZergSupport MyApp
$ zerg-xcode addlibrary ZergTestSupport MyAppTests

The first command installs the tool that does the importing. The second command adds everything in the ZergSupport Xcode project to the MyApp project, and copies all the necessary files. The last two command add the right library dependencies - my application will include the "production" code in ZergSupport, and my tests will include the testing helpers.

Last, and probably most important - updating to a newer version of the reusable code is achieved by downloading the new version in the same place as the old one, and re-issuing the import command above. The importing logic will preserve dependencies.

Conclusion

Code reuse doesn't come easy in iPhone development, especially because frameworks are forbidden. This post presents a method that facilitates code reuse. I hope my work will facilitate the appearance of open-sourced components that will build up to an infrastructure of similar depth and quality as Rails.

Recognition

I learned about using static libraries on the iPhone in this awesome blog post. I used that as my starting point, and tried to automate the process as much as possible.

Hello, Xcode!

This post is "the making of" for a tool I'm writing that does automated modifications to Xcode projects. If you want to use the tool, or see the code, go to http://github.com/costan/zerg_xcode (scroll down to the bottom for the instructions).

Motivation
To the best of my knowledge, Xcode does not have a public API for interacting with its projects (asides from building with xcodebuild), and there is no tool out there filling this gap.

Xcode has a pretty good user interface, so it may seem that such a tool isn't worth the trouble. I beg to differ. Do you have unit tests? If so, you know they belong in a separate target... and once you have more than one target, you've seen Xcode's less-then-stellar UI for managing target membership. Wouldn't it be nice to have all the files ending in Test.m automatically placed in your unit test target?

Managing test case membership is not that difficult. But then came the iPhone. Xcode has no decent way of incorporating other people's code in a project. My standard for "decent" is being able to add someone's code quickly, and then change that code, as well as receive upstream updates.

Last but not least, the project format is rather readable. I think it was designed to be understood, and I assume Apple folks won't be unhappy to see programmatic access to their format.

My Goals
I hope that, one day, iPhone applications will be as easy to develop as Rails applications. This is what I would like to get done, eventually:

• merge targets from an Xcode project to another project; then we could have libraries that are as easy to integrate into projects as Rails' plugins (done)
• sync between Xcode file listing and on-disk files; this would allow me to move files around in sub-folders, and have Xcode reflect my actions automatically; also, I would be sure I didn't forget to delete files that I removed from my Xcode project; this is especially important for headers, which can impact a build even if they're not in the project
  
• build an iPhone application with all the settings needed for submission to the iTunes store
  

Method
I know I can't possibly figure out how people will want to interact with their Xcode projects. After all, Apple tried and didn't get it perfectly right. So I wanted to make my tool inviting to use and learn, so fellow developers can code up the functionality they need quickly, and hopefully contribute it back to the project, so everyone's life is easier.

I developed the tool in Ruby, because I know and like the language. I packaged it using Rubygems, which comes pre-installed on OSX Tiger and above, so the tool can be installed with one command (the download happens automatically). I hope the quick installation will lower the barrier to adoption, and get me users, some of which will become developers.

I wrote a setup for Ruby's interactive sell (irb) so people can easily explore the structure of Xcode projects, and even experiment with changes. The 22 lines of code paid for themselves many times over, as I used the interactive shell myself to figure out. In the end, people will try to improve the tool if the time it saves them exceeds the time it takes them to learn the tool plus the time it takes to code the change. I hope the interactive shell tilts the scale in my favor.

I tried to make my code look decent, so it doesn't turn away developers who want to try changing it. I pushed big parts in separate directories, because having less to read is always nice. I used a plug-in architecture (don't think it's a big deal, it's less than 40 lines of code) to make it easy for others to implement new commands, and make it easy for me to intergrate the changes.

I wrote tests while developing, so I can feel when my API sucks, and so I can have good examples for using the API. This is asides from the traditional use of tests to assue quality.

Call for Contributions
I wrote this code because I didn't want to do repetitive actions while working with Xcode. I hope I'm not alone.

My code is a good foundation for tweaking Xcode projects. But a good foundation is nothing without good features on top. This is where you come in. Fork the project on Github, do something, and send me your changes!