Fat iPhone Static Libraries: One File = Device + Simulator Code

Abstract
This post explains how to build a static iPhone library that contains ARM code (for iPhones and iPod Touches) and x86 code (for the simulator), in a single file.

This is useful for companies which do not wish to expose their source code when distributing iPhone code, e.g. AdMob. The currently accepted practice is to distribute separate static library files for devices and for the simulator. This means the that the library clients have to setup separate targets for the simulator and device. The separate targets duplicate most information, and only differ in the static libraries included. This violates the DRY principle,

Solution

Open a Terminal, cd into your library's Xcode project directory, and issue the following commands.

xcodebuild -sdk iphoneos3.0 "ARCHS=armv6 armv7" clean build

xcodebuild -sdk iphonesimulator3.0 "ARCHS=i386 x86_64" "VALID_ARCHS=i386 x86_64" clean build

lipo -output build/libClosedLib.a -create build/Release-iphoneos/libClosedLib.a build/Release-iphonesimulator/libClosedLib.a

The commands above assume your library's name is ClosedLib and you're targeting iPhone OS 3.0. It should be straightforward to tweak the commands to accomodate different library names and SDK versions.


Assumptions
I assume that the library is not updated too often, so a couple of manual steps are acceptable. I would like to make add an automatic build feature to my zerg-xcode tool, but there is no ETA for that. I also assume that your project's default target is the static library.

I do not assume that the device and simulator SDKs have similar headers. The beauty of my method is that the simulator code is built with the simulator SDK, and the device code is build with the device SDK.

Explanation
The format used in Mac OS X and iPhone OS libraries and executable files is Mach-O. The format supports "fat" binaries, which are pretty much concatenated Mach-O files for different architectures, with a thin header.

Xcode can build fat binaries. In fact, if you set Architectures to Optimized in your project's build options, you'll get a fat binary containing both ARMv6 (iPhone 2G + 3G, iPod Touch) and ARMv7 (iPhone 3GS, iPod Touch 2G + 3G) code.

Xcode's output is controlled by the options ARCH (Architectures) and VALID_ARCH (Valid architectures). The architectures that actually get built are the intersections of these two options. Due to the different ARM processors in iPhones, device builds have VALID_ARCH set to include ARMv6 and ARMv7. However, simulator builds only target the i386 platform. I want a fat binary for the simulator as well, so I change the VALID_ARCH option to include the AMD64 platform.

The last step in the process is the lipo tool that comes with the Xcode Tools, and manages fat binaries. I use it to create a fat binary containing the union of the architectures in the two static libraries. The device build contributes the ARM architectures, and the simulator build contributes the Intel architectures.

The build process can be tweaked to throw out the AMD64 code, but I wanted to avoid hard-coding processor constants. Most importantly, using a fat library does not translate to larger iPhone applications, because the GNU linker strips out unused architectures when building binaries.

Testing
I tested this method by creating a iPhone static library, and an iPhone Window Application. I built the library using the steps above, and I included the headers and static library in the application. Then I used the debugger to confirm that the library code works both on the simulator and on an iPod Touch 2G.

My solution was only tested with the latest version of Xcode at the time of this writing (Xcode 3.1.2), but it should work on any version of Xcode 3, assuming no bugs come up. I tested on Snow Leopard, but the Leopard version of the iPhone SDK should work as well.

References
Mac Dev Center: Mac OS X ABI Reference
Amit Singh: Mac OS X Internals

Toolkit for Web Service-Backed iPhone Apps

This post describes the chunk of iPhone code that I have recently open sourced (edit: I wrote outsourced before; Epic FAIL). I wrote the code while developing the StockPlay trading simulation game, because the currently released iPhone SDK does not ship with a good infrastructure for building applications that talk to Web services.

Overview

I named the toolkit ZergSupport, and you can get it from my GitHub repository. The README file contains a thorough description of the library so, instead of rehashing that, my post will highlight the main reasons why you should care about the toolkit, and discuss some of the thoughts that went into writing this code.

The code is organized as a toolkit not a framework, which means that ZergSupport is a collection of supporting classes, and does not impose a rigid architecture on your application, like a framework would. As you read this post, please keep in mind that you can use the parts that you want, and ignore everything else. ZergSupport is liberally licensed under the MIT license, so feel free to go to GitHub and jump right into it, as soon as this post convinces you that it's useful.

Web Service Communication

Without further ado, this is how data exchange is done.

The code above makes a Web service call, passing in the data in the user and device models, and parsing data that comes back into models. The data that gets passed to the Web server is formatted such that Rails and PHP would parse the models into hashes, which fits right into how Rails structures its forms. The code expects the Web service to respond in XML format, as implied by the ZNXmlHttpRequest class name. The models in the Web service response are sent to processResponse:, as an array of models.

You have to agree that the code above is much more convenient than having to deal with low-level HTTP yourself. That is, unless setting up the models is a real hassle. Read on, to see how easy (I hope) it is to declare models.

Models

On Mac OS X, you have Core Data to help you with your models. Sadly, this feature didn't make it into iPhone 2.x, so you have to write your own model code. Since StockPlay works with a lot of models, I couldn't write quick hacks and ignore this underlying problem. Actually, I could have, but I didn't want to.

The following listing shows an example ZergSupport model declaration.

The model's attributes are defined as Objective C 2.0 properties. I did this to keep the code as DRY as possible, thinking that models will need accessor methods anyway, and having explicit property declarations makes Xcode be happy and not clutter up the code window with compilation warnings. Right now, the model declaration is a big FAIL in terms of DRY, because the iPhone Objective C runtime requires declaring fields to back up the properties.  However, the 64-bit Objective C supports omitting the field declarations, so I have reason to hope that the iPhone runtime will do this as well, eventually.

An advantage I liked for using properties to declare model attributes is that the model declaration is plain code, which is easy to work with using version control, and is easy to code-review. I think this is as close as it gets to the convenience of Rails models.

Models And The Web
Models change, usually by gaining more attributes. If you're writing an iPhone application on top of an MVC (e.g. Rails) Web service, your iPhone models will probably mirror the Web models. I assert that this strategy can only work well if the iPhone code is capable of ignoring model attributes that it does not understand. The motivation is that models change over the life time of the application, and most of the time they change by gaining attributes. If your iPhone code cannot handle unknown attributes, you have to synchronize your Web server changes with your iPhone application release dates, which is a pain.

So, the ZergSupport models accept unknown attributes. In fact, they go one step further, and store unknown attributes as they are, so these attributes survive serialization / de-serialization. This is particularly handy for using iPhone-side models to cache server-side models. As soon as the server emits new attributes, these are stored on the iPhone cache, ready to be used by a future version of the application.

Just One More Thing (x5)
ZergSupport model serializers and deserializers can convert between iPhoneVariableCasing and script_variable_casing on the fly, so your iPhone models follow the iPhone's Objective C naming conventions, and your server-side models follow the naming conventions in your Web application language.

The toolkit includes reusable bits of logic that can come in handy for Web service-based iPhone applications, such as a communication controller that regularly synchronizes models between the iPhone and the Web server.

The ZergSupport code base packages a subset of Google's Toolkit for Mac that provides unit testing. You create unit tests simply by adding a new executable target to your project, and including the testing libraries into it. The testing code is wrapped in a separate target from the main code, so you don't ship unit testing code in your final application.

Speaking of testing, ZergSupport has automated test cases covering all its functionality. The Web service code is tested by an open-source mock Web service which is hosted for free, courtesy of Heroku Garden. I used a hosting service because I wanted to make sure that the OS integration works correctly, and to allow any user and contributor to run the unit tests without the hassle of server setup.

Last, but definitely not least, the ZergSupport code can be automatically imported into your project, using the zerg-xcode tool that I have open-sourced earlier this year.

Conclusion
The final conclusion is yours. I hope you will find the ZergSupport toolkit useful, and incorporate it in your code. I promise to share all future improvements that I make to ZergSupport, and I hope you will do the same, should you find it useful enough to look through the code and change it.

iPhone Development and Code Sharing

This post describes an elegant method for sharing code between iPhone applications. According to my knowledge, the techniques in this post are the first major breakthrough over manually configuring a static library, or building a SDK.
The post starts with an explanation of how I package code in an Xcode project, so that the code can be reused. The structure description is followed by a step-by-step guide of importing reusable code in another Xcode project. My process has the following advantages:

• the reusable code is imported in the consumer Xcode project, so it is built together with the consumer code, and the developer has full browsing and debugging support
• the reusable code that was imported into a project can easily be updated to newer versions

Update: I have open-sourced the iPhone toolkit that I used when writing this article. This means you can find an example packaged project structure here, and you can read more about the toolkit here. 

Reusable Xcode Projects

The picture on the right shows an Xcode project wrapping the code to be reused.

First, notice how all the code has been tucked away under a single folder, with the same name as the project, which is ZergSupport for this example. This is because everything in the project will be imported directly into Xcode projects that use the code, and we don't want to litter their directory structure.

The other important feature of this project is its targets. The code is grouped into three targets:

1. All the "production" code that is supposed to end up on customers' iPhones is included into a static library with the same name as the project (ZergSupport).
2. Code intended to help test the production code above is included into another static library. I named the library ZergTestSupport in my example, because I don't know an obvious naming convention. This library includes the main library (ZergSupport).
3. Tests for all the code above are separated into another target, whose name is the project name plus the suffix Tests (in this case, ZergSupportTests). This target is an iPhone application, prepared according to the instructions in the Google Toolbox for Mac.

The first two targets are static libraries. Create a static library target by going to Xcode's menu, selecting Project > New Target. Name your library, then select OK.

In order to add files to the static library, first make it the active target, by selecting it from the top-left drop-down (see picture at the right). Then click on the folder holding all your sources in under Groups & Files (Xcode's project tree) and click the checkboxes next to the files that you want in your static library, as shown in the picture below and to the left.

Having to separate sources for each target can be a pain, but it's well worth it. Here's why:

• Shipping testing code in an iPhone application will increase its size. If the application goes above 10Mb, people cannot download it over a cellular connection, and need either WiFi or a computer. This means less customers. So separate test code from production code.
• Some open-source licenses (e.g. GPL, BSD with advertising clause) tend to have more relaxed requirements if the code is only used in-house. Testing code meets this condition, as long as it doesn't end up in the binary that is sold. So separate test code from production code :)
• People using your reusable code should not have to run your unit tests (and wait for them), if they don't modify your code. So separate test code from test-supporting code.

Last, maintaining target membership can be simplified by using good naming conventions, and a tool I wrote. For instance, suppose you have two targets, CodeTargetName (for the code that ends up in the client) and TestTargetName (for the test cases), and all your test code is in files whose names end in Test (example: RssReader.m, RssReader.h, RssReaderTest.m). The following commands in your project folder will bucket your files correctly between your targets.

$ zerg-xcode retarget . ".*" CodeTargetName
$ zerg-xcode retarget . "Test\.m$" TestTargetName

The commands above use zerg-xcode, a tool that is introduced below.

Importing Reusable Code

Fortunately, using code packaged in the manner described above is much easier than packaging the code.

The instructions below assume that the library project (ZergSupport in my example) and the project that consumes it are in sibling folders. For example, I have all my Xcode projects in ~/xcodes so I would download the reusable project in ~/xcodes/ZergSupport and my application, which wants to use it, would be in ~/xcodes/MyApp. The following commands will perform the import, when issued from the directory of the consumer application (MyApp).

$ sudo gem install zerg-xcode
$ zerg-xcode import ../ZergSupport
$ zerg-xcode addlibrary ZergSupport MyApp
$ zerg-xcode addlibrary ZergTestSupport MyAppTests

The first command installs the tool that does the importing. The second command adds everything in the ZergSupport Xcode project to the MyApp project, and copies all the necessary files. The last two command add the right library dependencies - my application will include the "production" code in ZergSupport, and my tests will include the testing helpers.

Last, and probably most important - updating to a newer version of the reusable code is achieved by downloading the new version in the same place as the old one, and re-issuing the import command above. The importing logic will preserve dependencies.

Conclusion

Code reuse doesn't come easy in iPhone development, especially because frameworks are forbidden. This post presents a method that facilitates code reuse. I hope my work will facilitate the appearance of open-sourced components that will build up to an infrastructure of similar depth and quality as Rails.

Recognition

I learned about using static libraries on the iPhone in this awesome blog post. I used that as my starting point, and tried to automate the process as much as possible.

Hello, Xcode!

This post is "the making of" for a tool I'm writing that does automated modifications to Xcode projects. If you want to use the tool, or see the code, go to http://github.com/costan/zerg_xcode (scroll down to the bottom for the instructions).

Motivation
To the best of my knowledge, Xcode does not have a public API for interacting with its projects (asides from building with xcodebuild), and there is no tool out there filling this gap.

Xcode has a pretty good user interface, so it may seem that such a tool isn't worth the trouble. I beg to differ. Do you have unit tests? If so, you know they belong in a separate target... and once you have more than one target, you've seen Xcode's less-then-stellar UI for managing target membership. Wouldn't it be nice to have all the files ending in Test.m automatically placed in your unit test target?

Managing test case membership is not that difficult. But then came the iPhone. Xcode has no decent way of incorporating other people's code in a project. My standard for "decent" is being able to add someone's code quickly, and then change that code, as well as receive upstream updates.

Last but not least, the project format is rather readable. I think it was designed to be understood, and I assume Apple folks won't be unhappy to see programmatic access to their format.

My Goals
I hope that, one day, iPhone applications will be as easy to develop as Rails applications. This is what I would like to get done, eventually:

• merge targets from an Xcode project to another project; then we could have libraries that are as easy to integrate into projects as Rails' plugins (done)
• sync between Xcode file listing and on-disk files; this would allow me to move files around in sub-folders, and have Xcode reflect my actions automatically; also, I would be sure I didn't forget to delete files that I removed from my Xcode project; this is especially important for headers, which can impact a build even if they're not in the project
  
• build an iPhone application with all the settings needed for submission to the iTunes store
  

Method
I know I can't possibly figure out how people will want to interact with their Xcode projects. After all, Apple tried and didn't get it perfectly right. So I wanted to make my tool inviting to use and learn, so fellow developers can code up the functionality they need quickly, and hopefully contribute it back to the project, so everyone's life is easier.

I developed the tool in Ruby, because I know and like the language. I packaged it using Rubygems, which comes pre-installed on OSX Tiger and above, so the tool can be installed with one command (the download happens automatically). I hope the quick installation will lower the barrier to adoption, and get me users, some of which will become developers.

I wrote a setup for Ruby's interactive sell (irb) so people can easily explore the structure of Xcode projects, and even experiment with changes. The 22 lines of code paid for themselves many times over, as I used the interactive shell myself to figure out. In the end, people will try to improve the tool if the time it saves them exceeds the time it takes them to learn the tool plus the time it takes to code the change. I hope the interactive shell tilts the scale in my favor.

I tried to make my code look decent, so it doesn't turn away developers who want to try changing it. I pushed big parts in separate directories, because having less to read is always nice. I used a plug-in architecture (don't think it's a big deal, it's less than 40 lines of code) to make it easy for others to implement new commands, and make it easy for me to intergrate the changes.

I wrote tests while developing, so I can feel when my API sucks, and so I can have good examples for using the API. This is asides from the traditional use of tests to assue quality.

Call for Contributions
I wrote this code because I didn't want to do repetitive actions while working with Xcode. I hope I'm not alone.

My code is a good foundation for tweaking Xcode projects. But a good foundation is nothing without good features on top. This is where you come in. Fork the project on Github, do something, and send me your changes!