Fat iPhone Static Libraries: One File = Device + Simulator Code

Abstract
This post explains how to build a static iPhone library that contains ARM code (for iPhones and iPod Touches) and x86 code (for the simulator), in a single file.

This is useful for companies which do not wish to expose their source code when distributing iPhone code, e.g. AdMob. The currently accepted practice is to distribute separate static library files for devices and for the simulator. This means the that the library clients have to setup separate targets for the simulator and device. The separate targets duplicate most information, and only differ in the static libraries included. This violates the DRY principle,

Solution

Open a Terminal, cd into your library's Xcode project directory, and issue the following commands.

xcodebuild -sdk iphoneos3.0 "ARCHS=armv6 armv7" clean build

xcodebuild -sdk iphonesimulator3.0 "ARCHS=i386 x86_64" "VALID_ARCHS=i386 x86_64" clean build

lipo -output build/libClosedLib.a -create build/Release-iphoneos/libClosedLib.a build/Release-iphonesimulator/libClosedLib.a

The commands above assume your library's name is ClosedLib and you're targeting iPhone OS 3.0. It should be straightforward to tweak the commands to accomodate different library names and SDK versions.


Assumptions
I assume that the library is not updated too often, so a couple of manual steps are acceptable. I would like to make add an automatic build feature to my zerg-xcode tool, but there is no ETA for that. I also assume that your project's default target is the static library.

I do not assume that the device and simulator SDKs have similar headers. The beauty of my method is that the simulator code is built with the simulator SDK, and the device code is build with the device SDK.

Explanation
The format used in Mac OS X and iPhone OS libraries and executable files is Mach-O. The format supports "fat" binaries, which are pretty much concatenated Mach-O files for different architectures, with a thin header.

Xcode can build fat binaries. In fact, if you set Architectures to Optimized in your project's build options, you'll get a fat binary containing both ARMv6 (iPhone 2G + 3G, iPod Touch) and ARMv7 (iPhone 3GS, iPod Touch 2G + 3G) code.

Xcode's output is controlled by the options ARCH (Architectures) and VALID_ARCH (Valid architectures). The architectures that actually get built are the intersections of these two options. Due to the different ARM processors in iPhones, device builds have VALID_ARCH set to include ARMv6 and ARMv7. However, simulator builds only target the i386 platform. I want a fat binary for the simulator as well, so I change the VALID_ARCH option to include the AMD64 platform.

The last step in the process is the lipo tool that comes with the Xcode Tools, and manages fat binaries. I use it to create a fat binary containing the union of the architectures in the two static libraries. The device build contributes the ARM architectures, and the simulator build contributes the Intel architectures.

The build process can be tweaked to throw out the AMD64 code, but I wanted to avoid hard-coding processor constants. Most importantly, using a fat library does not translate to larger iPhone applications, because the GNU linker strips out unused architectures when building binaries.

Testing
I tested this method by creating a iPhone static library, and an iPhone Window Application. I built the library using the steps above, and I included the headers and static library in the application. Then I used the debugger to confirm that the library code works both on the simulator and on an iPod Touch 2G.

My solution was only tested with the latest version of Xcode at the time of this writing (Xcode 3.1.2), but it should work on any version of Xcode 3, assuming no bugs come up. I tested on Snow Leopard, but the Leopard version of the iPhone SDK should work as well.

References
Mac Dev Center: Mac OS X ABI Reference
Amit Singh: Mac OS X Internals

iPhone Web Service Toolkit Upgrade: JSON FTW

I have recently open-sourced the ZergSupport code updates used in StockPlay versions 0.3 and 0.4. The high-visibility high-impact change is support for JSON parsing. This post shows what you can do with JSON parsing.

Compact Collection Initialization
Unfortunately, Objective C does not have literals for collections (arrays, dictionaries, or sets). Setting up complex nested structures normally requires ugly objective C code. Fortunately, the whole JSON specification is a compact literal notation. Compare and contrast the following.



To sweeten the deal even more, ZergSupport's JSON parser was extended so you can conveniently embed literals in Objective C strings. First, strings can be delimited by ' (single quotes) asides from the standard delimiter " (double quotes). Second, the extended parser understands sets, which look like arrays, but are delimited by angled brackets ( < > ) instead of square brackets ( [ ] ). Without further ado, here's how to use JSON literal support.



A small wrinkle to be aware of is that JSON parsing is slower than building the collections directly in Objective C, so JSON literals should be used in features that are not performance-sensistive, like tests and configuration files.

Web Services
The API for working with JSON Web services is very similar to the API for XML services, which is showcased in my first post on ZergSupport. When used from the Web service API, the JSON parser ignores everything up to the first { character, so it is able to parse JSONP output.

The code below is a complete implementation for stock ticker symbol search, using Yahoo Finance. The code uses Object Query (covered below) to indicate which parts of the JSON response should be transformed into ModelSupport models.



Object Query
Object Query implements a domain-specific language (DSL) for retrieving objects from a structure of deeply nested Cocoa collections. Queries are specified as strings, and are performed against the root object in a structure of nested collections. The result of a query is an array of zero or more objects matching the query.

The queries are property names, joined by a separator character (usually /). The first character in the query is the separator character. For example /results/1 matches the value that object['results'][1] would return in JavaScript. The special property names * and ? are inspired from glob expressions: * means the next property may be found in the current object or in some descendant object (some levels deeper in the object graph), whereas ? means the next property may be found in the current object or in the object's direct children (one level deeper in the object graph). For example /?/1 will return the same result as /results/1, if the initial object is {'results':['Jane','John']} (the result will be a NSArray containing the NSString @"John").

ObjectQuery can be used directly, outside of JsonHttpRequest, as shown in the following code sample.



The decision to introduce a DSL is motivated by the need to extract ModelSupport models from JSON Web service responses. XML objects have a tag/content separation, and tags are usually good indicators for model extraction purposes. For JSON objects, the closest equivalent to a tag name would be the property name whose value is the object hash describing the model. This is fragile, and does not work if the response contains an array of models. ObjectQuery is a bit more general than name tags, but does not degenerate into XPath's complexity. The implementation is 182 lines of Objective C, including comments and whitespace.

Conclusion
JSON support does not stop at the parser. The toolkit fully embraces JSON, which is now available for initializing ModelSupport models. The toolkit's new Object Query bridges the gap between XML-based and JSON-based Web Services, and preserves API consistency in WebSupport.

Thank you reading this post! I'm looking forward to receiving your feedback or (even better) pull requests for ZergSupport.

Community Effort for iPhone Application Security

This post is a short description of the community effort I'm trying to start around the iPhone application security model. It describes the effort, my motives for starting it, and the method I have chosen. The effort is hosted on George Hotz' theiphonewiki.com, with George's permission.

Effort

I have created an Application Copy Protection section on The iPhone Wiki. I hope that the wiki will become a place for developers to pool their knowledge on iPhone application security. In turn, this will make iPhone development less expensive and more enjoyable. Ideally, we would develop a code obfuscation method, as well as a server-side integrity check method, which are non-trivial to reverse. Once there is a barrier against automated programs and beginner crackers, piracy will hopefully go down to a more acceptable rate.

Motivation
I'm dissatisfied with the asymmetry in the iPhone security landscape. On one hand, application pirates have a good infrastructure, ranging from tutorials to the Crackulous application for automated piracy, and to the Appulous infrastructure for distributing pirated applications. On the other hand, developers have to fight many unknowns, like the unspecified signature system, because Apple designed the system on the assumption that developers will not have to worry about copy protection themselves. Application security information is spread across Apple's documentation and various blogs and forums, which makes it hard for developers to learn and implement application security.

I'm also unhappy with RIPdev's approach of charging setup fees and royalties, because the application developers are already paying Apple an up-front development fee, as well as distribution fees.

Last but not least, I'm obviously a bit upset that my application got pirated the next day after it launched in the iTunes store :)

Method
I am documenting my thought process and method for establishing the effort for historical reasons. They will hopefully be useful to other people who want to start similar initiatives.

I wanted to do a wiki on iPhone application security, but I didn't take the time to think the logistics until recently. I was initially thinking of opening a Google Site, and adding as a collaborator any person that would e-mail me an useful piece of information. Then I realized that Google don't look as open to contributions as Wikis do. At the same time, I also started thinking about getting visibility for the site. After a bit of thinking, I realized I'm better off hosting the effort on The iPhone Wiki, because it's already a well-known site, its topic is security on the iPhone, and it contains information that can be useful to developers researching application security.

After I decided on The iPhone Wiki, I did some googling to find out that it was started by George Hotz, and I read the wiki's Constitution to see if my effort belonged there. I was still unsure if my effort fits in, so I decided to ask for George's permission. After some more googling, I eventually tracked him down, and he gave his consent quickly.

Having gotten George's consent, I spent a bit of time thinking of the best way to blend the topics I wanted to add with the existing content on the Wiki. I chose to create a separate section named Application Copy Protection on the front page, and created a skeleton under it. This optimizes for visibility, and makes it easy for me to optimize my thoughts, but may not be the best solution for the overall site. Fortunately, it's a Wiki, so I don't have to worry too much. If I made a mistake, someone else will jump in and fix it.

My next steps are:

• contribute enough content to make the wiki worth reading for iPhone application developers
• create a skeleton for what I think the rest of the content should be, so other people can easily jump in and contribute their knowledge
• pitch the effort to high-traffic iPhone-related blogs, to make developers aware of the Wiki; the fact that the pages are hosted on The iPhone Wiki should help
  

Conclusion
This is my first grown-up attempt at starting a community effort. I would appreciate any suggestions or generic feedback. I hope you found the post at least amusing, if not useful.

Toolkit for Web Service-Backed iPhone Apps

This post describes the chunk of iPhone code that I have recently open sourced (edit: I wrote outsourced before; Epic FAIL). I wrote the code while developing the StockPlay trading simulation game, because the currently released iPhone SDK does not ship with a good infrastructure for building applications that talk to Web services.

Overview

I named the toolkit ZergSupport, and you can get it from my GitHub repository. The README file contains a thorough description of the library so, instead of rehashing that, my post will highlight the main reasons why you should care about the toolkit, and discuss some of the thoughts that went into writing this code.

The code is organized as a toolkit not a framework, which means that ZergSupport is a collection of supporting classes, and does not impose a rigid architecture on your application, like a framework would. As you read this post, please keep in mind that you can use the parts that you want, and ignore everything else. ZergSupport is liberally licensed under the MIT license, so feel free to go to GitHub and jump right into it, as soon as this post convinces you that it's useful.

Web Service Communication

Without further ado, this is how data exchange is done.

The code above makes a Web service call, passing in the data in the user and device models, and parsing data that comes back into models. The data that gets passed to the Web server is formatted such that Rails and PHP would parse the models into hashes, which fits right into how Rails structures its forms. The code expects the Web service to respond in XML format, as implied by the ZNXmlHttpRequest class name. The models in the Web service response are sent to processResponse:, as an array of models.

You have to agree that the code above is much more convenient than having to deal with low-level HTTP yourself. That is, unless setting up the models is a real hassle. Read on, to see how easy (I hope) it is to declare models.

Models

On Mac OS X, you have Core Data to help you with your models. Sadly, this feature didn't make it into iPhone 2.x, so you have to write your own model code. Since StockPlay works with a lot of models, I couldn't write quick hacks and ignore this underlying problem. Actually, I could have, but I didn't want to.

The following listing shows an example ZergSupport model declaration.

The model's attributes are defined as Objective C 2.0 properties. I did this to keep the code as DRY as possible, thinking that models will need accessor methods anyway, and having explicit property declarations makes Xcode be happy and not clutter up the code window with compilation warnings. Right now, the model declaration is a big FAIL in terms of DRY, because the iPhone Objective C runtime requires declaring fields to back up the properties.  However, the 64-bit Objective C supports omitting the field declarations, so I have reason to hope that the iPhone runtime will do this as well, eventually.

An advantage I liked for using properties to declare model attributes is that the model declaration is plain code, which is easy to work with using version control, and is easy to code-review. I think this is as close as it gets to the convenience of Rails models.

Models And The Web
Models change, usually by gaining more attributes. If you're writing an iPhone application on top of an MVC (e.g. Rails) Web service, your iPhone models will probably mirror the Web models. I assert that this strategy can only work well if the iPhone code is capable of ignoring model attributes that it does not understand. The motivation is that models change over the life time of the application, and most of the time they change by gaining attributes. If your iPhone code cannot handle unknown attributes, you have to synchronize your Web server changes with your iPhone application release dates, which is a pain.

So, the ZergSupport models accept unknown attributes. In fact, they go one step further, and store unknown attributes as they are, so these attributes survive serialization / de-serialization. This is particularly handy for using iPhone-side models to cache server-side models. As soon as the server emits new attributes, these are stored on the iPhone cache, ready to be used by a future version of the application.

Just One More Thing (x5)
ZergSupport model serializers and deserializers can convert between iPhoneVariableCasing and script_variable_casing on the fly, so your iPhone models follow the iPhone's Objective C naming conventions, and your server-side models follow the naming conventions in your Web application language.

The toolkit includes reusable bits of logic that can come in handy for Web service-based iPhone applications, such as a communication controller that regularly synchronizes models between the iPhone and the Web server.

The ZergSupport code base packages a subset of Google's Toolkit for Mac that provides unit testing. You create unit tests simply by adding a new executable target to your project, and including the testing libraries into it. The testing code is wrapped in a separate target from the main code, so you don't ship unit testing code in your final application.

Speaking of testing, ZergSupport has automated test cases covering all its functionality. The Web service code is tested by an open-source mock Web service which is hosted for free, courtesy of Heroku Garden. I used a hosting service because I wanted to make sure that the OS integration works correctly, and to allow any user and contributor to run the unit tests without the hassle of server setup.

Last, but definitely not least, the ZergSupport code can be automatically imported into your project, using the zerg-xcode tool that I have open-sourced earlier this year.

Conclusion
The final conclusion is yours. I hope you will find the ZergSupport toolkit useful, and incorporate it in your code. I promise to share all future improvements that I make to ZergSupport, and I hope you will do the same, should you find it useful enough to look through the code and change it.