A developer’s-eye view of Leopard

Steve Jobs had a lot of fun at Microsoft’s expense over Redmond’s difficulties shipping the operating systems that have become Vista and Windows Server 2008. So with Vista shipping by default on new PCs and Windows Server 2008 in a publicly downloadable beta, Apple should be catching hell from the press for making Leopard the last to arrive.

But the press never gives Apple hell about anything, and there’s nothing like a fresh, closed beta of Leopard to put Mac developers in a forgiving mood.

Now that I’ve done my objective journalist’s duty with an ineffectual finger-wag at Apple, I’ll confess that, personally, having the beta release of Leopard set aside for paid members of Apple Developer Connection (ADC) suits me fine. I’m an ADC Premiere member. I’ll be one of the 4,000 or so who’ll get my Leopard DVD at the WWDC (Worldwide Developers Conference), but far more important is the week of hands-on education that comes with it.

That front-row seat includes a catch: Everything that Apple reveals about Leopard at WWDC 2007 is covered by a blanket nondisclosure agreement. Fortunately, Apple hasn’t left many Leopard details to the imagination, so I am allowed to write at will about those features that Apple has taken public.

I have already written repeatedly, and at length, about Leopard the operating system and Leopard the user experience, but to date, I’ve had relatively little to say about Leopard the application platform. When it comes to the genuinely new facilities it places in the hands of developers, Leopard’s riches are vast, and I’m just scratching the surface here with two standouts: Xray and Core Animation.

The spot of Leopard that gets me all charged up is DTrace, Sun Microsystems’ revolutionary technology that bakes dynamic execution tracing directly into system software. Unfortunately, DTrace is as complicated to use as it is powerful. Fortunately, Mac developers enjoy an advantage that Solaris developers do not: Xray. This is no mere face transplant for DTrace. Xray is a serious developer power tool befitting the trend I see toward platform-aware, performance-oriented development. But even among developers of flat POSIX code, Xray will prove addictive.

Xray’s real-time and postmortem performance profiling capabilities let you record simultaneous traces of program execution, UI interaction, memory and resource allocation, and system counters such as CPU and network utilization, with a genuinely intuitive user interface that resembles a digital audio workstation. Xray can rewind its trace logs to show you a snapshot of the precise state of your application and its running environment, all the way down to the metal.

Developers don’t have to recode to take advantage of Xray, so its effects will be felt across a broad range of applications. Even those mysterious, seemingly random bugs can’t hide from Xray (hence its name): Just run your app until it fails, then rewind. Such tools exist elsewhere, but none can match Xray’s price tag: free.

Graphics get a move on Core Animation is certainly the most demonstrated of the new features that Leopard makes available to developers. In demos, texture-mapped tiles fly in various dazzling and perfectly orchestrated formations, each seeming to have a will and mission of its own. It seems that way because it’s true: Every one of those image tiles does have distinct behavior associated with it. Will it change developers’ lives? Maybe not, but Apple teases Core Animation by saying that it’s suited to development on set-top boxes. As I waited to enter the WWDC keynote, I carried a tiny seed of hope that Apple would open Apple TV to developers.

If you’ve ever tried to write code for OpenGL or, worse still, game code that talks directly to a GPU (graphics processing unit), you quickly understand why Core Animation has developers so excited. Core Animation produces real-time 3-D-like animation using Click for larger view. Apple’s blazingly fast Quartz rendering and compositing frameworks. However, it hides all of the complexity of Quartz and the rest of the Mac platform’s imaging facilities. If you can see it, you can use Core Animation to map it to a rectangle and make it fly around.

Core Animation really couldn’t be easier to use. It’s so easy, in fact, that I can see developers using it even for GUIs and content that doesn’t move. Core Animation is, to simplify the idea, a stack of intelligent surfaces (Apple’s term is “layers”) onto which visual content is projected (mapped). Each surface is intelligent in that it can have developer-defined motion and transformation (that is, shrink, stretch, flip, transparency) paths that play out autonomously.

By stacking and linking these surfaces, it’s possible to have dozens or hundreds of flying images that behave very much like multilayered effects one would create in Final Cut Pro, but without the extra tools. And what Core Animation does, it does in real time.

Core Animation is not a GUI toolkit; surfaces cannot be made directly sensitive to user interaction. However, because Core Animation just creates a 2-D on-screen image, and the position of every surface is temporally deterministic — you know where a surface will be at a given time — developers can map mouse-click sensitivity to Core Animation elements just by watching for mouse and keyboard actions for the entire rendered area.

Apple has also updated base UI window classes so that they can use motion and even incorporate Core Animation layers as part of their presentation. Apple makes extensive use of Core Animation as a user interface technology in core Leopard features such as Quick Look rich document preview services and the Stacks Dock-based folder navigator.

The relative ease with which this can be done paves the way for unimaginably beautiful, knowledge-dense, and productive user interfaces and information displays, not to mention the value in entertainment and consumer electronics software

It is possible to replicate Core Animation in Tiger, but it would require enormous effort. Tiger developers can get a preview of sorts by playing with the Quartz Composer tool bundled with Xcode. In fact, Quartz Composer animations can be used as Core Animation layers, creating movement within movement that can also include Core Image filters and distortions. Apple’s demos don’t do Core Animation justice.

See it, show it, send it Two more goodies round out Leopard’s eye-pleasing developer features: Image Kit coalesces the most commonly used elements among all of OS X’s image rendering and manipulation facilities into a single framework, and iChat Instant Message Framework allows developers to render arbitrary visual content that iChat automatically encodes and streams to participants in an iChat conference.

Apple’s iChat Theater makes use of this facility to share images, presentations, and full-motion video with conference participants. iChat Theater’s approach to content sharing is much cleaner, more bandwidth-efficient, and more secure than the full-screen sharing commonly employed in online conferences.

Image Kit provides simple high-level interfaces for taking snapshots with iSight Webcams, displaying slideshows, modifying images, and applying Core Image transformation filters. Again, these are facilities available to developers working on OS X Tiger, but only through the rather painstaking hand-knitting of multiple facilities. This sort of thing is easy for highly experienced Mac developers.

Platform newcomers, as well as developers in a hurry to get solutions to market, will find that Leopard not only provides unique and highly innovative facilities, it exposes them in ways that welcome developers of all stripes. As a result, Leopard’s new features will be taken up extremely rapidly by developers, extending the transformation of the Mac platform from the OS all the way up end-user applications.

PART TWO: XCODE AND OBJECTIVE-C

Apple has always been pulled in different directions by factions of its user and developer communities. Set-in-their-ways Mac developers whose experience predates OS X have a predilection for the C language, legacy Carbon function libraries, and esoteric development tools in the tradition of Macintosh Programmer’s Workshop and CodeWarrior. Meanwhile, Apple’s UNIX developers and emigrants from Linux go for C and C++ for native code and Python, Perl, and Ruby for dynamic apps, and demand a command line toolset and source code portability so that they can assemble the tools they prefer. And then there are those doing Java client development and programmers writing AppleScript programs who bring their own expectations to tools.

With Leopard, Apple makes its most concerted run yet at drawing developers into creating applications that exploit the full Mac platform, starting with Xcode 3.0, a dramatic set of enhancements to Apple’s free and official IDE. Xcode will be publicly released with Leopard, and it is available now to paid members of Apple Developer Connection.

Polishing Xcode 3.0

IDE editors are often burdened by new features, so I was wary when I learned that Xcode 3.0 was getting an editor overhaul. However, the changes Apple made are aimed at consolidation, not added complexity, and the new editor puts out a very comfortable welcome mat for developers who aren’t big fans of IDEs with crowded toolbars and jarring edit/build/debug modes.

Recognizing that a great many Unix coders live in their editors, Apple decided to bake a lot of traditional IDE functionality into the editor itself. For example, after building a project, the developer can see errors and warnings in their code without opening a build messages window.

Inline balloons point to the locations of errors. During debugging, the editor window becomes the interface for exploring symbols at run-time and for the setting and clearing of simple and conditional breakpoints. And during the all-important editing process itself, Xcode 3.0 aids without interfering thanks to the integrated Code Sense and Research Assistant.

Code Sense code completion is familiar to most IDE users, but Xcode 3.0 now presents a drop-down list of completion options; the selections narrow as the user continues to type. Xcode’s innovative code templates lay out fill-in-the-blank statement completion; merely navigating inside the editor is sufficient to perform context-sensitive documentation lookups. Research Assistant, which runs in its own window, displays a summary paragraph from the on-line documentation for any highlighted keyword.

Also of interest to those developers new to the Mac — and possibly to object-oriented development — are improvements that speed the editor’s handling of large source code files. Apple claims to open and scroll through big monolithic code files up to 10 times faster than before.

Lastly, Xcode incorporates code folding, which collapses sections of code (such as a function or object definition), as well as block highlighting features that distinguish logical blocks of code through shading.

Beyond the editor, Xcode 3.0 has a number of new “grown-up” features that are hard to come by in noncommercial environments and were previously non-existent for Objective-C. One is refactoring, which allows a single code change to ripple through an entire project. Xcode 3.0 refactoring obviates the need for file-by-file find-and-replace after renaming and restructuring classes. It also updates existing Objective-C code to take advantage of Objective-C 2.0 enhancements.

Lastly, Xcode 3.0 strengthens the IDE’s integration with third-party SCM (source code management) facilities. Developers can check files and projects in and out, create new source tree branches, and import and export files to and from the SCM. For those projects that don’t call for full-blown SCM and to recover from unintentional mangling of checked-out files, Project Snapshots let you capture the state of a project so that something like a “global undo” can be done after something goes very wrong.

Objective-C gets an upgrade

Objective-C 2.0 is an update to Apple’s “official” language for the Mac platform. Apple recognized that the language can impose an intimidating learning curve on C, C++, and Java developers, so it introduced features that are more in line with C/C++ coders’ experience.

The greatest of these features is garbage collection. The bane of Mac developers’ existence is the need to manually track memory and resource allocation during the course of an application’s operation. That becomes a big challenge as objects are passed around by reference. Garbage collection addresses this by automatically freeing resources and allocated memory when they’re no longer needed. It takes a toll on performance, but nothing near the penalty imposed by a move to a dynamic language like Java or C#.

Objective-C 2.0 adds a “for” keyword that vastly simplifies enumerating through a collection of objects. It also adds object properties, a technique that I came to appreciate in JavaScript that allows the creation of new member variables using object.member syntax.

Taken as a whole, Xcode 3.0 and the updated Objective-C form the foundation of Mac platform development. They continue to welcome existing Mac developers, greatly enhance the experience of developers who embrace the full platform, and bring up developers who are more attuned to the old school of big code and simple editors. The result will be more high-performance native-code applications for the Mac, and that will be a windfall for Mac users post-Leopard.

PART THREE: COCOA AND OBJECT-ORIENTED FRAMEWORKS

Mac users have high expectations. A newcomer to the platform quickly discovers that all Mac applications share consistent appearance and behavior, are incomparably fast, and come with “Wouldn’t it be great if I could …?” user interface experiments that tend to just work.

Even when Apple supplies several approaches to achieving the same end in order to accommodate developers’ various preferences, the company ensures that all those tactics result in a completely consistent user experience. No matter what software you add to a Mac, the whole system feels like an integrated suite.

Delicious Cocoa Cocoa is the root framework, the primary API, for all native Mac applications written in Objective-C. There are wrappers and bridges that allow using some Cocoa facilities in other languages, including C, C++, Python, and Java, but the only way to suck all of the goodness out of Cocoa and the Mac’s other object-oriented frameworks is to use Objective-C.

Now that Apple is using Objective-C for all of its application development, visual and functional elements that Apple builds into its programs are finding their way into Mac frameworks so that developers can evolve their apps into Leopard’s updated look, feel, and feature set. It’s surprisingly easy to do, and in some cases, developers don’t have to lift a finger — not even recompile their code to add the features that weren’t present in the Tiger release of their application.

The most accessible examples of the zero-effort Tiger-to-Leopard transformation are Text Services and unified window styles. In Tiger, all Cocoa applications that incorporate text-entry fields inherit OS X’s spelling checker. The Cocoa framework that ships with Leopard adds a grammar checker that tries to flag awkward or improper sentence construction and various barriers to readability. Its smart quote support converts open and close quote pairs to tails-up, tails-down style in formatted documents and flat, HTML-friendly quotes for plain text. Hyperlinks found in a Cocoa text-editing field are automatically detected and rendered in a familiar standout underlined style that launches the browser by default. Multiple regions of selected text can be copied and pasted with the same ease as contiguous sentences and paragraphs. With Leopard, text attachments can also be edited inside a document being viewed without the need to pop up a text-editor window.

These might seem like small changes, but they add up to key functionality that developers previously had to code into their applications. As with all features that Apple adds to revisions of its frameworks, everything is optional, and everything can be superclassed to add functionality and make objects that implement an application-unique feature set without sacrificing consistency or compatibility.

Compelling scripts Cocoa provides hooks for making applications scriptable, that is, usable as services by other Mac platform apps. The scripting interface to Mail, for example, is not merely capable of invoking a message composition window. A script that interacts with Mail can extract messages from the inbox, perform searches, and create rules to govern the handling of incoming mail. OS X incorporates an elaborate but thoroughly abstracted dynamic remote procedure invocation mechanism for native applications. It’s based on messages, as is all of Cocoa, and is therefore far more robust than one would expect from interpreted scripts.

While Cocoa’s scripting support exceeds other platforms’ capabilities, it has been practically limited to script-to-application interaction, that is, invoking calls into Cocoa applications from external scripts. Oddly, as useful as it would be to do so, it’s extremely difficult to invoke calls to Cocoa applications from other Cocoa apps.

In Leopard, Cocoa addresses this. A new facility, Scripting Bridge, generates Objective-C code that effectively turns scriptable Cocoa applications into objects whose methods can be invoked from other Cocoa applications; hooking two or more applications together requires no middleware. One inherits this capability simply by adopting Cocoa.

For native applications, adopting Cocoa means adopting Objective-C. C++ lacks Objective-C’s runtime dynamic linking capabilities, while Objective-C is both dynamic and fully compiled. However, there are fully dynamic (interpreted) object-oriented languages, and Apple has chosen to raise two to first-class status: Ruby and Python. By first-class, I mean that Ruby and Python code can share objects with and call into Cocoa and Objective-C, and Cocoa and Objective-C can share with and call Ruby and Python.

With this, Leopard lets some steam out of .Net by extending the platform’s native object-oriented paradigm — not an interpreted layer on top of it — to two established dynamic languages. Cocoa is a runtime layer for object-oriented dynamic applications; there was no need to create one as was done for .Net.

Objects of affection OS X’s pervasive object-oriented paradigm is extensible, and Apple has done a lot of extending in the form of new and enhanced frameworks. Among the frameworks added to Leopard are Publish/Subscribe, which provides applications with classes that handle creating and consuming RSS and Atom feeds. Publish/Subscribe abstracts networks, formats, and protocols, wiping out hundreds of lines of code that applications would require to incorporate RSS functionality without relying on an external reader.

The Publish half of Publish/Subscribe lets applications generate content that the framework packages and sends to a server that supports the Atom Publishing Protocol, which will include Apple’s .Mac service and the Weblog server included in Leopard Server. File deposition via FTP and WebDAV is supported as well.

The Core Data framework embeds high-performance client database functionality into applications. Core Data doesn’t rely on a connection to an external database, but rather on a lightweight database engine embedded in OS X and used by system facilities such as Spotlight. With Leopard, Core Data 2.0 will implement transaction-safe handling of arbitrary file formats to reduce the need to format data expressly to fit in a fixed schema.

The structure of data that needs to be stored changes over time, and a perennial headache for database administrators and developers is managing multiple versions of database schema and nondestructively migrating data from one version of the schema to another. Core Data 2.0 framework handles this. It has also been updated for compatibility with garbage-collected and 64-bit Objective-C applications, and it will feature performance boosts in the form of prefetching and rapid retrieval of object identifiers for those circumstances where only a list of IDs is needed.

Spotlight on Time Machine Leopard’s application framework enhancements are too numerous to squeeze into even a series of stories, but one last standout deserves mention: Leopard incorporates changes to its file system to support enhancements to Spotlight and to enable Time Machine automated backup. Leopard supports metadata searches on remote systems, whereas Spotlight’s implementation in Tiger is limited to local disks. Leopard allows applications to replace the contents of files without overwriting their metadata. Files that take on minor changes don’t need to have potentially extensive metadata rewritten from scratch, which can be problematic when metadata is under the control of another application. This facility also allows the construction of default metadata sets that can be applied to newly created files.

Leopard’s Time Machine tracks changes to files and maintains time-stamped logs of those changes on external servers or media, creating real-time incremental backups. One of Time Machine’s foundations is lineage information, a structured means of establishing relationships between versions of a file. Developer access to lineage data allows them to navigate the file system and effectively open a file as it existed at a given point in time. Imagine adding “when” criteria alongside the file name in a call to open a file.

Cocoa and Objective-C equip developers with capabilities they could only create for themselves with tremendous effort, and for which they’d pay dearly to third parties on Windows. Just as Mac users grow accustomed to having the answer to “Can I …?” come back “yes,” Mac developers find they get the same answer when they ask whether something they need is already built into Leopard’s application frameworks.

PART FOUR: 64-BIT SUPPORT

With its transitions from Mac OS to OS X, PowerPC to Intel, and Panther to Tiger under its belt, Apple is all about moving on. Now it’s the developers’ turn to move on. If you haven’t done it yet, it’s time to bid farewell to C and Carbon, and to embrace Objective-C and Cocoa for your GUI applications. It’s time to count on Universal Binaries, not Rosetta (Apple’s PowerPC translator for Intel Macs), to get your software out to the whole Mac market, which will soon be dominated by 64-bit Intel Macs. If you haven’t yet broken the habits of jamming new icons into the menu bar and turning every convenience utility into a CPU-sapping background process with its own always-on-top window, you should get to know Dashcode. If your application terminates because it can’t locate a critical file, learn the ways of Time Machine. And if, when you think of Web applications, your mind automatically zeroes in on Java, you might look at Ruby on Rails as a far simpler, much lighter-weight open source alternative that’s remarkably well appointed.

Now you’re 64 One of the clearest lines of demarcation separating Leopard from Tiger is 64-bit. Tiger has some 64-bit capabilities, but just enough to satisfy Apple’s old message that 64-bit is all about accessing more than 4GB of RAM. Having a product line tilted heavily toward 32-bit systems, Apple characterized the need for 64-bitness as rare and advised developers to apply it sparingly. For Leopard, Apple’s message has shifted dramatically: Leopard is 64-bit from stem to stern, from kernel to presentation, and developers have the green light to leverage 64-bit platform features. At the 2007 Worldwide Developers Conference, Steve Jobs pointed out that “practically all Macs are 64-bit” (only the Mac Mini remains 32-bit) and presented a demo showing that demanding applications run markedly faster as 64-bit software. The performance advantage is not just about memory: 64-bit gives applications access to double-wide (128 bit) CPU registers and a much larger total register set for speeding complex calculations and reducing the number of trips that a 64-bit optimized application must make to slower RAM.

All of that sounds inviting, but how does one get an application from the 32-bit world to 64? The same way you get from PowerPC to Intel: Let Xcode, OS X, and Apple worry about it. Apple’s Universal Binary executable format folds multiple native architecture types into a single application file with one installer. You can develop and debug your app once on a 64-bit Mac Pro, and then spin out your app to target 32-bit PowerPC, 64-bit PowerPC, 32-bit Intel, and 64-bit Intel in a single executable. Or start with any one of the other architectures and target the other three. You ship your application with one installer, confident that it will work exactly the same way on every Mac that runs Leopard. The OS X loader automatically picks the correct architecture out of the executable at runtime, and the frameworks optimize your application for the user’s system by default.

There’s another reason that Apple wants 64-bit all the way down the line: Apart from speed, Apple wants OS X to be taken seriously as a commercial Unix, right down to equipping Darwin to qualify for the Unix trademark. Apple hopes to raise Darwin to the stature of Solaris, AIX, and HP/UX. If Apple comes up short in its bid for Unix certification for Leopard, it has an ally in Sun Microsystems, which has open-sourced most of its Solaris System V Unix and which makes widespread use of Mac client systems internally. In other words, while Unix certification for Leopard is not a done deal, Apple has a ready base of thoroughly certified code from which to borrow. Leopard will be Unix. Your software should wear that label with pride.

Apple is using the advent of Leopard’s 64-bit pervasiveness to prepare the way for the gradual de-emphasis of the Carbon C API. For example, Apple is not porting the QuickTime, QuickDraw, and Sound Manager C APIs to 64-bit. Instead, it is supplanting them with the 64-bit Objective-C framework’s QuickTime Kit, Cocoa, and Core Audio. It’s reasonable to assume that the OS and API facilities Apple chooses not to bring forward to 64-bit are heading for deprecation.

Dashcode widgetry

While it’s Mac tradition to make utilities and helper applications look like they’re part of the core OS and desktop, Apple has created a far better and less intrusive place for such things to live. Dashboard, which debuted in Tiger, gives users a customizable palette of JavaScript graphical widgets and a dedicated secondary desktop on which to place them. A Dashboard widget is a simple, single-purpose JavaScript app that uses an HTML canvas for its display window. Widgets can tap native Mac platform APIs and executables, so there are very few trade-offs in moving from the desktop to Dashboard.

As suited to its role as Dashboard is, it is at a disadvantage relative to native Mac apps with regard to development. The automated, structured, and self-contained Xcode IDE doesn’t extend to Dashboard widgets, so developers have been left to their own devices until now. Leopard ships with Dashcode, an IDE with an editor, an interface designer, a media browser, and a JavaScript debugger created expressly for the development of Dashboard widgets. Xcode 3.0 could have been extended to incorporate Dashcode, but Apple wants Dashboard widgets to be accessible to casual developers and to nonprogrammers with HTML design experience.

Apple has endowed the Ruby scripting language, and by association the Ruby on Rails Web application framework, with two-way connectivity to Objective-C native code and the Cocoa and other object-oriented frameworks available to Objective-C applications. While other languages lay claim to being easy to learn, Ruby rivals BASIC’s ease with regard to the swift movement from zero knowledge to productive code. Ruby has a large community behind it that has generated a massive repository of RubyGems classes covering practically every imaginable purpose.

Ruby on Rails is a server foundation that makes the creation of basic dynamic Web sites and applications almost effortless. It scales to handle more complex server-side requirements while retaining its simplicity and the readability of code. Apple bundles Mongrel, an extremely lightweight HTTP framework for Ruby. Mongrel does not rely on Apache for communication with HTTP clients, so Mongrel integrates easily with any HTTP server. Mongrel has the ability to operate as a stand-alone Web server, which takes a Ruby on Rails application straight to the Web.

Action items Over the four parts of this series, I’ve done my best to highlight the new and updated technologies that Leopard places in developers’ hands. There is no question that Mac developers should count on using Leopard for application development; the Xcode 3.0 toolset and its satellite components such as Xray elevate developer skills, productivity, and code quality.

Commercial software, custom solutions, and applications of scale that will run exclusively on Mac Pro workstations or Xserve servers are candidates for direct targeting of Leopard. For workstations, Leopard’s 64-bit presentation layers and optimized frameworks will have a huge impact on performance and capacity. OS X Leopard Server, to which I’ll devote separate coverage, marks a fresh start for Apple’s enterprise programs. Unix workstation and server solutions routinely cite the latest OS release as a system requirement. Keeping up with users’ expectations requires plying all of the advantages that the latest revision of the platform brings to the table. For workstation and server apps, “most recent release” is expected in RISC Unix, and it can safely be established as a standard on the Mac platform starting with Leopard.

The rule of thumb for all other Mac applications will be to hold off on building in dependencies on features unique to Leopard until it has reached critical mass. However, it is important that Mac developers avoid inventing features in their own style that overlap with facilities inherent in Leopard. This trap will result in applications that not only seem dated in Leopard, but will be viewed by users as incompatible with the platform. Even if you’re not targeting Leopard directly, it’s important that your designs and road maps take Leopard into account.

And by all means, whether you start now or upgrade in October, plan to do your development in Leopard. Once you use Leopard, you will get hooked on the strengths that Leopard headliners such as Core Animation and Time Machine bring to your applications and their users. Leopard will spawn ideas for creative solutions that wouldn’t have occurred to you for any prior release of OS X.