Tenerife Skunkworks

Low level Mac and Embedded Linux

Grand Central Dispatch

Now that Grand Central Dispatch is part of iPhone OS 4.0, I would like to showcase its unique benefits for inter-thread communication.

Passing data from one thread to another is normally a royal pain on the Mac, something involving the use of Mach ports, something I’d rather not talk about. Ever! Assuming that your threads use Core Foundation or Cocoa, your threads have a run loop and you can shuffle bits of data back and forth with no trouble whatsoever. You can even do it in C++.

We will look at running a code block in another thread and waiting for it to finish (synchronous execution), as well as running a block in another thread and having it run a callback that we supply (asynchronous execution).

Tracking IO Patterns in Memory-mapped Dynamic Libraries

The Mac OSX dynamic linker uses mmap to load dynamic libraries into memory. The memory range occupied by the libraries is then backed by a set of virtual memory pages, chunks of 4096 bytes each.

Using mmap is efficient because pages are lazily loaded from disk as needed and the virtual memory pager is free to evict them behind the scenes when memory is needed for something else.

Each page is filled with code for functions that live in the dynamic library and pages are fetched from disk when a call is made to a function in the dynamic library.

Firefox Startup: Where Does Time Go?

The dyld shared cache lives in /var/db/dyld/. The two files of interest are dyld_shared_cache_i386.map (for x86-32) and shared_region_roots/Applications.paths. Both are regular text files. The former shows the contents of the shared cache for the i386 architecture and the latter is what update_dyld_shared_cache inspects.

There’s no prebinding on newer versions of Mac OSX and the dyld shared cache is automatically updated as needed. Tracing Safari disk activity during startup reveals that basically all its dynamic libraries are pulled from the dyld shared cache.

Creating Mac Binaries on Any Platform

I’m in love with Forth but there are no commercial Forth environments for Mac OSX. GForth is a free, fast and portable implementation of ANS Forth but it requires GCC and does not allow for binary distribution of code that uses foreign functions.

There are two excellent commercial implementations of ANS Forth and both run on Linux. I asked one of the companies if I could port their Forth to the Mac and promptly ended up with a tarball on my lap. There were no C or assembler files, it was all Forth source code.

The proper bootstrapping approach turned out to generate a Mac kernel on Linux, copy it over to the Mac and use it to compile the rest of the Forth environment. It’s called cross-compiling!

Hacking the Mac OSX Unified Buffer Cache

Files read and written get cached in the Unified Buffer Cache (UBC) on Mac OSX.

The UBC was hindering me because I was processing a huge file in chunks but throwing out each chunk, never to be reused again, after writing out the processed chunk. I would see gigabytes of memory get eaten away by the UBC until the system started swapping and became unresponsive.

Writing a Mac OSX USB Device Driver That Implements SCSI Pass-through

I’ve been on a coding tear since the beginning of this year, when I decided to dump Erlang and focus on all things low-level. I’ve been much happier since, although not much richer. Do you need a Mac OSX device driver written? Talk to me!

In this post I will explain how I wrote a Mac OSX USB device driver for the IntellaSys 24-core CPU on a thumbstick, also known as FORTHdrive. I will skip the parts that are reasonably clear from Apple documentation and focus on the bits I had trouble with. I will also leave two-machine driver and kernel debugging over FireWire for another post.

Parsing Text and Binary Files With Erlang

Erlang originated in the telecommunications industry where one of the major tasks is conversion of text and binary data from one format to another. This is a task that Erlang excels at!

Parsing text and binary data is something that you will be doing very often in the course of writing your super-scalable internet servers so lets take a look at some efficient approaches to parsing text and binary data.

Transparency and Masking in Lispworks CAPI

CAPI is a good cross-platform GUI toolkit and has been used to write apps like Prime Trader. Your apps will have a native look on each platform and you won’t have to do anything special to make it happen. Still, bits of platform speific code are still required and one example is alpha-blending.

There’s no support for alpha-blending in CAPI. You can do masking but specifying a transparent color in your images but this is way too basic to render a poker room so I have to resort to platform-specific code to make it happen.

HiPE for Mac X86

It started innocently enough with my wondering why High-performance Erlang (HiPE) was not available on my shiny new MacBook Pro. This turned into a long but awesome email exchange and culminated in victory, and my port of HiPE to Mac Intel, less than two weeks later.

I learned a fair number of things about the Mac OSX kernel, the FPU, floating-point exceptions, Intel 32-bit architecture and SSE2 along the way. I also had an awesome time with assembler code.

I could not have done it without the mach_override library and numerous tips from Mikael Pettersson as well as other folks on the Erlang Questions mailing list. I will be shipping my patches to the HiPE team this week.