Tenerife Skunkworks

Trading and technology

Writing a Mac OSX USB Device Driver That Implements SCSI Pass-through

I’ve been on a coding tear since the beginning of this year, when I decided to dump Erlang and focus on all things low-level. I’ve been much happier since, although not much richer. Do you need a Mac OSX device driver written? Talk to me!

In this post I will explain how I wrote a Mac OSX USB device driver for the IntellaSys 24-core CPU on a thumbstick, also known as FORTHdrive. I will skip the parts that are reasonably clear from Apple documentation and focus on the bits I had trouble with. I will also leave two-machine driver and kernel debugging over FireWire for another post.

It will be helpful for you to first read about IOKit fundamentals, as well as Mass storage device driver programming and the SCSI device architecture model device interface. SCSI in particular is how I started down this slippery slope.

Introduction

The USB flash drive format is popular with hardware vendors. It’s possible to buy security tokens on a thumb stick and even 24-core Forth processors. The stick will most likely have a small disk partition that will house the vendor development kit or tools. It will look like a regular flash drive to the operating system (OS) and the OS will use SCSI over USB to access the data.

The manufacturer will implement vendor-specific SCSI commands to give you access the core functionality of the device such as the encryption API of a security token or storing and fetching data from a custom CPU. The OS will let you send custom SCSI commands to a SCSI device, this is called SCSI pass-through. You can use SGIO for SCSI pass-through on Linux. This boils down to a series of ioctl calls from your application and all is well… except on Mac OSX.

In its infinite wisdom, Apple decided to disable SCSI pass-through lest you send a format command to an attached device or do something equally evil. Apple [really really wants you to go through official and established channels] to talk to devices under Mac OSX, particularly SCSI devices. Apple did not and cannot establish channels for every custom device out there, which means that the hard work to implement SCSI pass-through on Mac OSX falls squarely on your shoulders.

Writing a Mac OSX device driver is not particularly hard. It took me all of about a week to get my driver ready and working. There’s definitely a dearth of information on writing Mac OSX device drivers and existing examples are too simple to be of much use.

I hunted far and wide (and way back in time!) through various Apple driver development lists to collect the information I needed and I’m summarizing it for you here, as well as providing full working source code to my driver.

Did I mention that Mac OSX drivers are written in C++? Not C, not Objective-C but C++! The original IOKit used to be called DriverKit and was written in Objective-C. Apple, apparently, felt C++ would be easier on third-party driver writers. Say what you want but C++ does simplify reuse. You don’t need to re-implement the full driver, you can subclass and change or add tiny bits and pieces.

Fundamentals

Your application lives in user land whereas the driver lives in kernel land. The two cannot talk to one another, except through a Mach port. Normally, your application would first locate the driver in the I/O registry.The SimpleUserClient and VendorSpecificType00 examples that Apple provides for developers show you how this is done.

Once you get a handle to your driver (service), you can open a connection to it like this

1
2
io_connect_t connect;
kern_return_t kernResult = IOServiceOpen(service, mach_task_self(), 0, &connect);

This gets you a handle that you can use to access your driver in kernel land.

User client

Once you get through the Mach port, you land in something called the user client. The user client mechanism is designed to allow calls from a user process to be dispatched to any IOService-based object in the kernel. Your driver would normally be a subclass of IOService but you would not access it directly. You would create a series of “adapter” functions that verify and perhaps massage the data and then pass it to your driver.

You can invoke user client functions that are set up via the external method dispatch table. This is a series of structures that describe each method of your user client, including the function pointer, number of integer arguments that the method takes in, number of integer values it returns and the same for structures. The table will look like this

1
2
3
4
5
6
7
8
9
10
11
const IOExternalMethodDispatch UserClientClassName::Methods[kNumberOfMethods] =
{
  { // kS24ClientOpen
    (IOExternalMethodAction) &UserClientClassName::sOpenUserClient,
    0,
    0,
    0,
    0
  },
  ...
}

The SimpleUserClient example shows you how to set up and use various external method configurations.

Your user land method invocation will end up in externalMethod below. This is where you will look up your method using the selector to index your method table.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
IOReturn UserClientClassName::externalMethod(uint32_t selector, IOExternalMethodArguments* arguments,
    IOExternalMethodDispatch* dispatch, OSObject* target, void* reference)

{
    IOLog("%s[%p]::%s(%d, %p, %p, %p, %p)\n", getName(), this, __FUNCTION__,
        selector, arguments, dispatch, target, reference);

    if (selector < (uint32_t) kNumberOfMethods)
    {
        dispatch = (IOExternalMethodDispatch *) &Methods[selector];

        if (!target)
     target = this;
  }

A lot of methods in the user client are boilerplate but you do not want to miss initWithTask! This is the method where you should take owningTask and save it. This is the Mach task of your user land application and you will need it to map memory buffers from user space to kernel space. owningTask here will correspond to mach_task_self() in the call to IOServiceOpen above.

1
2
3
4
5
6
7
8
9
10
11
12
bool UserClientClassName::initWithTask(task_t owningTask, void* securityToken, UInt32 type)
{
    bool success = super::initWithTask(owningTask, securityToken, type);

  // This IOLog must follow super::initWithTask because getName relies on the superclass initialization.
  IOLog("%s[%p]::%s(%p, %p, %ld)\n", getName(), this, __FUNCTION__, owningTask, securityToken, type);

    fTask = owningTask;
    fProvider = NULL;

    return success;
}

Methods in your dispatch table will be static and you will need a way to map those to methods of your user client class. Fortunately, every method has a target argument just for this purpose.

1
2
3
4
IOReturn UserClientClassName::sInit(UserClientClassName* target, void* reference, IOExternalMethodArguments* arguments)
{
    return target->S24IO(NULL, 0, 0, 0, kIODirectionNone);
}

The code above does not use any of the external arguments but this method does

1
2
3
4
5
6
7
8
9
10
IOReturn UserClientClassName::sRead(UserClientClassName* target, void* reference, IOExternalMethodArguments* arguments)
{
    return target->S24IO(
        arguments->scalarInput[0],
        arguments->scalarInput[1],
        arguments->scalarInput[2],
        0,
        kIODirectionIn
        );
}

Simply pull your values from external method arguments and pass them to a method in your user client class, e.g. S24IO. fprovider is our driver handle that we set up in the start method, invoked as a result of us calling IOService open in our user land application.

Talking to the user client

To talk to the driver’s user client from your application you will invoke methods like IOConnectCallScalarMethod and friends. The SimpleUserClient example shows how this is done.

Passing buffers into the kernel

Apple has guidelines for how to allocate and share memory with user space from an I/O kit driver but what do you do if you need to pass a buffer from user space into the kernel? Simple! The kernel works with I/O memory descriptors and we need to create one for our user space buffer like so

1
2
3
4
5
6
IOMemoryDescriptor *iomd = IOMemoryDescriptor::withAddress(
      (vm_address_t)buffer,
      size,
      direction,
      fTask
    );

See IOMemoryDescriptor documentation for more details.

Note fTask and direction above. You must tell the kernel which task this memory pointer belongs to so that the kernel can properly translate this address into physical memory. You also must tell the kernel whether you are going to be reading from this memory buffer or writing to it. This is what direction is for.

This is by no means conclusive documentation for _IOMemoryDescriptor__. Please read about IOBufferMemoryDescriptor and feel free to poker around further.

We are still in driver adapter and glue code here but we are getting close to the driver itself.

Driver

The salient points here are the InitializeDeviceSupport method and the way to send SCSI commands to the device.

Use InitializeDeviceSupport if you need to send SCSI commands to your device during driver initialization. Do not use the probe method for this since the command gate (don’t ask!) will not be allocated yet and you will panic the kernel.

Here I’m initializing my device by sending it the vendor-specific initialization command in S24Init().

1
2
3
4
5
6
7
8
9
10
11
bool com_wagerlabs_driver_SEAforth24::InitializeDeviceSupport(void)
{
    bool result = false;

    result = super::InitializeDeviceSupport();

    if ( result == true )
        result = (S24Init() == kIOReturnSuccess);

    return result;
}

The S24SyncIO method is the heart and soul of my driver. Your driver will look different but things are easy and downhill from this point on since you have everything you need to send any kind of SCSI command to your device. You just need to go through a few more steps before you are done.

1) You get hold of a SCSI task.

1
2
3
req = GetSCSITask();

require(req != NULL, ErrorExit);

2) You populate the SCSI Command Descriptor Block (CDB) according to your vendor’s instructions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
switch (kind)
{
    case kS24Write:
        direction = kSCSIDataTransfer_FromInitiatorToTarget;
        b1 = 0xFB;
        b2 = 0x00;
        break;
    case kS24WriteLast:
        direction = kSCSIDataTransfer_FromInitiatorToTarget;
        b1 = 0xFB;
        b2 = 0x02;
        break;
    case kS24Read:
        direction = kSCSIDataTransfer_FromTargetToInitiator;
        b1 = 0xFB;
        b2 = 0x01;
        break;
    default: // kS24Init
        direction = kSCSIDataTransfer_NoDataTransfer;
        b1 = 0xFA;
        b2 = 0x00;
}

SetCommandDescriptorBlock(req, 0x20, b1, b2, 0x00, 0x00, 0x00, 0x00, hi, lo, 0x00);

3) You set a timeout for completion of your SCSI request and the data transfer direction.

1
2
SetTimeoutDuration(req, 10000);
SetDataTransferDirection(req, direction);

4) Mac OSX uses virtual memory which means that at the time of your SCSI command your buffer may be paged out to disk and not in physical memory. It’s crucial that you tell Mac OSX to prepare your memory buffer by mapping it back into memory and do any necessary housekeeping for your driver to be able to access your memory.

Other than that, don’t forget to tell the SCSI task to use your buffer and tell it how many bytes you are looking to transfer. It’s not necessary to set the direction of the transfer (from driver to device or vise versa) if this has already been set in the I/O memory descriptor (which is what we did).

1
2
3
4
5
6
if (buffer != NULL)
{
    buffer->prepare();
    SetDataBuffer(req, buffer);
    SetRequestedDataTransferCount(req, buffer->getLength());
}

5) Finally, send the command to the device and tell Mac OSX that your are done using your memory buffer for direct memory access (DMA) by invoking the complete method of the I/O memory descriptor. You will also want to check the status of your SCSI request, the number of bytes transferred and you may also want to use the “SCSI Request Sense command”:http://en.wikipedia.org/wiki/SCSI_Request_Sense_Command if your request was unsuccessful.

1
2
3
4
5
6
serviceResponse = SendCommand(req, 10000);

if (buffer != NULL)
{
    buffer->complete();
}

That’s it folks! Let me know if I have omitted something crucial and I’ll try to expand this post as time allows.

Comments