Files read and written get cached in the Unified Buffer Cache (UBC) on Mac OSX.

The UBC was hindering me because I was processing a huge file in chunks but throwing out each chunk, never to be reused again, after writing out the processed chunk. I would see gigabytes of memory get eaten away by the UBC until the system started swapping and became unresponsive.

UBC cannot be limited to a given maximum amount of memory.

UBC cannot be inspected programmatically.

UBC can be cleared by running ‘purge’ which allocates a lot of memory to force the cache to clear. The following bit of code can be used turn caching off for a particular file:

1

fcntl(fd, F_GLOBAL_NOCACHE, 1);

This can be done in any process and the file can be closed after. The setting persists through out the lifetime of the file. If the file is removed and re-created then the setting is lost.

How can you tell if the setting took hold and the file is indeed NOT being cached?

1
2
3
4
5
6
7
8

dd if=/db1/cdr.csv bs=1m count=1024 of=/dev/null
1073741824 bytes transferred in 12.030688 secs (89250242 bytes/sec)

dd if=/db1/cdr.csv bs=1m count=1024 of=/dev/null
1073741824 bytes transferred in 11.867947 secs (90474101 bytes/sec)

dd if=/db1/cdr.csv bs=1m count=1024 of=/dev/null
1073741824 bytes transferred in 12.037562 secs (89199278 bytes/sec)

Tried reading the file three times. Speed is about the same.

What about a regular file that’s cached by default?

1
2
3
4
5

dd if=/db1/cdr1.csv bs=1m count=1024 of=/dev/null
1073741824 bytes transferred in 11.505857 secs (93321325 bytes/sec)

dd if=/db1/cdr1.csv bs=1m count=1024 of=/dev/null
1073741824 bytes transferred in 0.500468 secs (2145475416 bytes/sec)

Notice that reading from the cache is much faster the second time around.

Kudos to Dominic Giampaolo from Apple for explaining all this to me!