Unix: Under the spell of magic numbers

Magic numbers play a very important role on Unix systems. They help the OS recognize details about files that aren’t obvious even to Unix wizards.

What are magic numbers? Magic numbers are values, not generally visible unless you dump file contents using a command like od that displays file contents in hex, that serve as signatures for file type identification. Some are fairly obvious once you see them — like if you spot “x89PNG” in a file dump. Others won’t offer much of a clue.

In general, they are special values set up at or near the beginning or nearby in files that allow command like file to distinguish files by their type so even if you call a jpg file my.image, the OS can still figure out what it is.

How are they used? The tool for file identification is a command named “file”. Use the file command to examine your oddly named my.image file and might just tell you something like this:

$ file my.image my.image: JPEG image data, JFIF standard 1.01

The “JFIF” tag in this description stands for JPEG File Interchange Format. This output tells you that this complies. This tells you that the magic number that is associated with JPEG files is both stored in this file and stored in the right location to identify the file. The same value at some other location would have no effect and would likely be coincidental.

Use file to examine a file that isn’t a JPEG file, but has been assigned a .jpg file extension and the file command comes through and identifies this file as an executable.

$ file bash.jpg binfile.jpg: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/ Linux 2.6.9, dynamically linked (uses shared libs), stripped

If we follow up this inquiry by examing the beginning of the file with the od command, we can see what appears to be an identifier for ELF files strikingly present in the first four bytes.

$ od -bc bash.jpg | head -4 0000000 177 105 114 106 001 001 001 000 000 000 000 000 000 000 000 000 177 E L F 001 001 001