Memory Mapping Files

The Linux approach to I/O places the emphasis on files, but there are times when reading and writing a file to an external device like a disk drive is too slow. To solve this problem Linux has a memory mapping function which will read any portion of a file into user memory so that you can work with it directly using pointers. In principle, this is a very fast way to access any file, including the mem pseudo file.

This may seem be a very convoluted route to get at memory. First implement memory access as a file you can read and then map that file into memory so that you can read it as if it was memory - which it is. However, if you follow the story, it is logical. What is more it solves a slightly different problem very elegantly. It allows the fixed physical addresses of the peripherals to access the user space virtual addresses. In other words, when you memory map the mem file into user memory, it can be located anywhere and the address of the start of the register area will be within your program’s allocated address space. This means that all of the addresses will change. Of course, as long as we work with offsets from the start of memory this is no problem - we update the starting value and use the same offsets. 

Let’s see how this works in practice, noting that the key function is mmap:

void *mmap(void *addr,size_t length,int prot,
                     int flags,int fd,
                                 off_t offset);

This function memory maps the file corresponding to the file descriptor fd into memory and returns its start address. The offset and length parameters control the portion of the file mapped, i.e. the mapped portions starts at the byte given by offset and continues for length bytes.

There is a small complication in that, for reasons of efficiency, the file is always mapped in units of the page size of the machine. So if you ask for a 1-Kbyte file to be loaded into memory then on ARM processors, such as the Raspberry Pi, which have a 4-Kbyte page size, 4 Kbytes of memory will be allocated. The file will occupy the first 1 Kbytes and the rest will be zeroed. 

You can also specify the address to which you would like the file loaded in your program’s address space, but the system doesn't have to honor this request, it just uses it as a hint. Some programmers reserve an area of memory using malloc and then ask the system to load the file into it. However, as this might not happen it seems simpler to let the system allocate the memory and pass NULL as the starting address. The parameters prot and flags specify various ways the file can be memory mapped and there are a lot of options - see the man page for details. 

Notice that this is a completely general mechanism and you can use it to map any file into memory. For example, if you have a graphics file image.gif, you could load it into memory to make working with it faster. Many databases use this technique to speed up their processing. 

Now all we have to do is map /dev/mem into memory. First we need to open the /dev/mem device as usual:

uint32_t memfd = open("/dev/mem", O_RDWR | O_SYNC);

As long as this works we can map the file into memory.

Memory mapping /dev/mem means that you can work with fixed physical addresses anywhere in your user space. As well as being useful, it is also generally fast because of the way the hardware is used. It is a much better way to work with I/O devices than sysfs simply because it is faster, but it only works under Linux.

For example, in the case of the Raspberry Pi we want to map the file starting at either 0x20200000 for the Pi 1 or starting at 0x3F200000 for the Pi 2 or later. If we only want to work with the GPIO registers then we only need offsets of 0000 to 00B0 i.e. 176, bytes but as we get a complete 4-Kbyte page we might as well map 4 KBytes worth of address space:

uint32_t * map = (uint32_t *)mmap(NULL,4*1024,
      (PROT_READ | PROT_WRITE),MAP_SHARED,memfd,0x3f200000);

If you try this, remember to change the offset to be correct for the device you are using.

Notice that we haven't set an address for the file to be loaded into - the system will take care of it and return the address in map. We have also asked for read/write permission and allowed other processes to share the map. This makes map a very important variable because now it gives the location of the start of the GPIO register area in user space.

Now we can read and write a 3KByte block of addresses starting at the first GPIO register, i.e. FSEL0. 

For example to read FSEL0 we would use:

printf("fsel0 %X \n\r",*map);

To access the other registers we need to add their offset, but there is one subtle detail. The pointer to the start of the memory has been cast to a uint32_t because we want to read and write 32-bit registers. However, by the rules of pointer arithmetic, when you add one to a pointer you actually add the size of the data type the pointer is pointing to.

In this case when you add one to map you increment the location it is pointing at by four, i.e. the size of a 32-bit unsigned integer. The rule is that with this cast we are using word addresses which are byte addresses divided by 4. Thus, when we add the offsets, we need to add the offset divided by 4. For example, to read the memory location at offset 0xFF you have to read from *(map+0x3F) because 0xFF/4=0x3F. You can read smaller units than 32-bit words from memory by casting the pointer to other types.