Writing to HW registers via /dev/mem needs extra write

Question

Writing to HW registers via /dev/mem needs extra write

I want to write to some HW registers using /dev/mem on Linux. The target board is ZYBO (Zynq, ARM Cortex-A9), and the HW is AXI4 Lite Slave with 4 registers which is automatically generated by Xilinx Vivado.

Here is the C code to write to the HW registers.

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>

#define MAP_BASE        (0x43c30000)
#define MAP_RANGE       (0x10000)

int main(int argc, char *argv[])
{
        volatile unsigned int *p;
        void *iomap_ptr;
        int fd, i, v;

        fd = open("/dev/mem", O_RDWR | O_SYNC);
        iomap_ptr = mmap(0, MAP_RANGE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, MAP_BASE);

        p = (volatile unsigned int *)iomap_ptr;
        for (i = 0; i < 4; i++)
                p[i] = i;
        for (i = 0; i < 4; i++)
                printf("%04X: %08X\n", i, p[i]);

        munmap(iomap_ptr, MAP_RANGE);
        close(fd);
        return 0;
}

When I run the C code first time, the result is:

0000: 00000000
0001: 00000001
0002: 00000002
0003: 00000000

It seems that the final write is not applied.

Then, when I run the C code once again, the result is:

0000: 00000000
0001: 00000001
0002: 00000002
0003: 00000003

After investigation, I realized that I need an extra write to the HW in order to apply the last write.

How can I write to the HW registers without an extra write?

Answer 1

To preface this, I somehow missed important information in the question, so the content of this answer is far less portable than it needs to be, but I believe it may still solve the problem. There are two related questions on SO specific to the Zynq and mmap:

Flush cache to DRAM

How to get writes via an mmap mapped memory pointer to flush immediately?

The Xilinx cache functions listed here appear to ultimately use the same method as the top answer in the first linked question.

---

This is likely a caching issue. In this specific case, you can use the Xilinx library to flush the address range that your registers are in. The header file xil_cache.h has two functions that can be used for this:

void Xil_DCacheFlush(void); 
void Xil_DCacheFlushRange(unsigned int adr, unsigned len); 

Depending on version, you may have a third option (with slightly different prototypes):

void Xil_DCacheFlush(void);
void Xil_DCacheFlushRange(INTPTR adr, u32 len);
void Xil_DCacheFlushLine(INTPTR adr);

The first function, Xil_DCacheFlush will flush the entire cache. This isn't a particularly desirable behavior, as there will potentially be a lot of cached data that isn't relevant to the issue getting flushed and invalidated, so you may see a performance impact.

The third function, Xil_DCacheFlushLine if available, will likely work in this case, since you only have four registers. It will force the writes (if the line is dirty) and then invalidate the cache so the next read will be a miss and have to fetch from your register. This isn't the best solution, though, as it relies on a few assumptions about your peripheral.

The second function, Xil_DCacheFlushRange is probably the function you want. Using this function, you can specify the range of addresses of your peripheral. In this case, it is small, but this solution is scalable for complicated peripherals with very large register sets.

After you perform your register writes to the peripheral, you should be able to call whichever of the above functions you prefer to flush all or part of the data cache.

There may also be a way to disable caching of this address range in your design, though I am having difficulty locating a good reference for this, and you may not have the granularity in region location/size that you desire.