Exclusive locking in Go

Question

How can a file be opened such that only one process can write to it at a time in Golang?

I am new to Golang and was trying to learn it by writing something. Was attempting to create a secret storage tool which stores, say, api keys to a file; during which I came across an issue - if two instance of same program runs together (hypothetically) a race condition can occur and damage the file.

So, I was checking for a way to lock the file so that only one process can write to it at a time. Reading simultaneously is ok.

---

Found this answer : How to get an exclusive lock on a file in go on stackoverflow. but in the comments of the same, it's mentioned

your package uses flock() for locking on Unix so doesn't provide OS level mandatory locking

---

Tried os.ModeExclusive.

f, err := os.OpenFile(".secrets", os.O_RDWR|os.O_CREATE, os.ModeExclusive)

But it prevents the file from accessing the next time program runs. Can you please explain what this mode does. Not able to understand it from the docs.

---

Also tried:

f, err := os.OpenFile(".secrets", os.O_RDWR|os.O_CREATE, 0744)

On using the above, anyone can read/write when the file is opened.

---

Answer 1

I'll try to explain the os.ModeExclusive bit of your question:

Tried os.ModeExclusive.

f, err := os.OpenFile(".secrets", os.O_RDWR|os.O_CREATE, os.ModeExclusive)

But it prevents the file from accessing the next time program runs. Can you please explain what this mode does. Not able to understand it from the docs.

OpenFile() with os.ModeExclusive basically creates a file where not even the file owner has read and write permissions on it (at least this is the behavior I observed on unix). that is why it "doesn't work" the next time when it tries to open the file again:

$ cat prog.go
package main

import (
        "fmt"
        "log"
        "os"
)

func main() {
        f, err := os.OpenFile(".secrets", os.O_RDWR|os.O_CREATE, os.ModeExclusive)
        if err != nil {
                log.Fatal(err)
        }
        defer f.Close()

        fmt.Fprintf(f, "this is my secret\n")
}
$ go build prog.go
$ ./prog
$ ls -al
total 2.1M
drwxr-xr-x 2 marco marco 4.0K Aug 28 13:53 ./
drwxr-xr-x 4 marco marco 4.0K Aug 28 13:22 ../
-rwxr-xr-x 1 marco marco 2.1M Aug 28 13:53 prog*
-rw-r--r-- 1 marco marco  230 Aug 28 13:49 prog.go
---------- 1 marco marco   17 Aug 28 13:53 .secrets    <-- HERE IT IS
$ ./prog
2021/08/28 13:54:27 open .secrets: permission denied
$ cat .secrets
cat: .secrets: Permission denied
$ chmod u+rw .secrets
$ cat .secrets
this is my secret
$

os.ModeExclusive can also be used with Chmod() of course on already existing files.

Answer 2

The convention (at least for the standard library) is the following: No function/method is safe for concurrent use unless explicitly stated (or obvious from the context). It is not safe to write concurrently to an os.File via Write() without external synchronization.

source: https://stackoverflow.com/a/30746629/13067552

You can use RMutex from sync package to avoid data race. *sync.RWMutex

package main

import (
    "fmt"
    "os"
    "sync"
)

type MyFile struct {
    l    *sync.RWMutex
    file *os.File
}

func (f *MyFile) Write(b []byte) (int, error) {
    f.l.Lock()
    defer f.l.Unlock()

    return f.file.Write(b)
}

func (f *MyFile) Read(b []byte) (int, error) {
    f.l.RLock()
    defer f.l.RUnlock()

    return f.file.Read(b)
}

func main() {
    f, err := os.Open("tmp")
    if err != nil {
        // handle here
    }
    defer f.Close()

    myfile := &MyFile{
        l:    &sync.RWMutex{},
        file: f,
    }

    myfile.Write([]byte("secret"))

    b1 := make([]byte, 5)
    myfile.Read(b1)

    fmt.Println(string(b1))
}