13

I have a quick issue with python's os.path.getmtime() function. I have observed some weird behavior. I am working on a web app that checks periodically to see if a certain file has been modified and decides whether or not to refresh based on that.

In my local python command line, when I change the file and call os.path.getmtime(file_name) the return value from mtime has changed to reflect the change in the file.

However, when I call os.path.getmtime() in my web app the return value before and after the change is the same. I did some research online and found some stuff to suggest that the os module needs to be reloaded for the change to the file to be registered. So, in my web app I reloaded the os module, but mtime still does not reflect changes to the file. Has anyone else encountered this problem before or know a solution? I have included a code snippet below from the webapp:

import os

def function_name():
    reload(os)
    file_path = '/dir/lib/some_file.js'

    try:
        mtime = os.path.getmtime(file_path)
    except os.error:
        pass

    return mtime
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Mars J
  • 902
  • 3
  • 15
  • 23
  • 1
    No, reloading the `os` module has **nothing** to do with this. – Martijn Pieters Sep 27 '13 at 20:54
  • Aah, okay. Yeah I read in one of the python docs that `os.environ` is set only when the os module is loaded and I thought that might have something to do with that. – Mars J Sep 27 '13 at 20:56
  • 1
    `os.path.getmtime()` doesn't cache anything. It simply returns `os.stat(filename).st_mtime`. `os.stat()` doesn't cache anything, it simply calls into the C library, which asks the OS for that info. – Martijn Pieters Sep 27 '13 at 20:59
  • What operating system are you using? I recall that some OSes delay updating `mtime` until after the file is closed. – Robᵩ Sep 27 '13 at 20:59
  • I am developing on a Mac. – Mars J Sep 27 '13 at 21:02
  • But your development machine is working correctly isn't it? I mean, what operating system has the problem? – Robᵩ Sep 27 '13 at 21:03
  • Yeah the python command line on my machine is working fine. But for some reason, when I am testing locally (on localhost) on the same machine, mtime doesn't register the change. – Mars J Sep 27 '13 at 21:32
  • On both environents, is the examined file precisely the same file? If not, can you write a 5-line program that calls `.getmtime()` on the file that fails from the web app? – Robᵩ Sep 27 '13 at 21:35
  • Yeah the examined file is precisely the same. And still fails from the web app. I am now thinking it might have to do with git somehow. Could that maybe affect it? – Mars J Sep 27 '13 at 22:29

3 Answers3

3

I encountered this today and found this question, so I thought I'd document it here. My case was a unit test, so it might be slightly different since it concerns smaller time scales than a manual test.

The modification time is limited by your file system. If you check the modification time, then write a small amount of data, then check the modification time again, the two timestamps might be exactly equal. They will be equal if the time between the first time stamp check and the end of the write is less than the time resolution.

Some statistics on the time resolution of various common file systems:

  • FAT32: 2s
  • ext3: 1s
  • exFAT: 10ms
  • NTFS: 100ns
  • ext4: 1ns

You can expect embedded systems to use FAT, and have a time resolution of 2 seconds. Older Windows systems will be in the range of 2 seconds. Newer Windows systems will have 100ns or 10ms. Older UNIX-systems will often have 1s. Newer UNIX-systems will have a resolution of 1ns.

If <time for time stamp check> + <time for file write> is less than the time resolution, the file could appear as if not modified.

I see these possible solutions:

  • Include a more accurate modification time in the header of the file you're writing. The file writer could even check if the file already exists before writing and increment the nanosecond modification time by at least 1 to guarantee that it updates (at the cost of time stamp accuracy).
  • Store elsewhere how often each file was edited. Use this number to see whether it was edited since you last checked. Note that it is probably not possible to atomically write the file and update the modification count together.
  • Perhaps some trickery can be devised with sleeps, such that the time between the first time stamp check and the file write is always at least the minimum time resolution. This would depend heavily on your type of set-up, and it will block the thread.
Ghostkeeper
  • 2,830
  • 1
  • 15
  • 27
1

I don't have enough reputation to add this as a comment...

It is not clear how you are testing, does one single page of your web app

  • print mtime
  • update the file
  • print mtime

Or

  • simply print mtime

If your web-app test process is

  • request test mtime page
  • manually update the file
  • request test mtime page
  • note that mtime is same on both page views

my first guess is web client, proxy, or server caching.

Sean
  • 200
  • 1
  • 9
0

Perhaps you could try getting general stats on the file besides mtime such as size.

Is the expected size/mtime of the file before and after the change on the server (i.e. when viewing ls -l in a terminal window) the same or different.

If the stats when using such command line tools are the same then it may be the file is not being edited where you think.

If the size/mtime is different perhaps use 

os.stat(filename)

And see if it gives any of the correct values.

Paul
  • 7,155
  • 8
  • 41
  • 40