1

We have a python script + associated modules on a network drive. We are running this script simultaneously from several computers at once.

Sometimes one of them randomly fails with an error like this: 

Traceback (most recent call last):
  File "\\chifs02.int.tt.local\group\Development\Server Products\Automation\repos\mksutils\fetch.py", line 16, in <module>
    from scripts import write_set_environment
NameError: Can't find file for module scripts
(filename \\chifs02.int.tt.local\group\Development\Server Products\Automation\repos\mksutils\scripts.pyc)

One oddity that I noticed was that the .py file was last modified months ago, but the .pyc is only hours old, despite being run at least daily since then.

I have never seen this NameError: Can't find file for module scripts error before, and Google isn't being much help.

All computers involved are running Windows.

Matthew Scouten
  • 15,303
  • 9
  • 33
  • 50
  • 1
    This problem is solved in Python 3.x, which puts `.pyc` files in a separate `__pycache__` directory and stamps them with the particular Python flavour for which they are compiled. – Katriel May 13 '11 at 18:25
  • @katrielalex Nice to know, but for several technical reasons, we will be not python 2.6 or 2.7 for the foreseeable future. – Matthew Scouten May 13 '11 at 18:31

2 Answers2

4

Don't do that, where "that" is have multiple computers trying to use the same .pyc(s).

The likely reason the pyc gets modified is that you likely have slightly different versions of Python on different computers. One writes its version of the .pyc, and another comes along and sees that it's not what it expected, and writes its own version.

It would surprise me not at all to learn that strange bugs are possible in the situation where different computers are simultaneously trying to deal with the same .pyc files. This is an invitation for race conditions, as there is no locking mechanism in use.

Either have each system copy the script to a local place before running it, or disable .pyc generation (-B command line flag or set the PYTHONDONTWRITEBYTECODE environment variable).

Nicholas Knight
  • 15,774
  • 5
  • 45
  • 57
2

It's an odd error (I've never seen one in the wild).

My hypothesis is that it has to do with one node regenerating the pyc file while another node tries to look at it. Trying deleting the pyc files and specifying -B on the Python command line:

-B     : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
NPE
  • 486,780
  • 108
  • 951
  • 1,012