18

I have the following function, which has been working great for months. I have not updated my version of Python (unless it happens behind the scenes?). 

def Blast(type, protein_sequence, start, end, genomic_sequence):
    result = []
    M = re.search('M', protein_sequence)
    if M:
        query = protein_sequence[M.start():]
        temp = open("temp.ORF", "w")
        print >>temp, '>blasting'
        print >>temp, query
        temp.close()
        cline = blastp(query="'temp.ORF'", db="DB.blast.txt",
                       evalue=0.01, outfmt=5, out=type + ".BLAST")
        os.system(str(cline))
        blast_out = open(type + ".BLAST")
        string = str(blast_out.read())
        DEF = re.search("<Hit_def>((E|L)\d)</Hit_def>", string)

I receive the error that blast_out=open(type+".BLAST") cannot find the specified file. This file gets created as part of the output of the program called by the os.system call. This usually takes ~30s or so to complete. However, When I try to run the program, it instantly gives the error I mention above.

I thought os.system() was supposed to wait for completion?
Should I force the wait somehow? (I do not want to hard code the wait time).

EDIT: I have ran the cline output in the command line version of the BLAST program. Everything appears to be fine.

Stylize
  • 1,058
  • 5
  • 16
  • 32
  • Try adding a 60 second wait between `system` and `open`. If the problem is still there, then the external program is failing somehow – Nick ODell Dec 27 '12 at 18:46
  • You could also print out the returnvalue from the os.system call this should indicate if the program fails or not. Which usually is: 0 = OK; >0 = not OK – Daniel Figueroa Dec 27 '12 at 18:47
  • 2
    Also, consider using [`subprocess`](http://docs.python.org/2/library/subprocess.html) instead - it has far better capabilities for handling errors produced by system calls, and should generally be used [in place of `os.system`](http://docs.python.org/2/library/subprocess.html#replacing-os-system). – David Cain Dec 27 '12 at 18:51
  • What OS are you running this on? – Keith Dec 27 '12 at 18:57
  • Does `str(cline)` contain exactly what you expect? The `os.system()` call *does* wait until the executed command has completed. Your external command is no longer running properly for some reason. – Greg Hewgill Dec 27 '12 at 19:09
  • os.system works through a shell, and will normally wait, but will not wait if there is a `&` at the end of the command line. Can you show us the actual `cline` value? – Keith Dec 27 '12 at 19:15
  • actual cline value:query="'temp.ORF'", db="DB.blast.txt", evalue=0.01, outfmt=5, out=JX174437.1 – Stylize Dec 27 '12 at 21:35
  • that doesn't look like what you are passing to os.system. Or is it? where is the actual program? – Keith Dec 28 '12 at 00:56

3 Answers3

10

os.system does wait. But there could be an error in the program called by it, so the file isn't created. You should check the return value of the called program before proceeding. In general, programs are supposed to return 0 when they finish normally, and another value when there is an error:

if os.system(str(cline)):
    raise RuntimeError('program {} failed!'.format(str(cline)))
blast_out=open(type+".BLAST")

Instead of raising an exception, you could also return from the Blast function, or try to handle it in another way.

Update: Wether the called program runs fine from the command line only tells you that there is nothing wrong with the program itself. Does the blast program return useful errors or messages when there is a problem? If so, consider using subprocess.Popen() instead of os.system, and capture the standard output as well:

prog = subprocess.Popen(cline, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = prog.communicate()
# Now you can use `prog.returncode`, and inspect the `out` and `err` 
# strings to check for things that went wrong.
Roland Smith
  • 42,427
  • 3
  • 64
  • 94
9

You could also replace the call to os.system with subprocess.check_call, and that will raise an exception if the command fails:

import subprocess as subp
subp.check_call(str(cline), shell=True)
  • It depends... Basically if the process Python creates to run the shell in returns something other than 0, `check_call` will raise an exception, but given just how many weird UNIXy things there are that could still also be a success. –  Sep 25 '18 at 18:32
  • Right. I was asking what this function defines to be "success." If you know that, you can determine whether it's appropriate for your use case. – jpmc26 Sep 25 '18 at 21:10
  • I was answering what it defines to be "success" with some provisos as to what impact this might have on any given application and what alternatives one might find. I'm sorry if this wasn't clear from what I was saying. –  Sep 27 '18 at 00:49
0

This answer is a bit late. However, I had the same problem and subprocess didn't seem to work. I solved it by writing the command into a bash-file and executing the bash-file via python os.system:

vi forPython.sh (write 'my command' into it)
chmod +x forPython.sh

(in Python script)

os.system("./forPython.sh")

This makes python wait for your process to finish.

Magdalena
  • 11