How can I use urllib.request.urlretrieve with python 2.7

Question

I am trying to get images dowloaded from image-net.org so that I can create a haar cascade classifier. I am following this tutorial https://www.youtube.com/watch?v=z_6fPS5tDNU&list=PLQVvvaa0QuDdttJXlLtAJxJetJcqmqlQq&index=18 but I am using python 2.7 instead of python 3. So in the tutorial he has the line:

urllib.request.urlretrieve(img, pathToImage)

Instead of import urllib.request I did this import urllib2 So I tried this but it isn't vaild

urllib2.urlretrieve(i, "Negatives/"+str(num)+".jpg")

Thank you in Advance!

"but it wasn't valid" isn't particularly descriptive, what exactly went wrong? — Tadhg McDonald-Jensen, Aug 19 '16 at 00:23
It prints this error 'module' object has no attribute 'urlretrieve' — Loanb222, Aug 19 '16 at 00:24
[It is in the `urllib` module in python2](https://docs.python.org/2/library/urllib.html#urllib.urlretrieve) — Tadhg McDonald-Jensen, Aug 19 '16 at 00:25
That said, why are you following a tutorial for py3 but using py2, why not just get python 3? — Tadhg McDonald-Jensen, Aug 19 '16 at 00:34
I have python 2 installed on my raspberrypi becuase it was hard to find a tutorial for installing opencv with python 3, So I test out the code on my mac and then I can just transfer the file. — Loanb222, Aug 19 '16 at 00:36

score 20 · Accepted Answer · answered Aug 19 '16 at 00:25

20

You just need to import urllib without '2'

import urllib
urllib.urlretrieve(i, "Negatives/"+str(num)+".jpg")

answered Aug 19 '16 at 00:25

g1zmo

487
3
7

score 1 · Answer 2 · answered Jul 26 '18 at 20:38

I found that on different build systems, I'd have different version of Python available, totally out of my control.

So I adjusted my script to get urlretrieve this way:

import sys
print("Python: " + sys.version)
sys.stdout.flush()

import os, re, difflib

# because somewhere along the line they  may have deprecated `urlretrieve`
#   mentioned in docs for python [3.5.5, 3.6.6, 3.7.0, 3.8.0a0] that:
#       `[urlretrieve] might become deprecated at some point in the future.`
def UrlRetrieve(url, fname=None):
  if sys.version_info[0] <= 2:
    import urllib
    return urllib.urlretrieve(url, fname)
  elif sys.version_info[0] <= 3:
    import urllib.request
    return urllib.request.urlretrieve(url, fname)
  else:
    import shutil
    import tempfile
    import urllib.request
    with urllib.request.urlopen(url) as response:
      if fname is None:
        with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
          shutil.copyfileobj(response, tmp_file)
          return (tmp_file.name, response.info())
      else:
        with io.open(fname) as the_file:
          shutil.copyfileobj(response, the_file)
          return (fname, response.info())

Then, use it this way:

url = "http://...whatever.../bootstrap.zip"
pair = UrlRetrieve(url)

And then, because what I was importing was definitely python 2, I needed to do this in the 3 world:

if sys.version_info[0] >= 3:
  import zipfile
  zip_ref = zipfile.ZipFile(pair[0], 'r')
  zip_ref.extractall(pair[0] + ".d")
  zip_ref.close()
  import os
  os.system("2to3 -w " + pair[0] + ".d")
  sys.path.append(pair[0] + ".d")
else:
  sys.path.append(pair[0])
from bootstrap_common import *

I now keep these snippets handle for future scripts where I need to use urlretrieve on both Python 2 and 3.

Johan Park · Answer 3 · 2018-07-24T23:57:47.223

0

on this Python urllib urlretrieve behind proxy

import urllib2

download = opener.urlretrieve(URL, "Negatives/"+str(num)+".jpg")

the code can be transformed to

import urllib2

with open("Negatives/"+str(num)+".jpg",'wb') as f:
    f.write(urllib2.urlopen(URL).read())

edited Jul 24 '18 at 23:57

answered Aug 19 '16 at 00:46

Johan Park

11
4

Won't the `with` handle the `f.close()` for you? – Jesse Chisholm Jul 24 '18 at 20:28

How can I use urllib.request.urlretrieve with python 2.7

3 Answers3