0

I am using python2 for web scraping, I have written a spider that uses headless Firefox (no GUI) to go on a website, log in with my account and furthermore interact with the website by pressing buttons, filling forms, calendars, etc. It works as expected on my personal computer, however, once I deploy it to Scrapinghub I get the error saying that geckodriver needs to be on path.. That directory already is on PATH on my computer, just not on Scrapinghub.

I tried copying geckodriver itself into a folder within the project, adding its subdirectory to the executable_path parameter for webdriver as shown in this small guide, and finally deploying to Scrapinghub again, but I still get the same error.

I would like to know how to add geckodriver to "PATH" on Scrapinghub (if possible) and whether there are other ways I can achieve this or not. I have read something about Python eggs but I am not sure that is something that would help me with this.

I use Windows 10 and python 2.7.

Luis Viguria
  • 1
  • 1
  • 2
    looks like this is what you want --> https://stackoverflow.com/questions/47290574/set-variable-on-shub-deploy-project/47305956#47305956 – gold_cy Jan 21 '19 at 18:05
  • Thanks, but that did not help me at all, I found out that the only way to accomplish what I want is by using a Docker image and deploying it to Scrapinghub. – Luis Viguria Jan 30 '19 at 17:41

0 Answers0