1

I'm trying to make a program that can get your info for the game Gears of War 4. I know how to parse webpages and such, but I've never taken the time to log into a site through python.

Now this poses a problem as I need to log in to my Microsoft Live profile to get my stats. I've tried what I thought would work, but hit a dead end every time.

This is what I have so far (that works)

import requests
import urllib2

username = 'USERNAME@hotmail.com'
password = 'PASSWORD'


payload = {'loginfmt': username,
           'passwd': password}

url = 'https://gearsofwar.com/en-us/cards'

initial_data = urllib2.urlopen(url).read()

sign_in = initial_data.find('signInUrl":"')

sign_in = initial_data[sign_in:].split('"')[2]

print sign_in

sign_in is the URL of the page I need to sign in to.

All I need is for someone to show me how to pass my username and password to the login page and then load the page that it redirects to. Everytime I try, I get html data saying that Javascript is not enabled.

Surely there is an easy way to do this that I'm overlooking?

Chanda Korat
  • 2,453
  • 2
  • 19
  • 23
Jebby
  • 1,845
  • 1
  • 12
  • 25
  • There's no easy way (just imagine the security issues an easy way would create). – thebjorn Mar 12 '17 at 20:05
  • This is the wrong approach. You should _**never** even have_ your users' passwords. Read about [OpenID Connect](https://en.wikipedia.org/wiki/OpenID_Connect) and similar technologies. They offer a _**much**_ better solution. (Also you won't have to try to inject data into a website manually; that should give you nightmares.) – ChrisGPT was on strike Mar 12 '17 at 20:08
  • Not exactly an "easy way" but a way that is easy to understand rather. – Jebby Mar 12 '17 at 20:08
  • No, this would be fetching MY INFO, I don't have anybody else's login info – Jebby Mar 12 '17 at 20:09

1 Answers1

0

So if I understand this correctly, you want to crawl a website, have it sign in as a user, then crawl through it as the user, i assume to record certain information and do data stuff to it.

Theres a couple of python apps for crawling, websites and go further into them via scripting, some get pretty powerful and can be used for this such as development testing.

http://www.guru99.com/selenium-python.html

This is a guide to do it in Selenium, but shows a difference between Java and Python. I recommend Selenium for this sort of thing, and it should be easy to understand if you put some time into it. Many security concerns will come up, and could make it impossible. But you could try Selenium, is your a python person.

Heres a sight that has a pretty decent guide to selenium that will make you a bot to do google searchs:

http://www.marinamele.com/selenium-tutorial-web-scraping-with-selenium-and-python

Here's a thread about how to handle authentication

How to handle authentication popup with Selenium WebDriver using Java

I know this isn't an answer, but if you familiarized selenium, or the many others, you can get a lot of stuff done programmatically, than trying to figure it out through sending manual packets through requests and handling authentication.

Community
  • 1
  • 1
Thomas Stefan Wire
  • 41
  • 1
  • 7
  • I appreciate the response Thomas, but it appears to me that you actually need to link to the browser's binary file, and it actually shows the browser? I prefer to do everything "Behind the scenes" or manually if you will. I rather not have extra things pop up (like the browser) if not needed – Jebby Mar 13 '17 at 19:10