Python urllib2 login to minecraft.net

Question

I got a problem. I am writing a simple script to login to minecraft.net, and then list all classic servers. But when I run my script, it just redirects me back to minecraft.net/login. Here is what I have so far:

import urllib2
import urllib
import re

url = "https://www.minecraft.net/login"
page = urllib2.urlopen(url)
data = page.read()
page.close()

authToken = re.search('name="authenticityToken"[\s]+value="(.+)"', data).group(1)

data_dict = {
    "username": "USERNAME",
    "password": "PASSWORD",
    "remember": "true",
    #"redirect": "https://www.minecraft.net",
    "authenticityToken": authToken
}
print urllib.urlencode(data_dict)
req = urllib2.Request(url, data=urllib.urlencode(data_dict))
page = urllib2.urlopen(req)
data = page.read()
page.close()

What I am doing wrong?

Well, you're opening that URL (you set it at the top and never re-set)... that's probably why you're opening the URL. — Ben, Dec 08 '13 at 09:06

Jon · Answer 1 · 2013-12-13T15:30:41.770

What about using Selenium instead of urllib? When doing complex stuff like authentication etc. I prefer to use Selenium as it is like web browsing. This solution requires an installation of a Firefox.

Installation

>>> sudo pip install selenium

Code

from selenium import webdriver

# initialize the browser and go to the site.
browser = webdriver.Firefox()
url = 'https://www.minecraft.net/login'
user_name = 'your_user_name'
password = 'your_password'
browser.get(url)

# retrieve all necessary page elements
user_name_field = browser.find_element_by_id('username')
password_field = browser.find_element_by_id('password')    
user_name_field.send_keys(user_name)
password_field.send_keys(password)
sign_in_btn = browser.find_element_by_id('signin')

# log in
sign_in_btn.click()

I don't own a Minecraft account so I cannot test it, but you probably can retrieve all Minecraft servers thanks to methods like:

browser.find_elements_by_partial_link_text
browser.find_elements_by_class_name
browser.find_elements_by_css_selector

Generally, I recommend to test your Selenium code with ipython, where you can see the browser work. (Installation: sudo pip install ipython)

In the case of a lot of Javascript processing - please add the following lines to your code:

browser = webdriver.Firefox()
browser.browser.implicitly_wait(30) # seconds

If you plan to additionally support another browser you can support it with try catch clauses

import sys

browser = None
try:
    browser = webdriver.Firefox()
except Exception as e:
    try:
        browser = webdriver.Chrome()
    except Exception as e2:
        try:
            browser = webdriver.Safari()
        except Exception as e3:
            print 'Could not find Firefox, Chrome or Safari!' 
            sys.exit(0)

This nested try-except clause is not the best programming style, but I am sure it helps you to use it for your purposes. Taking the tab completion in IPython Selenium supports

Safari
Firefox
Opera and
IE.

In my case, I used Selenium mostly with Firefox.

Yes - it supports Javascript. It is like a remote control of a browser. — Jon, Dec 10 '13 at 17:02
this answer could be the best, but selenium requires pre-installed Firefox... Maybe there is a way to automaticly detect any web browser on the computer and use it? Thanks — JadedTuna, Dec 13 '13 at 15:18
In the case of Selenium my best experiences were with Firefox. Chromium works as well. Of course you always can add `try-catch` statements to your code. Please see my future edit above. — Jon, Dec 13 '13 at 15:20
Yes, if you use a Linux system with X you can use `pyvirtualdisplay`. In this case you need to install a backend support this virtualdisplay like `xvfb` and `xepyhr`(installation through apt-get). See my other post for more details: http://stackoverflow.com/questions/20485360/selenium-with-pyvirtualdisplay-unable-to-locate-element/20499923#20499923 — Jon, Dec 13 '13 at 15:41

score 3 · Answer 2 · edited Dec 10 '13 at 16:50

3

For any kind of semi-complicated HTTP requests, you should use the Requests module (http://requests.readthedocs.org/en/latest/) instead of urllib. It'll save you a lot of pain.

You'll need to do something like:

import requests
import re

data = requests.get("https://minecraft.net/login")
auth_token = re.search('name="authenticityToken"[\s]+value="(.+)"', data).group(1)

# If needed, you may have to urlencode all that.
data_dict = {
    "username": "USERNAME",
    "password": "PASSWORD",
    "remember": "true",
    #"redirect": "https://www.minecraft.net",
    "authenticityToken": auth_token
}

logged_in_data = requests.post("https://minecraft.net/login", data_dict)

edited Dec 10 '13 at 16:50

cubuspl42

7,833
4
41
65

answered Dec 10 '13 at 16:06

Max Noel

8,810
1
27
35

2

If anyone needs to do some more complex website "parsing" then use `BeautifulSoup` instead of `re`. – cubuspl42 Dec 10 '13 at 16:45
Might be a good answer, but... Later i need to access "https://www.minecraft.net/classic/list", using cookies i got from login page... Is there a way to do this using requests? – JadedTuna Dec 10 '13 at 16:51
Looks like there is: http://requests.readthedocs.org/en/latest/user/advanced/. I've never used that feature though, so you'll have to figure it out yourself. The docs are pretty good. – Max Noel Dec 10 '13 at 16:52
Yes sir! Very easy way. It's called `Session`. But from now on you should follow reqests' manual. – cubuspl42 Dec 10 '13 at 16:53
Ahh, there is a problem :( minecraft.net says (on login page): 'Please, please enable JavaScript to use this site'. And I don't think requests supports JS – JadedTuna Dec 10 '13 at 17:01
What does it need Javascript for? Is it preventing you from accomplishing your goal? Both the login page and https://minecraft.net/classic/list give perfectly usable information without having to run any JS. – Max Noel Dec 10 '13 at 17:13
What code are you trying to run, and what output and errors are you getting? – Max Noel Dec 10 '13 at 23:37
@MaxNoel, basically, everything is ok, except one thing. `Session` doesn't keep cookies I think... As when I load server list, I get `If you login...` – JadedTuna Dec 11 '13 at 17:01
According to the docs, it does. But once again, we can't help you if you don't show us the code you're running and the errors your getting. – Max Noel Dec 11 '13 at 17:21
@MaxNoel, Oh, sorry. Here it is: http://randomgalaxy.com/tools/loadpy.php?script=stackoverflow/testmcsend.py – JadedTuna Dec 11 '13 at 19:33
*and the errors you're getting*. I don't know what your program is supposed to do, but the version you pasted downloads what looks to me like a server list on the second requests call (and doesn't require any form of login, which is good because I don't have a Minecraft account). Of course you need to parse some HTML to get to them, but that's trivial with lxml.html or BeautifulSoup. – Max Noel Dec 11 '13 at 20:11

score 2 · Accepted Answer · answered Dec 14 '13 at 21:39

You can add cookielib module to login to your site.

#!/usr/bin/python
import urllib, urllib2, cookielib, re

login_url = "https://minecraft.net/login"
cookie = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie))
respon = opener.open(login_url).read()

authToken = re.search('name="authenticityToken"[\s]+value="(.+)"', respon).group(1)

data_dict = {
    "username": "USERNAME",
    "password": "PASSWORD",
    "remember": "true",
    "authenticityToken": authToken
}

login = opener.open(login_url, urllib.urlencode(data_dict))
logged_in = login.read()

if login_url in logged_in:
    print 'Login failed'
else:
    print 'Login OK'

Python urllib2 login to minecraft.net

3 Answers3

Linked