Programmatically logging into a site

Question

It might sound silly; but can we programmatically login into a site such as Linkedin by passing our user credentials (userid and password)? I am not talking about using OAuth or other mechanism.

Edit:

I am using the following code in Page_Load method of an .aspx web page to login into Linkedin but it's not working. I see the login page again:

  try {
      string url = "https://www.linkedin.com/uas/login";
      string userName = "SomeEmail@SomeDomain.Com";
      string password = "--------";
      string domain = "linkedin.com";
      string postData = "session_key=" + userName + "&session_password=" + password;
      string userAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)";
      string method = "POST";
      string contentType = "text/html; charset=UTF-8";

      HttpWebRequest http = WebRequest.Create(url) as HttpWebRequest;

      http.UserAgent = userAgent;            
      http.Method = method;
      http.ContentType = contentType;            
      http.Credentials = new NetworkCredential(userName, password, domain);

      byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(postData);
      http.ContentLength = dataBytes.Length;

      http.Timeout = -1;
      http.KeepAlive = true;
      http.CookieContainer = new CookieContainer();

      using (Stream postStream = http.GetRequestStream())
      {
         postStream.Write(dataBytes, 0, dataBytes.Length);
      }

      HttpWebResponse httpResponse = http.GetResponse() as HttpWebResponse;

      if (httpResponse.Cookies != null)
      {
         foreach (Cookie cok in httpResponse.Cookies)
         {
            if (cok != null)
            {
               Response.Write(String.Format("Name = {0}- Value = {1}- Path = {2}<br/>", cok.Name, cok.Value, cok.Path));
            }
         }
      }   

      Response.Write(httpResponse.Headers.ToString() + "<br/>");
      Response.Write(new StreamReader(httpResponse.GetResponseStream()).ReadToEnd());
  } catch(Exception ex) 
  { 
      Response.Write(ex.ToString());
  }

jcomeau_ictx · Accepted Answer · 2011-06-10T01:37:51.780

2

You can login to many sites this way, using scripting. I generally prefer Python with BeautifulSoup, but there are many other possibilities for languages and libraries.

The general idea is to locate the code for the form that has the username and password fields, fill them in, then post the form.

This works for me on Linux, with my LinkedIn username and password in my .netrc in the standard format:

#!/usr/bin/python
import sys, os, urllib, urllib2, cookielib, urlparse, netrc
from BeautifulSoup import BeautifulSoup
URL = 'https://www.linkedin.com/uas/login'
COOKIES = urllib2.HTTPCookieProcessor(cookielib.CookieJar())
HANDLER = getattr(urllib2, urlparse.urlsplit(URL).scheme.upper() + 'Handler')
DEBUGLEVEL = 0  # set to 1 to see unencrypted data
OPENER = urllib2.build_opener(COOKIES, HANDLER(debuglevel = DEBUGLEVEL))
def fetch(url, data = None):
 connection = OPENER.open(url, data)
 page = connection.read()
 connection.close()
 return page
def login():
 soup = BeautifulSoup(fetch(URL))
 form = soup.find('form')
 fields = form.findAll('input')
 auth_info = netrc.netrc().authenticators(urlparse.urlsplit(URL).netloc)
 formdata = dict([[field['name'], field['value']] for field in fields])
 formdata['session_key'], ignored, formdata['session_password'] = auth_info
 assert form['method'] == 'POST'
 posturl = urlparse.urljoin(URL, form['action'])
 print fetch(posturl, urllib.urlencode(formdata))
if __name__ == '__main__':
 login()

edited Jun 10 '11 at 01:37

answered Jun 08 '11 at 08:05

jcomeau_ictx

37,688
6
92
107

how do you know your postdata is complete? the form doesn't have any hidden fields? most do, with random data generated to foil simple automated login schemes. – jcomeau_ictx Jun 08 '11 at 16:53
I used the TamperData to see what fields are being submitted. – S M Kamran Jun 09 '11 at 07:11
I don't know TamperData, but I just checked the form and there are several hidden fields. – jcomeau_ictx Jun 09 '11 at 07:27
You mean the fields.... source_app, session_redirect, csrfToken, sourceAlias and signin..... But I guess the values for session_redirect, csrfToken and SourceAlias are filled dynamically by the server and thus it might be verified on the server as well. I guess this means we can't login. Or should we first get the Login page parse these values and then send the request using these values. – S M Kamran Jun 09 '11 at 07:42
exactly. you get the login page, grab all the input fields and values from the form, fill in username and password, and post back. – jcomeau_ictx Jun 09 '11 at 07:45
see my most recently modified answer. I don't imagine your .net code will need to be much different. – jcomeau_ictx Jun 09 '11 at 08:21
yes, it works; when I grep the output, I see my full name "John Comeau", which it could not have gotten unless I'd logged in successfully. – jcomeau_ictx Jun 09 '11 at 08:29
@jcomeau_ictx it would really help i you can share your code :) to login to linked in . – confusedMind Jun 14 '12 at 21:31
it's all there, is it not working any more? perhaps linkedin changed their page, I haven't used this for a while. – jcomeau_ictx Jun 14 '12 at 22:06

score 0 · Answer 2 · answered Jun 08 '11 at 08:06

0

Yes, you could use a HttpWebRequest in order to send HTTP requests to a given URL. So the idea would be to POST your credentials to the given url which is supposed to authenticate the user and capture any cookies that could be used on subsequent requests to authenticated parts of the site.

Of course that's pretty raw work, if the remote site provides an API you might also use this API as it will simplify things.

answered Jun 08 '11 at 08:06

Darin Dimitrov

1,023,142
271
3,287
2,928

The remote site provides API's such as Linkedin but that API requires OAuth mechanism... Which in turn requires user's intervention. – S M Kamran Jun 08 '11 at 08:13

score 0 · Answer 3 · answered Jun 08 '11 at 08:14

0

You can use tools like selenium to automate and generate code for programmatically login. Also a search in google sure reveals more tools aimed to do "mashups" of web sites, like BeautifulSoup recommended in first answer.

Regards

answered Jun 08 '11 at 08:14

franblay

306
3
8

Programmatically logging into a site

3 Answers3

Linked