Logging in to phpBB forum programmatically through C#

Question

I'm working on a C# application that needs to scrape some data from a phpBB forum. The forum scraping requires logging in. The application will prompt the user for their login credentials to connect.

I've scraped websites before with C#, but what I'm not sure how to do is login to phpBB and keep a session open during the duration of the screen scraping. I've done some searching and haven't had much luck. Is there a good way to programmatically do something like this?

score 1 · Accepted Answer · answered Jun 14 '11 at 02:13

You don't say what you've tried, but if you used an HttpWebRequest object to retrieve pages and/or logon, then you need to assign a new CookieContainer collection to the HttpWebRequest to store any cookies returned by the website. Share this amongst HttpWebRequest objects to remain logged in

score 0 · Answer 2 · edited May 23 '17 at 12:04

0

look for the names of the username and password fields using Firebug or Chrome (or even View Source), then use my answer here: Programmatically logging into a site, replacing 'session_key' and 'session_password' as appropriate. that should work.

and then translate to C#!

edited May 23 '17 at 12:04

Community

1
1

answered Jun 14 '11 at 01:45

jcomeau_ictx

37,688
6
92
107

score 0 · Answer 3 · edited Nov 28 '17 at 03:55

0

I recommend using HTML Agility Pack.

edited Nov 28 '17 at 03:55

wp78de

18,207
7
43
71

answered Jun 14 '11 at 01:46

Richard Schneider

34,944
9
57
73

I plan to use HTML Agility Pack to scrape and parse the content, but how will it help logging in to the forum? – Tyler Treat Jun 14 '11 at 01:47
You can use a NetworkCredential to specify a username/password when loading a document. See http://htmlagilitypack.codeplex.com/discussions/207254 – Richard Schneider Jun 14 '11 at 02:08
NetworkCredential will do you no good--the login system is all via POST and HTML, not HTTP 401. – Jeff Hubbard Jun 14 '11 at 02:16
@Jeff: this is kind of what I figured, unless I'm missing something. – Tyler Treat Jun 14 '11 at 02:23

score 0 · Answer 4 · edited Dec 21 '11 at 15:18

0

I would recommend using WatiN API for doing screen scraping. I have done screen scraping using this API and it does good work. Check it out !

edited Dec 21 '11 at 15:18

Widor

13,003
7
42
64

answered Jun 14 '11 at 02:08

Bibhu

4,053
4
33
63

WatiN is a testing framework that opens a browser and executes commands. Automating browser functionality is not what I'm looking for. – Tyler Treat Jun 14 '11 at 03:12

Logging in to phpBB forum programmatically through C#

4 Answers4