I struggled to parse data from external website, for example, stackoverflow.com, using only Java.
To find out what this webpage is about I went to Chrome Development Tools and found there XMLHTTPRequest response with all the information I need! If its useful, response has a JSON format. The question is how to get this data using only Java and without Servlets.
Asked
Active
Viewed 298 times
0
Garbee
- 10,581
- 5
- 38
- 41
antongarakh
- 570
- 6
- 16
-
**I tried to use something like WebDriver, but found it too hard to work with cssSelector or things like this** . What exactly you were intended to do? Can you provide code examples that you tried? – Andersson Mar 17 '16 at 10:31
-
I think no matter what exactly i tried to do. The question is how to get response of XMLHTTPRequest which has all data I need – antongarakh Mar 17 '16 at 10:48
1 Answers
0
Don't try to grab and parse the web page, just use the StackExchange API and standard Java tools to make a request and get the raw data.
If your actual question isn't related to SO directly then, well, look for an API to the site in question. Otherwise you are going to need to literally scrape the web page by parsing it manually or running the page in a browser engine and using standard JS in a headless engine to get the data out.
Garbee
- 10,581
- 5
- 38
- 41
-
well the answer for me is to use Apache HTTPClient which provides tools to execute the needed request – antongarakh May 29 '16 at 20:14
-
@antongarakh could you please give me you solution ? I 'm in desperate need for it please – maryem neyli Apr 18 '20 at 15:17