python selenium: possible to cancel redirect on driver.get()?

Question

Is there a way to stop a url from redirecting?

driver.get('http://loginrequired.com')

This redirects me to another page but I want it to stay on that page without redirecting by default.

Selenium just acts and behaves like you're using your browser, so no I don't think you can stop this, unless you have configured your browser to prevent it from happening — Anzel, Jan 05 '15 at 07:28
You could send the 'Esc' key to the page before the redirect page starts load — Ryan, Jan 05 '15 at 08:56

score 6 · Accepted Answer · edited Jun 20 '20 at 09:12

There are two ways that what users call "redirection" typically happens:

You load a page and the page loads some JavaScript code which performs a test and decides to load a different page. This process can be interrupted in some browsers by hitting the ESCAPE key. Selenium can send an ESCAPE key.

However, this redirection could happen before Selenium gives control back to your script. Whether it would work in any specific case depends on the page being loaded.
You load a page and get an HTTP 3xx (301, 303, 304, etc.) response from the server. There are no opportunities for users to interrupt these redirections in their browser, so Selenium does not provide the means to interrupt or prevent them.

So there is no surefire way to prevent a redirection in Selenium.

Is there any way to tell that there is a redirect happening or about to happen? I tried using `EventFiringWebDriver` with the `Navigating` event but it doesn't fire unless you load the address explicitly. I want to test that a form submitted and then redirects to an external website, there's no way to check for it. — Bron Davies, Feb 03 '16 at 01:46

score -1 · Answer 2 · edited May 23 '17 at 12:06

A solution, in case you do not need to visualize the page but access to the source of "http://loginrequired.com" would be the usage of Selenium with Scrapy.

Basically you tell the Scrapy middleware to stop redirecting, and while the spider access to the page the redirect is handle the redirection (302).

In the setting.py you have to set

"REDIRECT_ENABLED=False"

The spider code is:

class LoginSpider(CrawlSpider):
    name = "login"
    allowed_domains = ['loginrequired.com']
    start_urls = ['http://loginrequired.com']
    handle_httpstatus_list = [302]

def __init__(self):
    self.driver = webdriver.Firefox()

def parse(self, response):
    if response.status in self.handle_httpstatus_list:
        return Request(url="http://loginrequired.com", callback=self.after_302)

def after_302(self, response):
    print response.url
    # Your code to analysis the page by here

Idea taken from how to handle 302 redirect in scrapy

python selenium: possible to cancel redirect on driver.get()?

2 Answers2

Linked