Unable to locate the Sign In element within #shadow-root (open) using Selenium and Python

Question

I'm trying to use selenium to automate some actions but am unable to find the first element on the page https://developer.servicenow.com/dev.do and so cannot login

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver_path = "../bin/chromedriver.exe"
driver = webdriver.Chrome(driver_path)

driver.get("https://developer.servicenow.com/dev.do")

driver.find_element_by_xpath("/html/body/dps-app//div/header/dps-navigation-header//header/div/div[2]/ul/li[3]/dps-login//div/dps-button//button/span")

I get the error

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/dps-app//div/header/dps-navigation-header//header/div/div[2]/ul/li[3]/dps-login//div/dps-button//button/span"}

https://stackoverflow.com/questions/36141681/does-anybody-know-how-to-identify-shadow-dom-web-elements-using-selenium-webdriv — Chris, Jul 08 '20 at 16:27

undetected Selenium · Accepted Answer · 2021-01-22T18:59:44.617

To Sign In button is deep within multiple #shadow-root (open)

shadow-root-open-multiple

Solution

Tto click() on the desired element you can use shadowRoot.querySelector() and you can use the following Locator Strategy:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://developer.servicenow.com/dev.do')
SignInButton = driver.execute_script("return document.querySelector('dps-app').shadowRoot.querySelector('dps-navigation-header').shadowRoot.querySelector('header.dps-navigation-header dps-login').shadowRoot.querySelector('dps-button')")
SignInButton.click()

Browser Snapshot:

shadow-root-open

References

You can find a couple of relevant detailed discussions in:

score 0 · Answer 2 · answered Nov 06 '21 at 09:20

You could look at the Automatepro app on the servicenow store. It uses selenium but they have pre-written the selenium behind the scenes to interact with all the servicenow UI components, including those using shadow DOM. You just pick the UI element and supply the data for your test. We found it saves a lot of time writing and maintaining the selenium code ourselves..

score 0 · Answer 3 · answered May 11 '22 at 20:48

This solution was what I was looking for to log into my instance using Powershell.

driver.executescript("return document.querySelector('dps-app').shadowRoot.querySelector('dps-navigation-header').shadowRoot.querySelector('header.dps-navigation-header dps-login').shadowRoot.querySelector('dps-button')").Click()

cquick123 · Answer 4 · 2023-06-23T01:20:53.100

I wrote a blog that answers a lot of your issues here:

https://www.seamlessmigration.com/servicenow-selenium-shadown-doms/

Here are some snippets of it to provide you with the answers you need.

# Check if an element is present using JS path.
# Particularly useful for shadow DOM elements
def is_element_present(driver, js_path):
    try:
        element = driver.execute_script(f"return {js_path}")
        return element is not None
    except Exception:
        pass
    try:
        outer_html = driver.execute_script(f"return {js_path}.outerHTML;")
        if outer_html:
            return True;    
    except Exception:
        return False
 
# used to click shadow elements w/ js path.
# uses javascript to allow for explicit waits 
def click_shadow_element(driver, errorName, js_path, wait_time = 10):
    try:
        WebDriverWait(driver, wait_time).until(lambda x: self.is_element_present(js_path))
        driver.execute_script(f"return {js_path}.click()")
    except (TimeoutException, ElementNotVisibleException, ElementNotInteractableException) as e:
        raise Exception(f"The {errorName} button was not found. Additional info: {str(e)}")

So, in essence, the functions are built to make this a replicable situation. The real solution lies in copying the JS path and using the execute_script() method to access the element.

What I have created is the function click_shadow_element() with the helper function is_element_present() to try and retain some of the explicit wait abilities that we love so much from Selenium. There are also functions to Log In, Log Out and accept alerts when they pop up.

Enjoy.

https://meta.stackexchange.com/a/8259/997587, [/help/referencing](/help/referencing) — starball, Jun 22 '23 at 00:26
While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/low-quality-posts/34582215) — Michael S., Jun 23 '23 at 00:12
@MichaelS. Thanks for the reply. You're absolutely correct. I created the blog recently, and while trying to refine it for google searches, I totally messed up the link. I'll fix the link and also provide some of the answers in the answer post. — cquick123, Jun 23 '23 at 01:04

score -1 · Answer 5 · answered Jul 08 '20 at 20:42

-1

I think its because you are trying to access the "Sign in" before the website has time to load all of the elements.

I suggest you use

driver.set_page_load_timeout(120)

so the selenium will look for the button after everything is loaded on the website.

answered Jul 08 '20 at 20:42

Michael Krezalek

104
7

Unable to locate the Sign In element within #shadow-root (open) using Selenium and Python

5 Answers5

Solution

References

Linked

Related