Methods to find HTML objects with Selenium

Welcome, everyone to the "Selenium with Python" third tutorial part. In this tutorial, I will show you how to find HTML elements and interact with them in different ways. As an example, I'll use Gmail log-in page. Here we'll try to find an input where you need to write your email and how to click the "Next" button.

Before continuing on this tutorial I would like to talk about exceptions. "Exception" is a quite common term when it comes to programming, regardless of which language you use to write codes. Exceptions in Selenium are like enemies for programmers - they are unavoidable. Even if you work with other automation testing tools similar to Selenium.

In every programming, language exception is known as an unusual or unprecedented event that occurs during the execution of a software program or application. It is a run time error of an unexpected result or event which influences and disrupts the usual program flow. An exception is also considered as a fault.

So firstly at the beginning of code, I will import several exceptions that occur mostly. These exceptions should handle most the unexpected faults:

from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoSuchFrameException
from selenium.common.exceptions import NoSuchWindowException
from selenium.common.exceptions import ElementNotInteractableException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.common.exceptions import MoveTargetOutOfBoundsException
from selenium.common.exceptions import UnexpectedAlertPresentException
from selenium.common.exceptions import ElementClickInterceptedException
from selenium.common.exceptions import JavascriptException

CheckExistsById:

Next, we'll try to with HTML elements by using id. We know that we are looking for id="identified". So we will use the Selenium command: driver.find_element_by_id. So below you can see that we created a function that will handle us one NoSuchWindowException exception.

def CheckExistsById(driver, CheckId):
    try:
        DriverId = driver.find_element_by_id(CheckId)
    except NoSuchWindowException:
        return False
    return DriverId

So when our browser started and we are on the Gmail login window, try to write the following line to Python shell. These Lines should write the word "email" to an input.

Id = CheckExistsById(driver, "identifierId"):
Id.send_keys("email")

CheckExistsByName:

We can do the same task by trying to find elements by name, let's write a similar function to find HTML objects by name:

def CheckExistsByName(driver, CheckName):
    try:
        DriverName = driver.find_element_by_name(CheckName)
    except NoSuchWindowException:
        return False
    return DriverName

So again, when our browser started and we are on the Gmail login window, try to write the following line to Python shell. We will get the same result as in the previous example.

Name = CheckExistsByName(driver, "identifier"):
Name.send_keys("email")

CheckExistsByXpath:

Now comes my favorite function. Most of the time I am using driver.find_element_by_xpath, because while using xpath we can do the same task by trying to find elements by name, id, input, or whatever we want. So here is the function:

def CheckExistsByXpath(driver, xpath):
    try:
        path = driver.find_element_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

When our browser started and we are on the Gmail login window, try to write the following line to Python shell. We will get the same result as in the previous example. Here are several examples to get the same result:
CheckExistsByXpath(driver, "//input[@type='email']")
CheckExistsByXpath(driver, "//input[@name='identifier']")
CheckExistsByXpath(driver, "//input[@id='identifierId']")
As you can see, we can achieve the same results as before with previous functions with simple one command, so why not use this everywhere?

Sometimes when we are looking for an HTML object on the website, we may get a few objects with the same name. For example, if we are looking for lists in tables or something like that. In this case, we would like to get an array of objects, so for this purpose, I wrote another very similar function:

def CheckExistsArrayByXpath(driver, xpath):
    try:
        path = driver.find_elements_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

And here comes the final up to date code:

from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoSuchFrameException
from selenium.common.exceptions import NoSuchWindowException
from selenium.common.exceptions import ElementNotInteractableException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.common.exceptions import MoveTargetOutOfBoundsException
from selenium.common.exceptions import UnexpectedAlertPresentException
from selenium.common.exceptions import ElementClickInterceptedException
from selenium.common.exceptions import JavascriptException

def CheckExistsById(driver, CheckId):
    try:
        DriverId = driver.find_element_by_id(CheckId)
    except NoSuchWindowException:
        return False
    return DriverId

def CheckExistsByName(driver, CheckName):
    try:
        DriverName = driver.find_element_by_name(CheckName)
    except NoSuchWindowException:
        return False
    return DriverName

def CheckExistsByXpath(driver, xpath):
    try:
        path = driver.find_element_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

def CheckExistsArrayByXpath(driver, xpath):
    try:
        path = driver.find_elements_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

def MinimizeWindow(driver):
    try:
        driver.minimize_window()
    except UnexpectedAlertPresentException:
        return False
    return True

def DriverClose(driver):
    try:
        driver.close()
    except NoSuchWindowException:
        print("NoSuchWindowException in driver.close()")
        return False
    return True

def SimpleClick(driver, ObjectToClick):
    try:
        ObjectToClick.click()
    except ElementNotInteractableException:
        return False
    except ElementClickInterceptedException:
        return False
    except TypeError:
        return False
    return True

def StartBrowser(browser=1, browser_location='', driver_path=''):
    if browser == 1:
        driver = webdriver.Firefox()
    if browser == 2:
        options = webdriver.ChromeOptions()
        options.binary_location = browser_location
        driver = webdriver.Chrome(executable_path=driver_path, chrome_options=options)
    #driver.implicitly_wait(30)
    #driver.set_window_size(1920,1080)
    #driver.maximize_window()
    return driver

def OpenNewTab(driver, Link):
    window_count = len(driver.window_handles)
    driver.execute_script('''window.open("'''+Link+'''","_blank");''')
    while len(driver.window_handles) != window_count+1:
        time.sleep(0.5)
    driver.switch_to.window(driver.window_handles[-1])

browser_location = "C:/Users/HOME/Desktop/chrome-win/chrome.exe"
driver_path = "C:/Users/HOME/Desktop/chrome-win/chromedriver.exe"
driver = StartBrowser(1, browser_location, driver_path)
time.sleep(1)
driver.get("https://www.google.com")
time.sleep(1)
OpenNewTab(driver, "https://www.google.com/gmail/about/new/")

So, now you should know a lot where to start with Selenium, I gave you very good basics: how to start a browser, open and close new tabs, how to find HTML objects and how to type into them. Of course, there is a lot to learn, but the best way to do this is just to start doing something. So if you want to learn more, keep reading future tutorials.