Selenium with Python

Posted February 05, 2019 by Rokas Balsys



Creating virtual click function part #4

When we are creating automated bot to do stuff for us on internet, usually we are doing this for a reason. Reason may be to get some cash, data or something like that. Usually these site may have a protection from automated programs. If we will use "click()" default Selenium function, we would click virtually on a center of object. It's easy to track or protect from such clicks. But if we would create a function that could click as a human, no one could say that we are using bot. So in this short tutorial I will create a virtual click function.

Continuing from the last tutorial, I modified two functions. At first, where we are starting a new Browser, I added an option to open a Chrome browser, when we use "2" as an option. "1" is Firefox as before and "3" is no for custom browser:

def StartBrowser(browser=1, browser_location='', driver_path=''):
    if browser == 1:
        driver = webdriver.Firefox()
    if browser == 2:
        driver = webdriver.Chrome()        
    if browser == 3:
        options = webdriver.ChromeOptions()
        options.binary_location = browser_location
        driver = webdriver.Chrome(executable_path=driver_path, chrome_options=options)
    #driver.implicitly_wait(30)
    #driver.set_window_size(1920,1080)
    #driver.maximize_window()
    return driver

In our 3rd tutorial part we had a quite simple function for opening new tabs, so I added more functionality for more advanced use. This new function has a possibility to chose to what window we will switch after new tab opens. There is a protection not to hang if our new tab opens only a blank page (possible when link doesn't work). And you can add functionality if our function opens a wrong link, or opened link redirects you somewhere else:

def NewTab(driver, Link, default_page=0, custom=0):
    window_count = len(driver.window_handles)
    driver.execute_script('''window.open("'''+Link+'''","_blank");''')
    while len(driver.window_handles) != window_count+1:
        time.sleep(0.5)
    if custom == 0:
        driver.switch_to.window(driver.window_handles[-1])
    if custom != 0:
        driver.switch_to.window(driver.window_handles[custom])    
    time.sleep(2)
    if driver.current_url == Link:
        current_window = driver.current_window_handle
        return current_window
    if driver.current_url == "about:blank":
        time.sleep(2)
        DriverClose()
        driver.switch_to.window(driver.window_handles[default_page])
        print("current window was about:blank we closed it, trying again in 10s")
        time.sleep(10)
        NewTab(driver, Link, default_page, custom)
    if driver.current_url != Link:
        pass

Here comes the final VirtualClick function we are creating. At first, lets "import random" library along with other imports. In text tutorial I will not explain about exceptions, I am doing this on video tutorial. In this function we can choose we will use random object coordinates click or we'll click always in center.

Here is the simplified function version, but if you will use it you will face errors quite soon(not recommende to use):

def VirtualClick(driver, click_object):
    size = click_object.size
    height_rand = random.randint(1,int(sizeList[0])-1)
    width_rand = random.randint(1,int(sizeList[1])-1)
    action = webdriver.common.action_chains.ActionChains(driver)
    action.move_to_element_with_offset(click_object, width_rand, height_rand)
    action.click()
    action.perform()
    return True

So our original function does the same as simplified version above. But we integrated exceptions to protect from errors. If we would try to find out what our code does above, so at first line it get x and y size of HTML object. In next two lines it picks a random height_rand and width_rand coordinates, it's random coordinates which are between 1-x and 1-y axis. In next line we are using standart ActionChains function, to do everythink like a human. Then we are moving our mouse to our chosen random coordinates and performing firtual click() function. At last line of code we return True if everything is fine there.

So bellow is the same code we'll use in our further coding, where we handle exceptions:

def VirtualClick(driver, click_object, UseRandom=True):
    try:
        size = click_object.size
    except StaleElementReferenceException:
        print("StaleElementReferenceException")
        return False
    sizeList = list(size.values())
    height = int(sizeList[0])-1
    width = int(sizeList[1])-1
    if UseRandom == True:
        try:
            height_rand = random.randint(1,height)
        except ValueError:
            height_rand = 1
        try:
            width_rand = random.randint(1,width)
        except ValueError:
            width_rand = 1
    if UseRandom == False:
        height_rand = height
        width_rand = width
    action = webdriver.common.action_chains.ActionChains(driver)
    try:
        action.move_to_element_with_offset(click_object, width_rand, height_rand)
    except StaleElementReferenceException:
        return False
    action.click()
    try:
        action.perform()
    except MoveTargetOutOfBoundsException:
        print("MoveTargetOutOfBoundsException with action.perform()")
        return False
    except StaleElementReferenceException:
        print("StaleElementReferenceException with action.perform()")
        return False
    return True

And here comes the final up to date code:
from selenium import webdriver
import time
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoSuchFrameException
from selenium.common.exceptions import NoSuchWindowException
from selenium.common.exceptions import ElementNotInteractableException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.common.exceptions import MoveTargetOutOfBoundsException
from selenium.common.exceptions import UnexpectedAlertPresentException
from selenium.common.exceptions import ElementClickInterceptedException
from selenium.common.exceptions import JavascriptException
import random

def CheckExistsById(driver, CheckId):
    try:
        DriverId = driver.find_element_by_id(CheckId)
    except NoSuchWindowException:
        return False
    return DriverId

def CheckExistsByName(driver, CheckName):
    try:
        DriverName = driver.find_element_by_name(CheckName)
    except NoSuchWindowException:
        return False
    return DriverName

def CheckExistsByXpath(driver, xpath):
    try:
        path = driver.find_element_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

def CheckExistsArrayByXpath(driver, xpath):
    try:
        path = driver.find_elements_by_xpath(xpath)
    except NoSuchWindowException:
        return False
    except WebDriverException:
        return False
    except StaleElementReferenceException:
        return False
    return path

def MinimizeWindow(driver):
    try:
        driver.minimize_window()
    except UnexpectedAlertPresentException:
        return False
    return True

def DriverClose(driver):
    try:
        driver.close()
    except NoSuchWindowException:
        print("NoSuchWindowException in driver.close()")
        return False
    return True

def SimpleClick(driver, ObjectToClick):
    try:
        ObjectToClick.click()
    except ElementNotInteractableException:
        return False
    except ElementClickInterceptedException:
        return False
    except TypeError:
        return False
    return True

def StartBrowser(browser=1, browser_location='', driver_path=''):
    if browser == 1:
        driver = webdriver.Firefox()
    if browser == 2:
        driver = webdriver.Chrome()        
    if browser == 3:
        options = webdriver.ChromeOptions()
        options.binary_location = browser_location
        driver = webdriver.Chrome(executable_path=driver_path, chrome_options=options)
    #driver.implicitly_wait(30)
    #driver.set_window_size(1920,1080)
    #driver.maximize_window()
    return driver

def NewTab(driver, Link, default_page=0, custom=0):
    window_count = len(driver.window_handles)
    driver.execute_script('''window.open("'''+Link+'''","_blank");''')
    while len(driver.window_handles) != window_count+1:
        time.sleep(0.5)
    if custom == 0:
        driver.switch_to.window(driver.window_handles[-1])
    if custom != 0:
        driver.switch_to.window(driver.window_handles[custom])    
    time.sleep(2)
    if driver.current_url == Link:
        current_window = driver.current_window_handle
        return current_window
    if driver.current_url == "about:blank":
        time.sleep(2)
        DriverClose()
        driver.switch_to.window(driver.window_handles[default_page])
        print("current window was about:blank we closed it, trying again in 10s")
        time.sleep(10)
        NewTab(driver, Link, default_page, custom)
    if driver.current_url != Link:
        pass

def VirtualClick(driver, click_object, UseRandom=True):
    try:
        size = click_object.size
    except StaleElementReferenceException:
        print("StaleElementReferenceException")
        return False
    sizeList = list(size.values())
    height = int(sizeList[0])-1
    width = int(sizeList[1])-1
    if UseRandom == True:
        try:
            height_rand = random.randint(1,height)
        except ValueError:
            height_rand = 1
        try:
            width_rand = random.randint(1,width)
        except ValueError:
            width_rand = 1
    if UseRandom == False:
        height_rand = height
        width_rand = width
    action = webdriver.common.action_chains.ActionChains(driver)
    try:
        action.move_to_element_with_offset(click_object, width_rand, height_rand)
    except StaleElementReferenceException:
        return False
    action.click()
    try:
        action.perform()
    except MoveTargetOutOfBoundsException:
        print("MoveTargetOutOfBoundsException with action.perform()")
        return False
    except StaleElementReferenceException:
        print("StaleElementReferenceException with action.perform()")
        return False
    return True

browser_location = "C:/Users/HOME/Desktop/chrome-win/chrome.exe"
driver_path = "C:/Users/HOME/Desktop/chrome-win/chromedriver.exe"
driver = StartBrowser(1, browser_location, driver_path)
time.sleep(1)
driver.get("https://www.google.com")
time.sleep(1)
NewTab(driver, "https://www.google.com/gmail/about/new/")
signin = CheckExistsByXpath(driver, "//a[starts-with(.,'Sign In')]")
VirtualClick(driver, signin)
time.sleep(0.5)
driver.switch_to.window(driver.window_handles[-1])
time.sleep(2)
email = CheckExistsByXpath(driver, "//input[@type='email']")
email.send_keys("email")
next = CheckExistsByXpath(driver, "//span[@class='RveJvd snByac']")
VirtualClick(driver, next)


So now we have quite nice function pack to use in our coding. You can use them for your own purposes or just test this code. I wrote a simple example, which may be used to login to gmail account. But this code not always works, because gmail usually changes his login page HTML code, so if it won't work, you'll need to adapt it. Other way you can continue on this tutorial series where you will find more interesting stuff on selenium.