How You Can Create Your Own Bot Using Selenium WebDriver And Python

When I created my first bot with Selenium library – I could not believe how easy and intuitively Selenium is. It’s like writing a scenario for a movie – You just describing step by step what should be going on. In my particular case – Selenium helped me to automate reporting of some advertising platforms, where I had no access to an API. But also can be used for web scraping, automation, etc.

I wrote a simple bot which sends me an email from my web page contact form as an example. This is how it works:

How to create your own bot

In order to lunch your bot you should have a special simple web browser app – the most popular are ChromeDriver (equivalent of Chrome) and GeckoDriver (equivalent of Mozilla Firefox). I’ll use ChromeDriver in this tutorial. So at first – go to official ChromeDriver page and download it:
https://chromedriver.chromium.org/downloads

Remember – You should have the same version of ChromeDriver as your normal Chrome browser installed on your computer, you can check it by typing this: chrome://settings/help
Next – install the Selenium library using pip as usual:

pip install selenium

Next – try to run the code below after replacing ‘/path_to_chromedriver/chromedriver’ with full path to chromedriver.exe on your computer:

import time
from selenium import webdriver

options = webdriver.ChromeOptions()
driver = webdriver.Chrome('/path_to_chromedriver/chromedriver', options=options)
driver.maximize_window()
driver.get('https://serhiipuzyrov.com/contact');
time.sleep(3)
driver.find_elements_by_xpath('//*[@name="your-name"]')[0].send_keys("selenium bot")
driver.find_elements_by_xpath('//*[@name="your-email"]')[0].send_keys("selenium@bot.com")
driver.find_elements_by_xpath('//*[@name="your-message"]')[0].send_keys("Hello! =)")
time.sleep(3)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(3)
driver.find_elements_by_xpath('//*[@value="Send"]')[0].click()
time.sleep(3)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(3)
driver.quit()

This script should do all the stuff you have seen in the example at beginning of this article. Waiting (time.sleep(x)) is necessary to give the page time to load, values inside – seconds.
With Selenium You can do almost anything what comes to your mind: clicking buttons, filling forms, downloading files, making screenshots and so on. While writing a script for Selenium bot – You should think about it like you’re writing a very detailed instruction for a human.

Of course you should tell Selenium somehow which particular element on page you want to interact with. You can use different methods for that: find element by it’s id, class, css selector, or even find an element which contains a specific text. There you can find the full specification:
https://selenium-python.readthedocs.io/locating-elements.html

Below You can find some popular commands:
This will click the first element (could be a button for example) which contains “download” text:

driver.find_elements_by_xpath("//*[contains(text(), 'download')]")[0].click()

This command will find form with “type=’search'” attribute and fill it with “hello” string:

driver.find_elements_by_xpath('//*[@type="search"]')[0].send_keys("hello")

This is how you can make a screenshot of page:

driver.save_screenshot('screenshot.png')

Also You can run Selenium in “headless mode” – it means that the script will be running in background. You will not see how it’s working but still you will have the results of it’s work. There is an example which opens Google News and makes a screenshot of the page:

import time
from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome('/path_to_chromedriver/chromedriver', options=options)
driver.get('https://news.google.com/');
time.sleep(5)
driver.save_screenshot('screenshot.png')
driver.quit()

Thanks to this mode You can run script on a server, I prefer using Google Cloud VM for that purposes.
Hope this article will help You to start your journey with Selenium and it will help You to automate some boring or repeatable suff. Cheers!