3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Submitting Forms with Selenium in

Python
When scraping, we often run into sites that need us to submit information. Forms play several essential
roles in the web ecosystem, serving as crucial elements that facilitate user interaction, data input, and
communication with web applications.

Selenium stands out as a powerful and versatile tool, providing a seamless solution for automating
browser actions. With Selenium, we can find any element on a page and we have the power to interact
with it just like a regular human user would. Not only can we find these elements quickly, but we can fill
and submit them at lightning speed in comparison to a regular user.

In this article, we will delve into the fundamental concepts, practical examples, and best practices for
harnessing the full potential of Selenium to navigate and interact with web forms.

TLDR
Interacting With Form Elements
Methods of Submitting Forms
Uploading Files with Selenium
Troubleshooting
Conclusion
More Web Scraping Guides

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 1/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

TLDR: How To Submit Form In Selenium

To submit a form in Selenium, these steps should be followed:

1. Locate the Form: Identify the HTML form on the webpage using Selenium's element locating
methods.

2. Fill in Data: Use Selenium to input data into the form fields, mimicking user interactions.

3. Submit the Form: Locate the submit button or trigger the form submission action using Selenium.

4. Handle Confirmation: If there's a confirmation pop-up or post-submission action, use Selenium to

handle it.

The example below shows a basic breakdown of how to fill a form in Selenium:

from selenium import webdriver

from selenium.webdriver.common.by import By
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://quotes.toscrape.com/login")
#find our elements
username_box = driver.find_element(By.ID, "username")
password_box = driver.find_element(By.ID, "password")
login_button = driver.find_element(By.XPATH, "/html/body/div/form/input[2]")
#send keys to the input boxes
username_box.send_keys("my_username")
password_box.send_keys("my-super-secret-password")
#click the submit button
login_button.click()
#close the browser
driver.quit()

First, we find our elements with the find_element() method

Once we have our input elements, we fill them with the send_keys() method
Finally, after filling the form, we click the "Login" button to submit our information

Interacting with Form Elements In Selenium

To interact with form elements, we first have to identify them. In Selenium, we use two methods to do
this.

1. find_element finds a single element on the page and returns it to us.

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 2/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

2. find_elements() finds all elements of a certain criteria and returns them in the form of a list.

Interacting With Text Input Boxes

As we have done already, we can select values by saving them as variables. Anytime you use
find_element() or find_elements() , you get a web element that you can try all sorts of various
methods to interact with. When filling an input box, we use the send_keys() method.

In the TLDR section above, we used find_element() to find each individual input field. Let's revise the
example to find both elements and return them in a list.

from selenium.webdriver.common.by import By

#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://quotes.toscrape.com/login")
#find our elements
elements = driver.find_elements(By.CLASS_NAME, "form-control")
#send keys to the input boxes
for element in elements:
element.send_keys("just some random text")
login_button = driver.find_element(By.XPATH, "/html/body/div/form/input[2]")
login_button.click()
#close the browser
driver.quit()

In the example above, we:

Launched a Chrome browser instance using the WebDriver.

Navigated to the Quotes to Scrape login URL.
Found all elements on the page with the class name "form-control" and store them in the elements
list.
Iterated through each element in the elements list and type "just some random text" into the input
boxes.
Located the login button using its XPath ("/html/body/div/form/input[2]").
Clicked the login button to simulate form submission.
And finally closed the Chrome browser instance using driver.quit() .

Deleting Values in Input Boxes

Deleting values in input boxes is often necessary for several reasons in web scraping, particularly to
clear existing search queries or filter criteria, enabling the scraper to input new parameters and
dynamically adapt to changing requirements, ultimately retrieving the desired data accurately.

If we wish to delete values from an input box, we can use the clear() method. This method removes all
input from our web element. Take a look at the example below:

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 3/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://quotes.toscrape.com/login")
#find our elements
username_box = driver.find_element(By.ID, "username")
password_box = driver.find_element(By.ID, "password")
#send keys to the input boxes
username_box.send_keys("my_username")
password_box.send_keys("my-super-secret-password")
#sleep so we can view the filled input
sleep(1)
#clear the input
username_box.clear()
#sleep so we can view the cleared input
sleep(1)
#close the browser
driver.quit()

In the example above, we:

Find the input boxes with find_element()

Fill input boxes with send_keys()
sleep() for one second so we can see the input boxes get filled
Delete input from username_box with clear()
sleep() for another second so we can see that the input has been cleared from username_box

Interacting With Checkboxes

Checkboxes are commonly used in forms and settings to allow users to select or deselect options.

When selecting checkboxes, we simply click() them just like you would when browsing the web like
normal. The click() method gives us the power to do anything we would normally do with our mouse,
and to think the way we would think when browsing the web.

The example below finds all elements, of a certain type and selects them.

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://designsystem.digital.gov/components/checkbox/")
#find our elements
elements = driver.find_elements(By.CLASS_NAME, "usa-checkbox__label")
for element in elements:
https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 4/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

if not element.is_selected():
element.click()
sleep(1)
else:
None
#close the browser
driver.quit()

In the code example above:

Created an instance of the Chrome browser using the webdriver.Chrome() constructor.

Opened the specified URL ("https://designsystem.digital.gov/components/checkbox/") in the
Chrome browser.
Located all elements on the page with the class name "usa-checkbox__label" and store them in the
elements list.
Iterated through each element in the elements list.
Checked if the checkbox is not already selected using element.is_selected() .
If not selected, simulated a click on the checkbox using element.click() and introduce a 1-
second delay using sleep(1) for visualization.
If the checkbox is already selected, did nothing ( else: None ).
Quit the Chrome browser instance using driver.quit() .

Interacting With Radio Buttons

Radio buttons are commonly used in forms to allow users to choose a single option from a set.

When interacting with radio buttons, we also use the click() method. Not all the elements returned
are clickable, so we use a try statement to attempt clicking them and an except statement to do
nothing with them if they're not clickable.

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://designsystem.digital.gov/components/radio-buttons/")
#find our elements
elements = driver.find_elements(By.CLASS_NAME, "usa-radio__label")
for element in elements:
if not element.is_selected():
try:
element.click()
sleep(1)
except:
None
else:
None

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 5/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

#close the browser

driver.quit()

In the example above, we followed a similar route with the previous example.

Located all elements on the page with the class name "usa-radio__label" and stored them in the
elements list.
Iterated through each element in the elements list.
Checked if the radio button was not already selected using element.is_selected() .
If not selected, attempted to simulate a click on the radio button using element.click() and
introduced a 1-second delay using sleep(1) for visualization. If an exception occurred during
the click attempt, did nothing ( except: None ).
If the radio button was already selected, did nothing ( else: None ).
Quit the Chrome browser instance using driver.quit() .

Interacting With Dropdowns

Dropdowns, also known as select elements, are commonly used in forms to allow users to choose from
a list of predefined options.

Once again, as before, we use the click() method to begin interacting with the element. Once we've
clicked the button, we get a list of options. We use find_element() to find the selection we'd like to click,
and then we use click() to select the option.

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://designsystem.digital.gov/components/time-picker/")
#find our dropdown element
element = driver.find_element(By.ID, "appointment-time")
#click the dropdown
element.click()
#find our choice
choice = driver.find_element(By.ID, "appointment-time--list--option-2")
sleep(1)
#click the choice
choice.click()
sleep(1)
#close the browser
driver.quit()

Let's review the code block step by step:

After we opened the specified URL, we located the dropdown element with the ID "appointment-
time."

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 6/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Clicked on the dropdown to reveal the available choices.

Located the desired choice element within the dropdown with the ID "appointment-time--list--
option-2."
Introduced a 1-second delay using sleep(1) for visualization purposes.
Clicked on the chosen option.
Introduced another 1-second delay using sleep(1) .
Closed the Chrome browser instance using driver.quit() .

Methods Of Submitting Forms In Selenium

Just like filling a form manually, there are multiple ways to submit forms in Selenium. Each method
usually has its own best usecases and sometimes it comes down to personal preference.

We can submit forms in the following ways:

1. Method 1: Clicking the Submit Button

2. Method 2: Using the Submit Method on an Element
3. Method 3: Sending Enter Key to a Form Element
4. Method 4: JavaScript Execution

The best choice usually depends on the page that you're working with. Depending on what you're doing
it may be more efficient click() the submit button, press the Enter key, use the submit() method,
or to use JavaScript Execution to submit the information.

Most importantly, you need to think, "What would a human do on this page?", the answer to that
question is most often the method that you should use. This is why we're using Selenium to begin with,
we want our program to behave like a human.

Method 1: Clicking the Submit Button

As you've done throughout this tutorial, to use the submit button, simply use find_element() to find the
button you're looking for and use click() to click on it.

submit_button = driver.find_element(By.ID, "submit_button_id")

submit_button.click()

When to use this method?

Use this method when the form includes a clearly identifiable submit button, and interacting with
the button triggers the form submission.
Appropriate for standard HTML forms where the submit button is a standard input element with the
type "submit."
Suitable for cases where simulating user interactions is the primary objective.

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 7/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Method 2: Using the Submit Method on an Element

Selenium also allows you to call the submit() method on any form element, not just the submit button.
This method submits the form that contains the element.

form_element = driver.find_element(By.NAME, "some_form_element")

form_element.submit()

When to use this method?

Use this method when you have a direct reference to the form element, and calling the submit()
method on that element is a straightforward way to trigger submission.
Helpful in scenarios where submitting the form aligns with accessibility practices, as the submit()
method can trigger the default form submission behavior.

Sending Enter Key to a Form Element

You can also submit information by sending the Enter key to the submit button. It works the same way
it would when using the Enter key in your browser.

from selenium.webdriver.common.keys import Keys

input_field = driver.find_element(By.ID, "input_field_id")
input_field.send_keys(Keys.ENTER)

When to use this method?

Use this method when the form does not have a visible submit button, and pressing the Enter key
is a valid way to submit the form.
Appropriate for cases where the user typically submits the form by pressing Enter after typing in a
text input or textarea.

JavaScript Execution

Selenium also gives us the execute_script() method which allows us to insert JavaScript code directly
into our Python script.

from selenium import webdriver

from time import sleep
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the page
driver.get("https://quotes.toscrape.com/login")
#find our element
driver.execute_script("document.querySelector('.btn').click();")
sleep(1)
#close the browser
driver.quit()

In the example above, we do the following:

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 8/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Launch Chrome and navigate to the site

Pass native JavaScript, document.querySelector('.btn').click(); into our execute_script()
method to execute JavaScript arbitrarily on our webpage

When to use this method?

Use JavaScript execution when form submission involves complex logic, validations, or custom
event handling that may not be effectively triggered by standard Selenium methods.
Consider using JavaScript execution as a last resort when other methods fail or when dealing with
intricate web applications where standard methods may not suffice.

Uploading Files In Forms With Selenium

As you've probably needed to do before as a regular user, we regularly need to upload files to the web.
When uploading a file, we pass the absolute path to the file into the send_keys() method. Different
operating systems have different path layouts, but Python's os module gives us a really easy way to
work around that.

First, inside of the same folder as your scraper, create a new file, example.txt. and just add some
random text to it. Then save the file and add the following code to a Python script:

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
import os
#launch a browser instance
driver = webdriver.Chrome()
#navigate to the site
driver.get("https://designsystem.digital.gov/components/file-input/")
#find the file upload element
file_upload = driver.find_element(By.ID, "file-input-single")
#pass the absolute path to the file into the send_keys() method
file_upload.send_keys(os.path.abspath("./example_file.txt"))
#sleep so we can view the file upload
sleep(2)
#close the browser
driver.quit()

In the example above, we do the following:

Start the browser and navigate to the site with webdriver.Chrome() and driver.get()
Find the upload element with driver.find_element(By.ID, "file-input-single")
Once we've found our element, we use
file_upload.send_keys(os.path.abspath("./example_file.txt"))

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 9/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

The OS module gives us simple and universal way to find the absolute path to example.txt no matter
which operating system we''re using!

Dealing With Form CAPTCHAs In Selenium

To solve CAPTCHAS in Selenium, you can either do it manually or use a CAPTCHA solving service. The
code example below uses 2captcha. In order to use 2captcha, you need to sign up on their site and pay
for service. You can install the 2captcha-python module with the following command:

pip install 2captcha-python

Next, we need to identify the sitekey of our captcha. You can do this with your developer console and
inspecting the page. You can view an example of this below.

Once you've found your sitekey, we'll make a Python script that uses both our API_KEY and site_key
as variables.

Replace site_key and API_KEY in the code below:

from selenium import webdriver

from selenium.webdriver.common.by import By
from twocaptcha import TwoCaptcha
#save API_KEY
API_KEY = "YOUR-SUPER-SECRET-API-KEY"
#save site_key
site_key = "YOUR-CAPTCHA-SITEKEY"
#create a TwoCaptcha instance
solver = TwoCaptcha(API_KEY)
#open Chrome
driver = webdriver.Chrome()
#website url
website = "https://recaptcha-demo.appspot.com/recaptcha-v2-checkbox.php"
#navigate to site
driver.get(website)
#save a sscreenshot before solving
driver.save_screenshot("before_solve.png")
#get the response

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 10/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

response = solver.recaptcha(sitekey=site_key, url=website)

#save the response code
code = response["code"]
#find the response element on the page
response_element = driver.find_element(By.ID, "g-recaptcha-response")
#use JS to add the response code to the page
driver.execute_script(f'arguments[0].value="{code}";', response_element)
#find the submit button
submit = driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]')
#click the submit button
submit.click()
#take a screenshot after solving
driver.save_screenshot("after_solve.png")
driver.quit()

In the code above, we:

Save our site_key and API_KEY as variables

Create a TwoCaptcha instance with solver = TwoCaptcha(API_KEY)
Save our website url and navigate to it with driver.get(website)
Take a screenshot before solving with driver.save_screenshot("before_solve.png")
Get the response object with response = solver.recaptcha(sitekey=site_key, url=website)
Save response["code"] as a variable
Find our response element with response_element = driver.find_element(By.ID, "g-recaptcha-
response")
Insert our code into the response element with driver.execute_script(f'arguments[0].value="
{code}";', response_element)
Find the submit button with submit = driver.find_element(By.CSS_SELECTOR,
'button[type="submit"]')
Submit the form with submit.click()

Before solving the code, we take the following screenshot:

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 11/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

After we've solved, we get the following screenshot:

Troubleshooting

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 12/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

When automating forms with Selenium, developers face a myriad of issues they run into that will cause
problems because at the end of the day, we're trying to simulate a human, but we're using a pre-
defined script that only does what we tell it to do.

Issue 1: Element Not Found

If we search for an element in the current page and it isn't there, Selenium will throw an element not
found error. The proper way to address this is by waiting for the element to pop-up on the site so it can
be found. We also need to make sure that our locators are correct.

If you pass an incorrect locator to find_element() or find_elements() , Selenium will look for the wrong
element, so it won't find the element that you're actually searching for.

To properly address this, we can need to ensure that our locators are correct and then use a proper
waiting strategy to find the element.

The code below uses an implicit wait while we interact with elements:

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#define our url
url = "https://espn.com"
#open Chrome
driver = webdriver.Chrome()
#set implicit wait to 10 seconds
driver.implicitly_wait(10)
#navigate to the url
driver.get(url)
#find the search button
search_button = driver.find_element(By.ID, "global-search-trigger")
#click it so the search bar opens
search_button.click()
#find the search bar after waiting implicitly for it to open
search_bar = driver.find_element(By.ID, "global-search-input")
#send input to the search bar
search_bar.send_keys("detroit")
#pause so we can view the screen
sleep(5)
driver.quit()

In the code above, we do the following:

Define our url

Open a Chrome instance with webdriver.Chrome()
Set an implicit wait of 10 seconds for the session with driver.implicitly_wait(10)
Find the search button with driver.find_element()
click() the search

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 13/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Use driver.find_element() to find the search bar while implicitly waiting 10 seconds for it to appear
Enter "detroit" into the search bar with search_bar.send_keys("detroit")
Pause for 5 seconds using the sleep() function so we can view the screen

Issue #2: Element Not Interactable

Sometimes, Selenium can find an element but we can't interact with it. To address this, you need to
know how your website behaves and what actions allow us to interact with the element.

For instance, if we are clicking a submit button, but the submit button isn't clickable until we've filled all
the required fields, we need to make sure that all the fields are filled and that the submit button is
clickable.

In the previous code example, if we try to click() any of our search results, we get an error because
the element is not interactable. In order to make it interactable, we need to hover our mouse over it. We
do this with ActionChains .

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from time import sleep
#define our url
url = "https://espn.com"
#open Chrome
driver = webdriver.Chrome()
#set implicit wait to 10 seconds
driver.implicitly_wait(10)
#create an ActionChains object
action = ActionChains(driver)
#navigate to the url
driver.get(url)
#find the search button
search_button = driver.find_element(By.ID, "global-search-trigger")
#click it so the search bar opens
search_button.click()
#find the search bar after waiting implicitly for it to open
search_bar = driver.find_element(By.ID, "global-search-input")
#send input to the search bar
search_bar.send_keys("detroit")
#find the search results and return them as a list
results = driver.find_elements(By.CLASS_NAME, "search_results__label")
#iterate through the results
for result in results:
#if we've found the result we want
if result.text == "Detroit Lions":
#save the result
target = result
#exit the loop
break
#move the cursor to the target and click it

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 14/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

action.move_to_element(target)\
.click()\
.perform()
#pause so we can view the screen
sleep(5)
driver.quit()

The example above is almost the same as the previous one, but with a few key differences:

After sending input to search_bar , we use find_elements() to find all search result elements
We iterate through the results with a for loop until we find out target result
Once we've found our target , we move to it and click on it with
action.move_to_element(target).click().perform()

When dealing with ActionChains , things can get very unreadable very quickly. We can use \ to
continue our code on a new line.

Issue #3: Incorrect Element Interactions

Occasionally, when we click() a button or send_keys() to an element, it simply won't work correctly!
When investigating issues like this, best practice is to use JavaScript execution in order to interact with
the element.

JavaScript has builtin DOM (Document Object Model) support for interacting with and manipulating
webpages. If JavaScript can't interact with your element, it is most likely a problem with the site and not
your scraper.

Let's pretend our previous method with ActionChains failed. We can use execute_script() to execute
native JavaScript and take advatage of JavaScript's first class DOM support.

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#define our url
url = "https://espn.com"
#open Chrome
driver = webdriver.Chrome()
#set implicit wait to 10 seconds
driver.implicitly_wait(10)
#navigate to the url
driver.get(url)
#find the search button
search_button = driver.find_element(By.ID, "global-search-trigger")
#click it so the search bar opens
search_button.click()
#find the search bar after waiting implicitly for it to open
search_bar = driver.find_element(By.ID, "global-search-input")
#send input to the search bar
search_bar.send_keys("detroit")

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 15/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

#find the search results and return them as a list

results = driver.find_elements(By.CLASS_NAME, "search_results__label")
#iterate through the results
for result in results:
#if we've found the result we want
if result.text == "Detroit Lions":
#save the result
target = result
#exit the loop
break
#click the element using execute_script()
driver.execute_script("arguments[0].click()", target)
#pause so we can view the screen
sleep(5)
driver.quit()

Issue #4: Form Not Submitting

Sometimes we can fill a form out in its entirety, but it still won't submit. This actually happens quite often
when filling forms manually as well. In this case, first double check your scraper to make sure that your
fields are getting filled correctly.

In the event your fields are getting filled correctly, try different methods of submission that we went over
earlier in this article: Keys.ENTER , driver.execute_script() ...etc.

Let's use try and except to attempt different ways of clicking our result:

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from time import sleep
#define our url
url = "https://espn.com"
#open Chrome
driver = webdriver.Chrome()
#set implicit wait to 10 seconds
driver.implicitly_wait(10)
#create an ActionChains object
action = ActionChains(driver)
#navigate to the url
driver.get(url)
#find the search button
search_button = driver.find_element(By.ID, "global-search-trigger")
#click it so the search bar opens
search_button.click()
#find the search bar after waiting implicitly for it to open
search_bar = driver.find_element(By.ID, "global-search-input")
#send input to the search bar
search_bar.send_keys("detroit")

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 16/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

#find the search results and return them as a list

results = driver.find_elements(By.CLASS_NAME, "search_results__label")
#iterate through the results
for result in results:
#if we've found the result we want
if result.text == "Detroit Lions":
#save the result
target = result
#exit the loop
break
try:
target.send_keys()
except:
print("failed to send keys..scrolling to item!")
#move the cursor to the target and click it
action.move_to_element(target)\
.click()\
.perform()
#pause so we can view the screen
sleep(5)
driver.quit()

Once again our code example is basically the same, here are the differences:

First, we try to interact with target by using the enter key

If we fail when using the enter key, we handle the exception with an except statement that then
moves to the target and clicks on it

Issue #5: Handling Alerts and Pop-ups

We see pop-ups all the time on the web... especially when submitting forms. Selenium has a builtin alert
methods such as dismiss() and accept() but these methods typically don't work.

More often than not, you will need to know the proper locators on the pop-ups or modals because they
are often custom made and the developer wanted people to have to manually click to dismiss them.

It is best practice to locate your accept, dismiss, or close button using proper waiting techniques and
then click() the proper button to handle the modal.

from selenium import webdriver

from selenium.webdriver.common.by import By
from time import sleep
#open Chrome
driver = webdriver.Chrome()
#set implicit wait
driver.implicitly_wait(10)
#navigate to the url
driver.get("https://www.hubspot.com")
#find the "accept" button
modal = driver.find_element(By.ID, "hs-eu-confirmation-button")

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 17/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

#click the button

modal.click()
#pause so we can see whats happening
sleep(2)
driver.quit()

In the code above, we:

Open Chrome with webdriver.Chrome()

Set an implicit wait of 10 seconds
Navigate to the website with driver.get()
Find the "accept" button with driver.find_element()
click() the accept button

Issue #6: Dynamic Content Issues

As has been mentioned previously in this guide, quite often, we need to wait for items to properly
appear. Content that changes on the screen is called dynamic content. When dealing with dynamic
content we need to properly use either implicit or explicit waits for objects to appear.

When we use an implicit wait, we set a default timeout for any object to appear on the screen, as
long as the object appears within the waiting period, our code can interact with it
When using an explicit wait, we typically want to use it in combination with ExpectedConditions
so that webdriver can wait explicitly until the element appears on screen

The example below finds and closes the same pop-up we closed earlier, but this time we do it with an
explicit wait.

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
#open Chrome
driver = webdriver.Chrome()
#navigate to the url
driver.get("https://www.hubspot.com")
#find the "accept" button
modal = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "hs-eu-
confirmation-button")))
#click the button
modal.click()
#pause so we can see whats happening
sleep(2)
driver.quit()

The differences you should notice from implicit wait code we've used throughout this article:

WebDriverWait(driver, 10) sets an explicit wait of 10 seconds

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 18/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

until(EC.presence_of_element_located((By.ID, "hs-eu-confirmation-button"))) tells Selenium to

wait until our specific element has been located

Issue #7: Iframe Handling

While <iframe> elements aren't as commonly used as they once were do to security reasons,
sometimes they still get embedded into sites. To handle an iframe, we first use find_element() to find
our iframe element.

Once we've located it, we can use driver.switch_to.frame(name_of_iframe) to interact with that
specific element. To find elements from within the iframe, we can simply use find_element() or
find_elements() just like we would when interacting with the native webpage.

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
#open Chrome
driver = webdriver.Chrome()
#set implicit wait
driver.implicitly_wait(10)
#navigate to the site
driver.get("https://iframetester.com")
#find the search bar
url_search = driver.find_element(By.ID, "url-search")
#enter a url into the search bar to create an iframe
url_search.send_keys("https://quotes.toscrape.com")
#submit with the enter key
url_search.send_keys(Keys.ENTER)
#find the iframe
iframe = driver.find_element(By.ID, "iframe-window")
#switch to the frame
driver.switch_to.frame(iframe)
#find the first link
link = driver.find_element(By.TAG_NAME, "a")
#print the text of the link
print(link.text)
driver.quit()

The code above executes just like any other Selenium script that we've used previously in this article but
with one main difference:

driver.switch_to.frame(iframe) switches the focus of driver to the iframe

Issue #8: Browser Compatibility Issues

These issues are actually far more uncommon than they used to be. If you believe you might be having
compatibility issues, it is best to make sure that you have the latest browser driver installed. If
Chromedriver is up to date, and you're still running into issues with this, try Geckodriver and Firefox. If you
started out on Gecko with Firefox (and it is up to date) try with Chromedriver and Chrome.

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 19/20
3/31/24, 9:31 PM Python Selenium Guide - Submitting Forms with Selenium | ScrapeOps

Conclusion
You now know how to properly submit form information with Selenium. Feel free to explore your new
superpower in the open web an see what you can do! You can automate actual form filling, or you
could even build your own chatbot that runs on the send_keys() and submit methods that we
discussed in this article!

If you'd like to learn more about Selenium, take a look at the Selenium Documentation, they small code
examples for Selenium in many different programming languages such as Python, Java, C#, Ruby,
JavaScript, and Kotlin.

More Web Scraping Guides

Want to know more about scraping in general but not sure where to start? Take a look at some of the
ScrapeOps Selenium Web Scraping Playbook guides!

The playbook contains a plethora of useful material like these below:

The Python Selenium Guide - Web Scraping With Selenium

Using Proxies With Python Selenium
Selenium Undetected Chromedriver - Bypass Anti-Bots With Ease

https://scrapeops.io/selenium-web-scraping-playbook/python-selenium-submit-form/ 20/20