Selenium WebDriver - DevTools

Quiz

The latest version of Selenium Webdriver is on the version 4.x. The Selenium 4 comes with a lot of improvements which include the new APIs on Chrome DevTools(CDP). It gives more control over the browser where testing is to be performed.

What are the Chrome DevTools?

The Chrome DevTools is a collection of tools for the Chromium based browsers namely, Chrome, Edge, and Opera. They help to enable debugging and get additional information about the application under test. The advantages of CDP are listed below −

Obtain the Console Logs
Run and debug JavaScript
Mock Geolocations
Inspect web elements in Document Object Model(DOM)
Mock Network Traffic
Monitor Network Traffic
Update web elements along with its CSS

Chrome DevTool APIs in Selenium 4

The Selenium 4 is available with new Chrome DevTool APIs which are capable of the functionalities listed below −

Obtain and keep a watch on network traffic.
Mock Geolocations for localization testing.
Update Device mode to check responsiveness of web application.

The ChromiumDriver class is introduced from the Selenium 4 version. This class consists of the methods getDevTools() and executeCdpCommand(). They help to access the CDP. The getDevTools() method gives back the new DevTools object which allows us to use the send() method (the default Selenium commands for CDP).

These commands are primarily the wrapper methods which help to invoke the CDP functions. On the other hand, the executeCdpCommand() method helps to run the CDP methods with the help of the parameters. It is devoid of any wrapper APIs.

The ChromeDriver and EdgeDriver classes are inherited from the ChromiumDriver class. Thus the Selenium CDP APIs can be accessed from these drivers too.

Update Device Mode using CDP

Let us take an example where we would access the below application in other devices. The application can be configured to various dimensions to verify its responsiveness. The CDP command used to achieve this, Emulation.setDeviceMetricsOverride command. The minimum parameters to be passed in this command are height, width, mobile and deviceScaleFactor.

As an alternative we can use DevTools::send() method by taking the help of the setDeviceMetricsOverride() method. However, the setDeviceMetricsOverride() takes twelve arguments as parameters. Out of twelve, four parameters are mandatory, and eight are optional. For the options ones we have to use the method Optional.empty().

Example

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chromium.HasCdp;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;

public class UpdateDeviceCDP {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // device configurations
      Map dM = new HashMap(){
         {
            put("width", 500);
            put("height", 600);
            put("mobile", true);
            put("deviceScaleFactor", 50);
         }
      };
      ((HasCdp) driver).executeCdpCommand("Emulation.setDeviceMetricsOverride", dM);

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/selenium_automation_practice.php");
   }
}

It will show the following output −

In the above example, we observe that the screen size has become smaller having the specifications passed in the code.

Simulate Network Speed using CDP

Let us take an example where we would check the application behavior when the internet connection is on the 2G. The CDP command Network.emulateNetworkConditions is used to achieve this. The minimum five parameters that need to be passed in this command are offline, latency, download throughput, upload throughput, and CONNECTIONTYPE. The CONNECTIONTYPE can have the values as 3G, 2G, 4G, BLUETOOTH, WIFI, ETHERNET, and NONE. For the rest of the parameters we have to use the method Optional.empty().

Example

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.network.model.ConnectionType;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.devtools.v124.network.Network;
import java.util.Optional;
import java.util.concurrent.TimeUnit;

public class UpdateNetworkCDP {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // network 2G configurations
      d.send(Network.enable(Optional.empty(),Optional.empty(), Optional.empty()));
      d.send(Network.emulateNetworkConditions(
         false,
         50,
         30,
         40,
         Optional.of(ConnectionType.CELLULAR2G),
         Optional.empty(),
         Optional.empty(),
         Optional.empty()
      ));

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/selenium_automation_practice.php");
   }
}

In the above example, we have opened the application with a simulated network connection of 2G.

Emulate Geolocations using CDP

Let us take an example where we would emulate a geolocation using the Emulation.setGeolocationOverride command.

Example

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.emulation.Emulation;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.Optional;
import java.util.concurrent.TimeUnit;

public class UpdateGeolocations {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // update geolocations latitude and longitude
      d.send(Emulation.setGeolocationOverride(
         Optional.of(48.78232),
         Optional.of(9.17702),
         Optional.of(80)
      ));

      // open application url
      driver.get("https://where-am-i.org/");
   }
}

It will show the following output −

In the above example, we had overwritten the geolocations to Germany.

Obtain HTTP Requests using CDP

Let us take an example where we would obtain the HTTP requests while an application is launched along with its response, data, headers etc. To begin capturing network traffic, we would set the Network.enable which is used along with the send() method. Also, we would get the URL and the method name using the getRequest().getUrl() and getRequest().getMethod() methods respectively.

Example

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.network.Network;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.Optional;
import java.util.concurrent.TimeUnit;

public class ObtainHttpReq {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // get network traffic
      d.send(Network.enable(Optional.empty(),
      Optional.empty(), Optional.empty()));
      d.addListener(Network.requestWillBeSent(),
      e -> {
         System.out.println("Get Request URI: " + e.getRequest().getUrl()+ "\n"
            + "along with method: "+ e.getRequest().getMethod() + "\n");
         e.getRequest().getMethod();
      });

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/login.php");

      // quit browser
      driver.quit();
   }
}

It will show the following output −

Get Request URI:
https://www.tutorialspoint.com/selenium/practice/login.php

along with method: GET

Obtain Console Logs using CDP

Let us take an example where we would obtain the console logs. This helps for debugging, and root cause analysis of test failures. To begin capturing the console logs, we would set the Log.enable which is used along with the send() method. Also, we would get the log text and the log level using the getText() and getLevel() methods respectively.

Example

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.log.Log;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.concurrent.TimeUnit;

public class LogLevelCdp {
   public static void main(String[] args) throws InterruptedException {

      // Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // enable logging 
      d.send(Log.enable());

      // get log levels and text
      d.addListener(Log.entryAdded(),
         logEntry -> {
            System.out.println("Log text: "+logEntry.getText());
            System.out.println("Log level: "+logEntry.getLevel());
         });

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/login.php");

      // quit browser
      driver.quit();
   }
}

It will show the following output −

Log text: [DOM] Input elements should have autocomplete attributes
(suggested: "current-password"): 
(More info: https://www.chromium.org/developers/design-documents/create-amazing-password-forms) %o

Log level: verbose

Obtain Performance Metrics using CDP

Let us take an example where we would obtain the performance metrics of an application. To begin capturing the metrics, we would set the Performance.enable which is used along with the send() method. Also, we would get all the metrics using the Performance.getMetrics() method.

Example 1

package org.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.performance.Performance;
import org.openqa.selenium.devtools.v124.performance.model.Metric;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;

public class PerformanceMonitoringCdp {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();

      // enable performance monitoring
      d.send(Performance.enable(Optional.empty()));

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/text-box.php");

      // get performance
      List<Metric> m = d.send(Performance.getMetrics());
      List<String> mN = m.stream()
         .map(o -> o.getName())
         .collect(Collectors.toList());

      d.send(Performance.disable());

      List<String> metricsToCheck = Arrays.asList(
         "Timestamp", "Documents", "Frames", "JSEventListeners",
         "LayoutObjects", "MediaKeySessions", "Nodes",
         "Resources", "DomContentLoaded", "NavigationStart"
      );

      metricsToCheck.forEach( metric -> System.out.println(metric +
         " is : " + m.get(mN.indexOf(metric)).getValue()));

      // open application url
      driver.get("https://www.tutorialspoint.com/selenium/practice/login.php");

      // quit browser
      driver.quit();
   }
}

It will show the following output −

Timestamp is : 15204.440213
Documents is : 7
Frames is : 4
JSEventListeners is : 30
LayoutObjects is : 170
MediaKeySessions is : 0
Nodes is : 528
Resources is : 10
DomContentLoaded is : 15204.419683
NavigationStart is : 15203.25931

Process finished with exit code 0

Example 2

Let us take an example where we would perform basic authentication using the CDP command Network.setExtraHTTPHeaders which is used along with the send() method, and with the header data. It would help to authenticate and bypass any popup.

Let us take the example of the below page, on clicking the Basic Auth link, we would get a popup, asking for credentials.

The credentials admin is passed in both the fields Username and Password, then the Sign In button is clicked to proceed.

Finally, after successfully authentication, we would get a page with the text - Congratulations! You must have the proper credentials.

Code Implementation

package org.example;

import com.google.common.collect.ImmutableMap;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v124.network.Network;
import org.openqa.selenium.devtools.v124.network.model.Headers;
import org.openqa.selenium.edge.EdgeDriver;
import java.util.*;
import java.util.concurrent.TimeUnit;

public class BasicAuthCdp {
   public static void main(String[] args) throws InterruptedException {

      //Initiate the Webdriver
      WebDriver driver = new EdgeDriver();

      //adding implicit wait of 15 secs
      driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

      // instance of DevTools
      DevTools d = ((HasDevTools) driver).getDevTools();

      // create a session
      d.createSession();
      d.send(Network.enable(Optional.empty(),Optional.empty(), Optional.empty()));

      String encodedAuth = Base64.getEncoder().encodeToString("admin:admin".getBytes());
      Map<String, Object> headers = ImmutableMap.of("Authorization", "Basic " + encodedAuth);

      d.send(Network.setExtraHTTPHeaders(new Headers(headers)));

      // open application
      driver.get("https://the-internet.herokuapp.com/basic_auth");

      WebElement e = driver.findElement(By.tagName("p"));
      System.out.println("Text is: " + e.getText());

      // quit browser
      driver.quit();
   }
}

It will show the following output −

Text is: Congratulations! You must have the proper credentials.

Process finished with exit code 0

In the above example, we had passed the authentication in the header, hence had not encountered the popup while triggering the test.

Print Page