#ATAGTR2021 Presentation - "Selenium 4 Observability – a 90 Min Hands on Lab"
Download as PPTX, PDF0 likes146 views
The document outlines a lab presentation on Selenium Grid 4.0 observability, discussing its importance in distributed architecture, installation processes, and execution of tests. It highlights the integration of OpenTelemetry for tracing and logs collection, along with the setup of Jaeger for visualizing telemetry data. Key components include setting up a GCP machine with Docker and ensuring necessary software installations and configurations for effective grid observability.
#ATAGTR2021 Presentation - "Selenium 4 Observability – a 90 Min Hands on Lab"
- 1. SELENIUM 4.0 GRID OBSERVABILITY LAB #ATAGTR2021 DAY 2 12TH DECEMBER 2021 Aditya Garg
- 2. MY SOCIAL MEDIA HANDLES https://twitter.com/adigindia https://www.linkedin.com/in/adigarg/
- 3. WHAT WILL BE COVERED 1. Selenium Grid and importance of traceability in Distributed architecture 2. Understanding Observability components in Selenium 4.0 and installing coursier 3. Setting up a GCP machine with docker 4. Opening up necessary ports 5. Running Jaeger on the GCP machine 6. Setting up traceability in SeleniumGrid 4.0 7. Executing the tests and observing the trace
- 4. Have you worked on Java and Selenium before (yes or no) ⓘ Start presenting to display the poll results on this slide.
- 5. Have you worked in Selenium Grid before ⓘ Start presenting to display the poll results on this slide.
- 6. MORE QUESTIONS 1. How many of you would be participating in this Lab with me ? 2. Do you have your laptops ready ? 3. Do you have the following installed 1. JDK 1.8 2. Maven 4. Do you have PowerShell installed ? 5. Do you have your account with GCP ?
- 7. CHECKS CONTINUED 1. Double check on few things Follow along with me 1. Java check 2. Maven check 3. Selenium 4.0 latest Jar 4. Powershell check 5. GCP account https://docs.microsoft.com/en- us/powershell/scripting/install/installing-powershell-on- windows?view=powershell-7.2#msi https://www.selenium.dev/downloads/ https://ataevents.online/LabGuideATAGTR2021
- 8. SELENIUM GRID AND IMPORTANCE OF TRACEABILITY/OBSERVABILITY IN DISTRIBUTED ARCHITECTURE
- 9. Grid 3.X 1. Multiple nodes Ref: https://www.lambdatest.com/blog/selenium-grid-4-tutorial-for-distributed-testing/
- 10. Grid 4.0 Ref: https://www.browserstack.com/guide/selenium-grid-4-tutorial
- 11. What can go wrong Any of the process Any of the nodes • Imagine running your tests on a fully distributed architecture and things are not going as planned. • How do you trace or observe your request going through multiple processes and running on a node (specially if things are not working as planned) This is where observability comes in We can trace every request in a time series manner
- 12. OpenTelemetry Grid 4.0 is special in more than many ways 1. It is designed to work with modern infrastructures like docker and Kubernetes 2. The code is by default instrumented with OpenTelemetry
- 13. OpenTelemetry 1. Telemetry Data such as traces, metrics and logs 2. Popular Opensource Backends (which help us analyze or use this data) a. Jaeger b. Prometheus
- 14. Grid 4.0 is fully observable Grid 4.0 is special in more than many ways 1. It is designed to work with modern infrastructures like docker and Kubernetes 2. The code is by default instrumented with OpenTelemetry This is where the entire request responses telemetry data ((metrics, logs, and traces) can be instrumented, collected, and exported to help us analyze our Grid’s performance and behavior. Selenium 4 Grid is thus fully observable – it helps us troubleshoot issues, optimize performance, and provide visibility into the system
- 15. Selenium Grid 4 Observability components 1 Open Telemetry which is instrumented in the Selenium 4.0 Jar files or code itself. • This helps in tracing every requests lifecycle • Trace contains details about the information called as spans • Span records time logs – called as events • All these are nothing but some form of logs – which we were already used to see on a command prompt. • This can be configured easily to send all the data to a backend data consolidator – Jaeger
- 16. Selenium Grid 4 Observability components • Since Console logs becomes too tedious to query and analyze • We need a mechanism for visualizing and querying traces • Jaeger seamlessly integrates with OpenTelemetry to provide a rich experience of querying, visualizing and collecting request traces https://www.jaegertracing.io/ 2 The second player in the Selenium 4.0 Observability is Jaeger
- 17. LET US GET INTO SOME ACTION 1 2 3 Using Selenium 4.0 Jar Installing coursier – which is used for updating the classpath for tracer (open telemetery) Running Jaeger We can install using binaries or run docker through docker images
- 18. Selenium Grid 4 Observability Setup java -jar selenium-server-4.1.0.jar info tracing This example uses a tool called "coursier" to generate a full classpath, but you can also write this manually. When started this way, the selenium server will inform you that it has found a tracer on stdout. 1
- 19. Selenium Grid 4 Observability Setup 2 Installing coursier – which is used for updating the classpath for tracer (open telemetery) https://get-coursier.io/docs/overview#coursier https://get-coursier.io/docs/cli-installation
- 21. Selenium Grid 4 Observability Setup 3 Running Jaeger We can install using binaries or run docker through docker images https://www.jaegertracing.io/ docker run -d --name jaeger -p 16686:16686 -p 14250:14250 jaegertracing/all-in-one:latest
- 22. Selenium Grid 4 Observability Setup Do you have docker installed on your machine ?
- 23. Selenium Grid 4 Observability Setup https://console.cloud.google.com 1. We will create a VM with pre- installed docker 2. Open required ports 3. Install and run Jaeger 4. If you already have docker installed, you do not need these steps.
- 24. Docker Engine Community on Ubuntu 18.04 LTS
- 25. Selenium Grid 4 Observability Setup docker run -d --name jaeger -p 16686:16686 -p 14250:14250 jaegertracing/all-in-one:latest We will run our docker VM in a GCP Open ports 16686 and also 14250 for our telemetry needs Run the Jaeger image
- 27. 0.0.0.0/0
- 28. Selenium Grid 4 Observability Setup 2 Running Jaeger We can install using binaries or run docker through docker images https://www.jaegertracing.io/ docker run -d --name jaeger -p 16686:16686 -p 14250:14250 jaegertracing/all-in-one:latest
- 29. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid
- 30. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-event-bus" -jar selenium-server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) event-bus a Start Event Bus – connect to Jaeger and enable opentelemetry using – ext option using coursier
- 31. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-sessions" -jar selenium-server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) sessions b Start sessionMap – connect to Jaeger and enable opentelemetry using –ext option using coursier
- 32. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-session-queue" -jar selenium-server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) sessionqueue c Start sessionQueue – connect to Jaeger and enable opentelemetry using –ext option using coursier
- 33. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-distributor" -jar selenium-server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) distributor --sessions http://localhost:5556 --sessionqueue http://localhost:5559 --bind-bus false d Start distributor – connect to Jaeger and enable opentelemetry using –ext option using coursier
- 34. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-router" -jar selenium- server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) router --sessions http://localhost:5556 --sessionqueue http://localhost:5559 --distributor http://localhost:5553 e Start router – connect to Jaeger and enable opentelemetry using – ext option using coursier
- 35. Selenium Grid 4 Observability Setup 3 Running Selenium Grid with OpenTelemetery Java extension classes connected to the running instance of Jaeger We will be running distributed grid java -D"otel.traces.exporter=jaeger" - D"otel.exporter.jaeger.endpoint=http://164.52.192.223:14250" - D"otel.resource.attributes=service.name=selenium-node" -jar selenium- server-4.0.0-rc-2.jar --ext $(cs fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.0.0 io.grpc:grpc- netty:1.35.0) node f Start node – connect to Jaeger and enable opentelemetry using – ext option using coursier
- 38. Selenium Grid 4 Observability Setup 4 Add the necessary telemetry dependencies in selenium client and run the code a Add telemetry related dependencies in POM file https://gitlab.com/adigindia/atagtr2021selenium4observability
- 39. Selenium Grid 4 Observability Setup <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-exporter-jaeger</artifactId> <version>1.0.0</version> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-netty</artifactId> <version>1.35.0</version> </dependency> <!-- https://mvnrepository.com/artifact/com.google.guava/guava --> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>31.0.1-jre</version> </dependency> <!-- https://mvnrepository.com/artifact/io.netty/netty-handler --> <dependency> <groupId>io.netty</groupId> <artifactId>netty-handler</artifactId> <version>4.1.68.Final</version> </dependency> a Add telemetry related dependencies in POM file 4 Add the necessary telemetry dependencies in selenium client and run the code
- 40. Selenium Grid 4 Observability Setup 4 Add the necessary telemetry dependencies in selenium client and run the code b Add system properties in selenium client code System.setProperty("otel.traces.exporter", "jaeger"); System.setProperty("otel.exporter.jaeger.endpoint", "http://164.52.192.223:14250"); System.setProperty("otel.resource.attributes", "service.name=selenium-java-client"); ImmutableCapabilities capabilities = new ImmutableCapabilities("browserName", "chrome"); driver = new RemoteWebDriver(new URL(hubURL), capabilities); String hubURL = "http://localhost:4444";
- 41. RUN THE CODE AND OBSERVE
- 44. BRING NODE DOWN AND OBSERVE ON JAEGER
- 47. Key references 1. https://www.selenium.dev/blog/2021/selenium-4-observability/ 2. https://www.selenium.dev/documentation/webdriver/remote_webdriver/ 3. https://github.com/manoj9788/tracing-selenium-grid 4. https://get-coursier.io/docs/cli-installation 5. https://www.ibm.com/docs/en/sdk-java-technology/7.1?topic=jjc-specifying-java- options-system-properties-2 6. https://github.com/SeleniumHQ/selenium/blob/selenium-4.0.0-beta- 1/java/server/src/org/openqa/selenium/grid/commands/tracing.txt 7. https://opentelemetry.io/docs/java/ 8. https://www.lambdatest.com/blog/selenium-grid-4-tutorial-for-distributed-testing/ 9. https://www.browserstack.com/guide/selenium-grid-4-tutorial 10.https://docs.docker.com/compose/environment-variables/ 11.https://github.com/SeleniumHQ/docker-selenium