Neoload

Introduction
• The Objectives of this Training
Load-Testing Principles
Design
Runtime
Results
For Advanced Users
AGENDA

What You Are Going To Learn
At the end of the class, you will know how to use NeoLoad
Introduction
Many tools to design
your VU
Some methodology
How to make your
own analysis

Introduction
• Definition
• Test Types
• Methodology
Design
Runtime
Results
For Advanced Users
AGENDA

Determine how a system performs in
terms of responsiveness and
stability under a particular workload
Definition
Web Load-Testing Principles

Why Do Load Testing?
If your website earns $100.000 a day, this year you could lose $2.5 MILLION
in sales.
16%
decrease in
customer
satisfaction
11% fewer
page views
7% loss in
conversions
A 1 second
delay in page
load time

Load-Testing Allows Us To
Predict
application
performance
Detect
bottlenecks
Determine
scalability

Several Tests
Load Test
 determine the behaviour of the system under a specific expected load
Scalability / Capacity
 determine the system’s maximum load capacity
Stress
 subject the application to a much greater activity than it normally handles
Endurance / Soak
 detect memory leaks and other resource consumption issues using long
tests and static load
Degraded
 determine the behaviour of the system under non-optimal conditions

Load-Testing Methodology
Test plan
Load-Testing
Consultant
Project manager
Functional
specialists
• Objectives
• Test platform
• Scenarios
• Scheduling
• Contributors
Preparation
Load-Testing
Consultant
Project manager
Functional
specialists
• Testing environment
installation
• Test platform validation
• Test plan validation
Design
Load-Testing
Consultant
Functional
specialists
Developers
• Test data generation
• VU profile design
• DB update procedure
• Test data validation
• Population & monitoring
Runtime
Load-Testing
Consultant
Server
specialists
Administrators
• Single-user baseline
• Ramp-up load test
• Combination test
• Maximum load test
Tuning
Validation
Analysis
Load-Testing
Consultant
Server
specialists
Administrators
• Response time analysis
• Benchmark comparison
• System metric analysis
• Tuning if required
• Report creation

NeoLoad Architecture
Monitor
(optional)
NeoLoad Monitoring Engine
3G+H+ WiFi4G+
Controller
Command
Cloud Load Generators On-premise Load Generators
Load
APP
Application Server Database ServerWebServer
• NeoLoad management console
• Design, run and analyze tests
locally
• Manage LG and MA
• Load generation as defined in
the controller
• Only active when the test is
launched
• Retrieve system information
• Understand the bottlenecks
• Alerts for faster analysis
• Network emulation
• Device emulation
• System to be tested

The Basics of A NeoLoad Project
Runtime
Set the number of users,
start the test and look at
the real-time information
Record and design VU
profiles
Create a population and
set each VU profile rate
Design
Setup the monitors
Analyze the results and
issue a report
Results

Introduction
Design
 Recording a Virtual User Profile
• Designing a Virtual User Profile
• Building a Population
• Monitoring the Infrastructure
Runtime
Results
For Advanced Users
AGENDA

How Does NeoLoad Record?
Recording a Virtual User Profile
The client
(web-browser)
Servers
Requests are
recorded
NeoLoad acts as a proxy
and listens on port 8090
Responses
are recorded

Starting the Recorder
• Use the button and fill-in the following pop-up.
Name of the
VU
Record in Init
or Actions
Start and set
the browser
Choose the
protocol to record
Choose the
recording mode
Set the user-agent

During the Record
• Use the recording bar to modify the user.
Stop or pause
the record
Change the
section
Create
containers
Set Rendezvous
points
Access the
documentation

Why Using Containers?
• Group actions together
• Organize VU’s for easier maintenance
• Extract statistics (average times, hits/s…)
Without
With

Objectives
After completing this exercise, you will be able to:
• Use the Neotys training environment.
• Record a simple VU profile.
• Recognize the components of a VU profile.
Scenario
1. Open the Neotys Training Applications portal and access the Jpetstore.
2. Record a VU profile, using containers, called ‘Browser’ that only browses the shop.
3. Stop the record and click on Finish to not execute the post-recording wizard.
4. Navigate into the VU profile to see the different components.
Exercise 1

The VU Profile at First Sight
Actions
container
Request
Page
Transaction

The VU Profile Structure
• Init
- Executed once, and only once, at the beginning of the test.
- Common uses: initialization routines, application log in, etc.
• Actions
- Repeatedly executed until the end of the test.
- Cookies and cache are preserved between iterations (default behaviour).
- Reset the user context from the runtime parameters.
• End
- Executed once at the end of the test.
- Common uses: application log out, etc.

The Runtime Parameters of the VU Profile
• A VU instance keeps running when an error occurs. It may be changed to:
- Go to the next iteration.
- Stop and start a new VU.
• Behaviour on assertions may be set the same way.
• All think times may be modified:
- Use think time defined on page.
- Override the think times.
- Apply a factor for each page.
- Add a random delay.

VU Profile: Pages
• NeoLoad representation of an internet page.
• Made of an HTML page and the embedded resources (GIF, PNG, CSS, JS…).
• May be renamed.
A page
The resources

VU Profile: HTTP Requests
• NeoLoad representation of a request sent over the network (HTTP/S).
Method
Server
URL
Path
Parameters

The End of the Record
• When the navigation is completed, finish the record either by:
- Clicking the stop button .
- Closing the web-browser.
• If the recorded application keeps sending data, this pop-up may be displayed.

The End of the Record
• Go through the post-recording wizard.
• It helps taking care of the:
- Think-time.
- Dynamic parameters: framework (pre-defined rules) and dynamic (comparison based
correlation).
- Authentication.

Adding Pages To A Record
• You can also start the recorder within an existing VU Profile.
• Right-click in the VU Profile and choose ‘Record here’.

Objectives
• Record a structured VU profile.
• Use the flags to find an information.
Scenario
1. Record a VU profile, using the containers, called ‘Searcher’ that:
a. Starts at the Jpetstore homepage
b. Enters the store
c. Enters ‘Amazon Parrot’ in the search field
d. Clicks on a product
e. Clicks on an item
2. Stop the NeoLoad recorder and do not execute the post-recording wizard. When
prompted to execute click on Finish.
3. Locate the chosen product and item.
Exercise 2

Introduction
Design
• Recording a Virtual User Profile
 Designing a Virtual User Profile
Runtime
Results
For Advanced Users
AGENDA

Why?
• Fix the issues due to dynamic parameters.
• Reach realistic behaviours.
• Define several types of VU profile.
Designing a Virtual User Profile

Design Methodology
This sequence may be reordered. However, do not forget any step!
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn
Validating the VU
behaviour
• Play a unit test
• Check the errors
Correcting the VU
behaviour
• Identify the request
that returns an error
• Identify the parameter
• Handle the data
Changing behaviour
using logical actions
• Adapt to the simulation
requirements
Using different login
accounts and values
• Define realistic
conditions of use
Validating key pages
• Set validations to make
sure responses are the
ones expected

Design Methodology
Correcting the VU
behaviour
Changing behaviour
accounts and values
Validating the VU
behaviour
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn

Why?
• Find any issue that may be in the recorded VU profile.
• The recorded pages may require tuning.
• Modern apps exchange a lot of parameters.
• Avoid bad surprises while the test is running.
Validating the VU Behaviour

How?
• Using the ‘Check Virtual User’ (NeoLoad debugger).
• A single VU test with no thinktime.
• Accessed via .

The Check VU in Details
• Comparisons with recording.
• Difference rate.
• Variables.
• Requests and responses.
• Failed assertions.
• HTML page rendering.

Objectives
• Check whether a VU profile is valid.
• Dig into the details of the page source code.
Scenario
1. Check the ‘Searcher’ VU profile and make sure it runs properly.
2. Examine the information provided, including:
- The HTML rendering.
- The requests that comprise the application home page.
- The Comparison of the recorded and replayed responses.
3. Locate the chosen product and item.
Exercise 3

Design Methodology
Validating the VU
behaviour
Changing behaviour
accounts and values
Correcting the VU
behaviour
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn

Why?
• Because a VU displays an unexpected error.
• Because HTML pages usually contain dynamic elements.
• Because NeoLoad cannot automatically handle all parameters.
Correcting the Virtual User Behaviour

Focus: The HTTP Parameters
• Static parameters
- Values never change, regardless of
which user is connected or the number
of times the VU profile is played.
• Functional dynamic parameters
- Entered by the end user.
- Making them dynamic makes the VU
profile more realistic.
• Technical dynamic parameters
- Generated by the server, often used by
the client in subsequent requests.
- Values change at each call.
Must be handled to have a valid
VU profile.

Objectives
• Understand how technical parameters may influence a VU profile.
Scenario
1. Record a VU profile called ‘Buyer’ that:
a. Starts at the Jpetstore homepage and then enters the store
b. Logs into the store (j2ee/j2ee)
c. Selects a category by clicking one of the images beneath the center bird
d. Adds a pet to the shopping cart
e. Checks out and enters payment information
f. Confirms the sale
g. Logs out
2. Stop the NeoLoad recorder and do not search for the dynamic parameters in the post-
recording wizard.
3. Check the ‘Buyer’.
4. Explain why it fails.
Exercise 4

How?
• Using a mecanism called variable extractor which extracts content of an HTTP response.
• The content is assigned to a NeoLoad variable.
• A variable may be called using the syntax ${<variableName>} or with the variable
picker that gives a quick access to the existing variables.
• The variable extractor is accessed from the request advanced settings .
• The variables created with the variable extractor are only accessible by the VU profile in
which they are extracted.

The Variable Extractor
Variable name
Target
extraction
Occurrence to
extract
Encoding
Test
XPath definition
Start boundary
Groups
End boundary

Hypothesis: a request fails to pass the Check VU
Make the replacement
Select the failed request
Validate the variable extractor
Go to the variable extractor settings and configure the extraction
Select the response containing the data to be extracted
Procedure: the Variable Extractor

Objectives
• Use the NeoLoad tools to properly take into account a dynamic paramater.
Scenario
1. From the observations made on the previous exercise, fix the VU profile using a variable
extractor.
The extraction must be set on a request before the one that fails.
2. Use the Check-VU function to make the VU profile works as expected.
Exercise 5

The Variable Extractor: Automatic Configuration
• Automation of the manual procedure.
• Avoid mistakes.
• Save time.
Select the automatic
configuration
All is automatically
setup

Objectives
• Take advantage of the automatic correlation feature.
Scenario
1. Duplicate the ‘Buyer’ and call the new user ‘Buyer_AutoConfig’.
2. Restore the value of the id parameter in the ‘Buyer_AutoConfig’.
3. Use the automatic configuration to fix the issue.
4. Check the VU to be sure the issue has been handled.
Exercise 6

The Framework Parameters: Practical Work
1. Record a VU profile called ‘BuySeveralItems’
that:
a. Starts at the Jpetstore homepage and then
enters the store
b. Logs into the store (j2ee/j2ee)
c. Selects a category by clicking one of the
images beneath the center bird
d. Adds a pet to the shopping cart
e. Clicks on “Main Menu”
f. Repeats steps c. to e. 5 times
g. Checks out and enters payment information
h. Confirms the sale
i. Logs out
2. Run a check-VU and observe the result

How should we handle the id
parameter?
Use the search and replace?
Do all extractions manually?
The Framework Parameters

The Framework Parameters: Why?
• Parameters are often the same across the scenario.
• “Teach” NeoLoad new correlations rules and save time.
• Re-use your framework parameters for future records.

The Framework Parameters: How?
• NeoLoad provides predefined correlation rules (.NET, Oracle, GWT, Siebel, Flex,
Silverlight…).
• For your own apps, create your own framework parameters.
Move the extraction
to the framework
Move the extraction
to the framework

The Framework Parameters: Advanced Settings
• Only edit existing configurations. Do not create a framework parameter from scratch.
Name
Replacement
Extraction rule

The Framework Parameters: Tips
• Framework parameters are part of the NeoLoad installation. Exporting a project does not
include them.
• Framework parameters must be exported to be shared (advanced settings, NeoLoad
preferences).
• Search for the framework parameters via .

Objectives
• Take advantage of the framework parameters for automatic correlation.
Scenario
1. Record a ‘SearchBooking’ user on the Hotel Booking Application:
a. Access the home page.
b. Start (click on ‘Start your Spring Travel Experience’).
c. Click on ‘Find Hotels’.
d. View the first hotel in the list.
e. Close your browser.
2. Do not execute the post recording wizard. Check the VU, and observe the results.
3. Define extractors and add them to a ‘HotelBooking’ framework for the following
parameters:
- dtid,
- execution,
- the first uuid_0 (rename it ‘uuid_Find’ when you add it to the framework)
- the second uuid_0 (rename it ‘uuid_View’ when you add it to the framework)
Exercise 7 (1/2)

4. Apply the framework parameters to the ‘SearchBooking’ user. Check it and observe the
results.
5. Record a ‘Logged_SearchBooking’ user following these steps:
a. Access the home page.
b. Login (credentials are provided on the page).
c. Start (click on ‘Start your Spring Travel Experience’).
d. Click on ‘Find Hotels’.
e. View the first hotel in the list.
f. Logout.
g. Close your browser.
6. Apply the framework parameters to the ‘Logged_SearchBooking’ user. Check it, and
observe the results.
Exercise 7 (2/2)

Generic Dynamic Parameters vs. Framework Parameters
• Generic Dynamic Parameters
- Found ‘on the fly’ using comparison.
- Less accurate identification.
- Slower identification.
• Framework Dynamic Parameters
- Defined using regular expressions.
- More accurate identification.
- Faster identification.

The Variable Extractor: Multiple Occurrences
• If an extraction template has several matches, all occurrences may be returned as a set.
• The following variables are created:
- <variableName>_matchNr : number of occurrences found.
- <variableName>_n : nth occurrence (n=1, 2...).
- <variableName>_rand : random occurrence.
- <variableName> : default value.
• Unless for specific purpose, when doing multiple extractions, do not use the only name of
the variable.

Design Methodology
Validating the VU
behaviour
accounts and values
Correcting the VU
behaviour
Changing behaviour
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn

Why?
• To simulate more realistic conditions.
• To make simpler records.
• To have new VU profiles by only modifying one already recorded.
Changing the Virtual User Behaviour

How?
• Using the logical actions.
• Adapt to the simulation requirements with:
- Loops.
- Conditional branching.
- Error management…
• Drag and drop the actions. No script.

Container
• Group together certain actions and statistics.
• Two execution modes:
- Sequentially: the elements of the container are played
in the order they are declared.
- Randomly: the elements of the container are played
randomly.
• The advanced mode allows to configure each element to
be repeated a number of times (repetition or probability).
• Pacing may be set on each container.
It is the minimum time for the container to be executed.

Delay
• Suspends the execution for a specified duration.
• Expressed in milliseconds.
• May be a variable.
Loop
• Repeat elements.
• The number of iterations may be constant or a variable.
• The variable <LoopName>_counter is automatically created. It gives the current
iteration.

While
• Execution of elements while a condition remains true.
• Several conditions may be applied.
• The variable <WhileName>_counter is automatically created. It gives the current
iteration.
If…Then…Else
• Execution of conditional actions.
- If condition is true, Then folder is executed
- If condition is false, Else folder is executed
• Several conditions may be applied to an If…Then…Else action.

Objectives
• Modify the sequence of a VU profile using the GUI.
• Change the VU profile to fullfill specific conditions.
• Use the ‘Update Recorded Content’ function.
Scenario
1. Duplicate the ‘Buyer’ and call it ‘LoopBuyer’.
2. Modify the ‘LoopBuyer’ in order to purchase a pet from each category (several pets in
the shopping cart and only one order):
- Set up multiple extractions.
- Use a Loop.
- Variables may be concatenated following the syntax ${var1${var2}}.
- Automatic variable extractions cannot be used.
3. Update the recorded content of the ‘LoopBuyer’ (advanced settings of the Check-VU).
4. Duplicate the previous VU profile and call it ‘WhileBuyer’.
5. Modify the ‘WhileBuyer’ using a While to include a condition on the total price (for
example, less than 500 dollars).
Exercise 8

Try…Catch
• Adapt the test execution behaviour in function of events (error management…).
• Actions included in the Try are executed sequentially.
• If an assertion or an error is detected then the Catch is executed.

Go to Next Iteration
• Interrupt the VU runtime and go to the next iteration.
• The behaviour depends on where the action is:
- Init: the VU stops its initialization and goes to the Actions container.
- Actions: the VU stops its current iteration and run a new Actions container or goes to the End
container (depends on the load policy).
- End: no interruption, the VU keeps executing the End and an error message is logged.
• Often combined with If...Then...Else or Try...Catch.

Shared Containers
• Share elements to re-use them in several VU profiles (login or logout for instance).
• Statistics are given across several VUs.
• Shareable elements are:
- Container
- Loop
- While
- Fork
• An element is shared with a drag-and-drop or a right-click interaction.

Design Methodology
Validating the VU
behaviour
Correcting the VU
behaviour
Changing behaviour
accounts and values
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn

Why?
• Test all branches of the application.
• Test how the servers handle the cache.
• Make things more realistic.
Using Different Values

How: the Variable Manager
• Numerous variable types (lists, strings, integers, counters…).
• Change policy.
• Distribution policy.
The variables created through the
variable manager are accessible by
all the VU profiles.

Focus: the SQL Variable
• The SQL variable embeds a guided mode for MySQL, Oracle, DB2, PostgreSQL and MS
SQL.
• When requesting other DB, the driver and the URL must be manually defined.
• The SQL variable is populated and sent to the LGs before the test is running. It is not
calculated in real-time.
• Use the syntax ${<variableName>.<columnName>} to access each column of the
table.

Objectives
• Create and use the NeoLoad variables.
Scenario
1. Make the authentication of the ‘Buyer’ dynamic with a File variable (accounts are
provided on the Neotys Application Portal). Call this variable ‘Accounts’.
2. Create a List variable called ‘Address’ with three columns called ‘City’, ‘State’ and ‘ZIP’
that contains four data sets (rows):
- Boston, MA, 02134
- Random Lake, WI, 53075
- Beverly Hills, CA, 90210
- Echo, Oregon, 97826
3. Data values for the variable are selected in a random order.
4. In the ‘Buyer’, find the request submitting the payment details and replace the hard-
coded values with the variable ‘Address’.
5. Create (without using it) an SQL variable for the authentication called ‘SqlAuthent’ (DB:
jpetstore, table: SIGNON).
Exercise 9

Design Methodology
Validating the VU
behaviour
Correcting the VU
behaviour
Changing behaviour
accounts and values
VU profile
ready
The check VU
The variable
extractor
The logical
actions
The NeoLoad
variables
The validation
You will
learn

Validation
• Check the server responses in real-time to ensure that the pages delivered by the server
are what you expect.
• Three criteria:
- Delay.
- Page size.
- Page content.
Validating Key Pages

Global Validation
• Add content assertions to a container or the entire VU profile.
• Only responses with a text/html or text/xhtml content-type are checked (default
behaviour).
Validating Key Pages

Objectives
• Prepare the VU profile for the test by implementing error management.
Scenario
1. In the ‘Searcher’, change the search criteria using a List variable containing:
- Goldfish
- Amazon Parrot
- Iguana
- Manx
2. Set a validation to check that the ‘Add to Cart’ button is in the page.
3. Use a Try…Catch to stop the VU iteration if an assertion fails.
4. Set random think times between 5 and 10 seconds.
5. Add a Loop to include several searches (between 1 and 4). Change the search criteria
value on each loop iteration.
6. Stop the VU iteration if the stock of an item falls below 9500.
Exercise 10

Introduction
Design
 Building a Population
Runtime
Results
For Advanced Users
AGENDA

Why?
• Test using different business behaviours.
• Test with specific network conditions.
• For example, test a web store by simulating 75% of users browsing the catalog (visitors),
20% carrying out a purchase through to its end (buyers) and 5% being administrators.
Building a Population

How?
Browser
selection
Populations
WAN
Emulation
Cache policy

Focus: Browser Picker
Simulate desktop, mobile
or tablet browsers

WAN Emulation
• All networks do not share the same characteristics.
• Typical characteristics are:
- Signal strength (for wireless networks)
- Bandwidth
- Latency
- Packets dropped.
• WAN emulation allows realistic network conditions.

WAN Emulation
• Settings available in the populations tab.
• Not configured by default (unlimited bandwidth, no packet loss, no latency).
• Pre-configured profiles carry real-world values.
• A LG may simulate several network conditions.
WAN emulation
profiles
WAN emulation
parameters

Objectives
• Create a complex population.
Scenario
The population ‘PetstorePopulation’ is made of:
1. 60% of ‘Searcher’.
2. 40% of ‘Buyer’.
3. The ‘Searcher’ users do not use a cache; the other users have a full cache at the
beginning of the iteration.
4. 50% of the ’Buyer’ users have an iPad and a 3G connection with a poor signal.
5. The rest of the ‘Buyer’ users have Chrome and a Wi-Fi connection with a medium
signal.
6. 10 % of the ‘Searcher’ users have a custom browser (called ‘Proprio’) that uses 6
simultaneous HTTP connections.
Exercise 11

Introduction
Design
 Monitoring the Infrastructure
Runtime
Results
For Advanced Users
AGENDA

Why?
• Without monitor, NeoLoad only retrieves the response times, the errors, the number of
hits and the throughput.
• Information from the server side is fundamental to perform an efficient analysis.
• Each layer of the architecture may be monitored (webservers, application servers,
database, network…).
Monitoring the Infrastructure

How?
• No installation on the server side.
• All actions are performed from the NeoLoad controller.
• Specific ports need to be open on the firewall.
• NeoLoad provides preconfigured counters.

The Configuration
Choose the layers
Set the
authentication
Choose the
counters

Monitoring Agent
• Simplified controller to agent communication.
• Single port (TCP 7200) to open in the controller to agent direction.
• Installed and started as separate process on the server side.
• MA and LG share the same color
code:
- Green: running
- Orange: different version
- Red: halted

Objectives
• Use the NeoLoad monitoring capabilities.
• Look at the architecture behaviour in real-time.
Scenario
1. Add a Tomcat 6 monitor (no authentication needed).
2. Add a MySQL monitor (authentication: monitor / no password).
3. Add a Windows monitor (localhost).
Exercise 12

The Thresholds: Why?
• Trigger real-time alerts during test runtime on performance counters.
- If CPU usage exceeds 80%
- If available memory falls below 15%
- …
• Highlight components that suffer a deterioration in performance during the load test.
• Alerts make understanding the causes of any bottleneck much easier.

The Thresholds: How?
• Alerts are set on some counters by default.
• Alerts are identified by .
• Alert thresholds may be manually added.
• An alert may be of two sorts: warning or critical.

Objectives
• Set thresholds and alerts.
Scenario
1. Based on the previous exercise, set an alert on the Windows monitor that follows the
condition: if the percentage of memory used is greater than 98%, raise up a critical alert.
2. Set an alert on the Tomcat monitor that follows the condition: if the
currentThreadCount(HTTP) exceeds 20, raise up a warning alert.
Exercise 13

Introduction
Design
Runtime
• Running the Test
Results
For Advanced Users
AGENDA

Setup and Watch
Running the Test
Test
completed
The runtime
settings
The real-time
analysis
You will
learn
Configuring the test
scenario
● The duration
● The load policy
● The LGs
● Advanced settings
Looking at the real-
time information
● Plot your graphs
● Correlate the data
● Check the errors
● Follow the VUs

Setup and Watch
Running the Test
time information
Test
completed
The runtime
settings
The real-time
analysis
scenario
You will
learn

Definition
• The test configuration.
• Several test scenarios may be created (short ramp-up test, long peak test…).
• All settings required are defined and applied to a population:
- Test duration.
- Load policy.
- Load generators.
- Runtime policy.
Configuring the Test Scenario

Duration Policy
• Unlimited.
• Iterations.
• Time.
• Iteration mode:
- Each user is run for a predetermined number of iterations with no break between iterations.
- Towards the end of the test, some users are finishing their iterations and some others are still
active. As a consequence the load decreases until all the users are finished.
- For ramp-up or peak loads, adding users for a new phase takes place once all existing users
have finished their iterations (the load may drop to 0).

Load Policy
• Constant: generate a fixed number of VU.
• Ramp-up: generate a number of VU that increases throughout the test.
• Peak: generate a fixed number of VU with periodic phases of high load.
• Custom: set the load to be applied by plotting the VU variation curve.

Load Generators
• LGs may be manually added by entering:
- The machine's name.
- The IP address.
• LGs may be automatically discovered.
• Define from the advanced settings:
- The network interface card.
- The IP spoofing.
- The load factor.
- The upgrade.

The Cloud Load Generators: Why?
• Perform heavy load tests without hosting the necessary infrastructure.
• Load test applications from several geos.
• Test the entire delivery chain.
• Benefits of the hybrid approach: cloud LGs / on premise LGs.

The Cloud Load Generators: How?
Open the Cloud console
and choose an on-demand
or reserved session

The Cloud Load Generators: On-Demand Session
Choose the duration and the
geos for an immediate use

The Cloud Load Generators: Reserved Session
Book your session
online and use it later

The Cloud Load Generators: Managing the Session
Cloud and hosted LGs
are used the same way
The session is managed
from the console

Population Parameters
How VUs start
Start policy
Stop policy

Scenario Settings
URL filtering
Debug mode
settings
Runtime users and
monitoring parameters
Apply SLA

Scheduling a Test
• NeoLoad -project <file> -launch <scenario> [-options].
• Export SLA results in JUnit XML format.
• Generate reports (XML, HTML, JUnit XML).
• Publish results to the Neotys Team Server.
• Specify monitored hosts.

Setup and Watch
Running the Test
scenario
Test
completed
The runtime
settings
The real-time
analysis
time information
You will
learn

Runtime Overview
• Time is absolute or relative (Preferences>General Settings>Advanced).
The Real-Time Information
Main statistics
LGs status
Runtime
information

Runtime Graphs
• Time is absolute or relative (Preferences>General Settings>Advanced).

Runtime Errors
Comparison
with record
Stop failed
VUs
Errors
Detail

Runtime Alerts

Runtime Users
Add / remove
VUs
Follow a VU

Objectives
• Set a load-test scenario and launch a test.
Scenario
1. Create a load test scenario called PetstoreScenario.
2. Make sure the population PetstorePopulation is active.
3. Define an increasing workload that starts with a single VU and adds an additional VU
every 15 seconds. Indicate the maximum number of VUs allowed by the training license.
4. Start the test with a duration of 5 minutes and explore the real-time data displayed
including:
- Overview data.
- Performance data for containers.
- Server-side metrics.
- Runtime errors, if any.
Exercise 14

Introduction
Design
Runtime
Results
• Analyzing the Results
For Advanced Users
AGENDA

Test Summary
Analyzing the Results
Select a test result from
the drop-down list
Statistics and graphs
are displayed

Test Summary

Test Summary
Top 10 first alerts
Chronological chart

Test Summary
Cloud statistics

Values
Choose a statistic
Export aggregated or
raw data (only for
containers)
Filter data
Plot any element

Values
• Tree view mode
• List view mode

Graphs
Alerts and SLA
Percentile graphs
Choose the
statistics to plot

Memo: Graphs
• Zoom in with left mouse button selection.
• Export data with (CSV file or picture).
• Time, user load or monitor for horizontal axis.
• Plot percentile graphs.
• Do comparisons with the drop-down list.
• Use and create your own templates .

Errors
20.000 errors
displayed max
Request, response
or assertions
Error list
Detailed error
information
Detail of the
previous request

Alerts
Red zones for critical
alerts, yellow zones for
warning alerts
Alerts triggered on
performance counters
during the test

Logs
• Check the log files of the LGs and MAs.
Check the details
Select MAs or LGs
The log level

Results Manager
Tools
Time
Actions
Test preview
Test result list

Generating a Report
Select the output
Select the detail
Select the test

Objectives
• Create charts showing performance indicators.
• Drilldown through a series of measurements to highlight the cause of an issue.
Scenario
Based on the previous exercise:
1. Determine the average duration for the container where the category is selected.
2. Determine the page with the slowest average response time.
3. With a graph template determine the ‘health’ of the LGs used in the test.
4. Examine the errors encountered during the test.
5. Check if the overall page response time changed when the user load increases.
6. Explore the impact the user load had on the number of Tomcat threads.
7. Determine whether the number of current Tomcat threads affects the amount of OS
memory available.
Exercise 15

For Advanced Users
 Advanced Installation
• Servers
• Request & Page Settings
• Service Level Agreement
• Logical Actions
• Variables
• Data Exchange API
• Mobile Support
• Advanced Results
• The NeoLoad License
AGENDA

Using a Firewall
Advanced Installation
TCP 7100 CTRLLG
TCP 7200 CTRLMA
UDP 1359 LGCTRL
TCP 4569 LGCTRL
for the automatic discovery
function
Monitor
(optional)
NeoLoad Monitoring Engine
3G+H+ WiFi4G+
Controller
Command
Cloud Load Generators On-premise Load Generators
Load
APP
Application Server Database ServerWebServer
TCP 7100 CTRLLG
TCP 443 CTRLLG

For Advanced Users
• Advanced Installation
 Servers
• Logical Actions
• Variables
• Mobile Support
AGENDA

The Graphical User Interface
Servers
• Servers are automatically created during the
record.
• The target server may be quickly changed.
• Variables may be used (load-balancing).
• Server side authentication.
• Filled in during the record.
• Accounts may be dynamic (variables).
• Automatic session tracking when cookies
are used.
• When detected during recording, the
argument is automatically filled in.

Authentication Mechanisms
• NeoLoad supports Basic, Digest, NTLM and Negotiate mechanisms.
• On standard environments, the default protocol for Negotiate is Kerberos. If Kerberos
fails, Negotiate tries NTLM. However, for performance reasons NeoLoad uses NTLM in
place of Kerberos.
• If Kerberos is absolutely required, NeoLoad configuration file needs to be modified.
• NeoLoad prioritizes schemes in the order Negotiate, NTLM, Digest and Basic.
Servers

URL Rewriting and Browser Profile
• Deactivation forces the server to use URL rewriting to keep the session alive.
Servers
Cookie activation

For Advanced Users
• Servers
 Request & Page Settings
• Logical Actions
• Variables
• Mobile Support
AGENDA

Page Settings
Advanced Page Settings
Page resources
Request playback

Page Settings
• Request playback: NeoLoad plays the requests sequentially (by default requests are
played back in parallel).
• Dynamically execute the page resources: NeoLoad analyzes the HTML response of the
main request to execute dynamically the page resources.
• NeoLoad retrieves resources from the HTML code. Resources referenced by CSS style
sheets and JavaScript scripts are ignored. Also, this setting has no effect on the module
requests.
Advanced Page Settings

Request Settings
Advanced Request Settings
Change the
referrer
Encode or put an
equal
Response storage
Edit request
headers

For Advanced Users
• Servers
 Service Level Agreement
• Logical Actions
• Variables
• Mobile Support
AGENDA

Why?
• Hard to define what ‘good performance’ is.
• User standpoint evolution:
- 2006: 75% of users would not return to a website that take longer than 4 seconds (Akamai
research)
- 2009: a website needs to load up in 2 seconds (Akamai research)
• As a consequence, assessment of the service quality must be defined with end users
and/or functional experts.
• These thresholds are called Service Level Agreement (SLA).
Service Level Agreement

Two Types of SLAs in NeoLoad
• Per run: evaluated at the end of the test.
• Per time interval: evaluated each time a result is received during test runtime (real-time
alerts).

How?
Check the
summary
Apply the profile
Threshold definition
(acceptable and
failed conditions)

Statistics on SLAs
Check the status of
each SLA per profile
Check the status of
each SLA per VU

Edit SLA Profiles After a Test
• If an SLA profile is not enough accurate, it may be modified from the result manager.
• Transfer an SLA profile from Design to Results with to move it into several test results
and re-compute the result summary using the new configuration.
• Transfer an SLA profile from Results to Design with to copy it into the Design.
Same options as
the Design

Objectives
• Apply service level agreements that meet your needs.
Scenario
1. Open the project called ‘Training_Project’ (available on the resources page) and look at
the result called ‘Reference Test’.
2. Create a new SLA profile called ‘Reference’ following the criteria:
- Average response time per time interval must be less than 0.5 second.
- If the average response time per time interval is greater than 1 second the transaction must be
failed.
- The SLA profile must be applied to all containers of all VU profiles.
3. Once done, apply your changes and look at the results.
Exercise 16

For Advanced Users
• Servers
 Logical Actions
• Variables
• Mobile Support
AGENDA

Variable Modifier
• Used to modify the value of a variable before the value change policy is applied.
• Re-initialize or modify value (change policy is applied).
• Manage the Shared Queue actions.
Logical Actions

Fork
• Used to play actions in a different thread to the current one.
• When the VU main thread stops, all the threads created using Fork action are immediately
halted.
• Example of use, an action that plays every x seconds on the server to refresh a
component (Ajax, Flash plug-in…).
• Existing variable values may be copied locally. When a value is modified in another
thread, the variable value remains unchanged in the Fork action.
Logical Actions
Unchecked by
default

Wait Until
• Pauses the current execution thread until certain conditions have been verified.
• A condition is composed of two operands and an operator (except with Exists and
Doesn’t exist operators).
• A timeout may be defined. Once the delay is timed out, the thread is restarted. The
following action is then executed, even if the conditions have not been verified.
Logical Actions

Objectives
• Use the Fork logical action.
Scenario
1. Create manually a VU profile called ‘SearcherAndBrowser’ that only contains a Fork
action in the Actions container.
2. Copy the content of the ‘Searcher’ in the main thread and the ‘Browser’ in the second
thread.
3. Check the ‘SearcherAndBrowser’. Observe the concurrency of both paths.
4. Create a variable called ‘isComplete’ that has two values, TRUE and FALSE.
5. Use a Wait Until action to have the main thread waiting for the end of second thread
before its execution.
6. Check the ‘SearcherAndBrowser’. Verify that it works as expected.
Exercise 17

JavaScript
• Execute JavaScript in a VU profile.
• The NeoLoad API provides functions to manipulate variables and set up cookies.
• When using the Logger API, information is written to the LG log file
(loadGenerator.log.00x in the Logs directory).
• Handle strings, dates, calculation...
Logical Actions

JavaScript
Logical Actions
Access the variable
manager
Get a value from the
variable manager
Log a message
Assign a value

Java and JavaScript
• Use Java classes and functions in JavaScript actions.
• Copy the JAR files into <NeoLoad-project>/lib/jslib . The file is taken into
account automatically.
• The class names must be fully qualified within the JavaScript (even for the Java classes
like java.util.List) and preceded by Packages. For example var obj = new
Packages.com.company.MyClass();.
Logical Actions

JavaScript Libraries
• Create functions used in all the VU profiles.
• It helps reduce memory usage on the LGs, since the code stored in the libraries is shared
by the users.
Logical Actions

Objectives
• Insert a JavaScript action into the VU profile to perform specific actions.
Scenario
1. Add a JavaScript action to the ‘Buyer’ to log an entry every time an item is added to
the shopping cart.
2. Include a time-stamp for each log entry.
3. Add a second JavaScript action to retrieve the name of the LG to the log.
Exercise 18

Rendezvous
• Synchronize VUs to perform a specific task simultaneously.
• Create a precise load peak in the VU execution.
• Policies are global, regardless the number of LGs.
• Rendezvous may be controlled from a JavaScript action.
Logical Actions
Timeout
Rendezvous
policies

Custom Action
• Take advantage of predefined ‘high-level’ actions embedded in NeoLoad
• Full GUI integration
• More to come on the Neotys Community
Logical Actions
The command line
action

Objectives
• Use the command line custom action.
Scenario
1. At the end of the ‘Buyer’, use a custom action to log the total price of the order into a file
(use the ‘Echo’ function).
2. The title of the file must contain the instance number of the VU being executed.
Exercise 19

For Advanced Users
• Servers
• Logical Actions
 Variables
• Mobile Support
AGENDA

The Variable Manager
• Variables are created before the test is running.
• Depending on the policies, values are provided either by the controller or the LGs.
• With the Unique policy, a NoUniqueValueAvailableException exception is logged
when there is no more available value.
Variables
Global
• Sequential
• Random
Unique
• Sequential
• Random
Local
• Sequential
• Random
• Any
Global
• Any
Values provided by
the LGs
Values provided by
the controller

Shared Queue: Why?
• Share dynamic content amongst VU profiles.
• A FIFO pile with the following constraints:
- When full, all new arrivals are lost.
- When empty, the requesting VU has to wait for a new arrival to be added.
• A swap file may be used to pre-populate the queue at the start of the test or to save it at
the end.
• The Check VU does not have any effect on the swap file.
Variables

Shared Queue: How?
Variables
Queue size
Timeout
(${queueName} is
returned when the
queue is empty)
Swap file

• Values are provided to the LGs by the controller when the test is running.
Shared Queue: How?
• Populating the Shared Queue
Variables
– Consuming the
information
Select where the
value from the
queue is copied
Select the Shared
Queue that contains
the information
Select the variable
to add
Select the Shared
Queue to populate

Shared Queue and JavaScript
• A Shared Queue may be controlled by a JavaScript action.
• NeoLoad provides a comprehensive set of APIs.
• Main functions are:
- addSharedValue.
- createSharedQueue.
- getSharedQueueSize.
- peekSharedValue.
- pollSharedValue.
Variables

Objectives
• Share data accross two different VU profiles.
Scenario
1. Duplicate the “Buyer” and call it “SQ_Buyer”.
2. In the ‘SQ_Buyer’, populate a Shared Queue with the order number generated at the
end of the scenario.
3. Set the ‘SQ_Buyer’ authentication to j2ee / j2ee.
4. Record a new VU profile called ‘BuyOneItem’ that:
a. Logs in the Jpetstore.
b. Open the last order referred by the order number stored in the Shared Queue (manually alter
the URL in the browser using jpetstore/shop/viewOrder.shtml?orderId=1000 for
the record).
c. Buys the first item that appears in the shopping cart.
d. Logs out.
5. Launch a test combining both ‘SQ_Buyer’ and ‘BuyOneItem’ to have the ‘SQ_Buyer’
creating a list of orders and the ‘BuyOneItem’ using that list.
Exercise 20

JS Variable
• A variable computed by a piece of Javascript.
• A new function evaluate returns the variable value:
function evaluate() {
return new function() {
this.firstField = "a value";
this.secondField = myLibraryFunction();
}
}
• The field values are accessed by ${<variable name>.col_<field
number>} or ${<variable name>.<field name>}.
• The JavaScript variable is always computed on the LG side during the test.
Variables

JS Variable
Variables
Script
Value change policy

For Advanced Users
• Servers
• Logical Actions
• Variables
 Data Exchange API
• Mobile Support
AGENDA

The Data Exchange API Module makes it easy
to integrate data from a third party tool
Why?
Data Exchange API

Why?
• RESTful data service based on the Open Data Protocol
• OData is a standardized protocol for creating and reading data APIs
• NeoLoad receives external data to generate the results displayed in the GUI
• More information here
Data Exchange API

How?
• NeoLoad Open API accepts JSON format
• Create your own scripts to send data to the NeoLoad Server from any application
Data Exchange API

Objectives
• Create your script to plot any information from any application.
Scenario
1. Duplicate the buyer and call it “DEA_Buyer”.
2. In the ‘DEA_Buyer’, create a script that uses the Data Exchange API to plot the total
price of what the “DEA_buyer” buys.
3. Launch a test with a few virtual users to see the results.
Exercise 21

For Advanced Users
• Servers
• Logical Actions
• Variables
 Mobile Support
AGENDA

Mobile App Load Testing (Web-based and Native)
• Record native and web-based apps directly from the mobile device.
• Record web-based apps from the desktop browser with the Identify as feature.
• Generate realistic traffic with the WAN emulation.
Mobile Support

Recording Native and Mobile Web from Device
Mobile Support
Navigate as usual, the
traffic is recorded
Do not launch any
browser
Set the NeoLoad IP
and the recording port
Connect the device
(and NeoLoad) to a
WIFI network

Recording Native and Mobile Web from Device
Mobile Support
Parameters sent by
the mobile app
Server used by the
mobile app

For Advanced Users
• Servers
• Logical Actions
• Variables
• Mobile Support
 Advanced Results
AGENDA

Debug Mode
• Look at how behave the VU profiles in detail once the test is completed.
• Start the debug mode with .
• Two modes available (scenario advanced settings):
- Only the iterations containing errors are displayed (default behaviour).
- All VUs iterations are displayed.
• Launching a scenario in debug mode may severely hamper performances.
Advanced Results

Debug Mode
Advanced Results
Open the Check
VU window
The VU iterations that
contain errors
The instance of the
VU containing errors

Filters
• Filter any test result by time, population, zone and error.
• When a filter is applied, a new set of results is created.
• Filtered results may be viewed and used as any other test runs.
• Test reports may be generated following the usual procedure.
Advanced Results

Filters
Advanced Results
Time filter Error filter
Zone filterPopulation filter

Analysis: a Multi-Tiers Architecture
Advanced Results
Web browser Web server App server DB server
Potential issues:
• Thread saturation
• Cache usage
Potential issues:
• Thread saturation
• Data pool saturation
• Internal application issue
(code, memory leak,
garbage collection…)
Potential issues:
• Cache usage
• Internal database issue
(no index, costly
requests, memory…)• Apache
• IIS
• Tomcat
• JBoss
• webLogic
• webSphere
• OAS…
• MySQL
• DB2
• Oracle
• MS SQL
• PostgreSQL
• Internet Explorer
• Firefox
• Chrome
• Opera
• Safari…

Analysis: HTTP Server Contention
• Adding workers (depending on server resources, especially CPU) or an additional Apache
server may improve the situation.
Advanced Results
Limit of the
busyworker is
reached
Response times
are impacted

Analysis: App Server Contention
• Garbage collection causes the threads to be suspended (not the case with parallel
garbage collection). If it takes too long, response times are impacted.
Advanced Results
JVM overload
Response times
are impacted
Errors raise up

Analysis: Database Contention
• NeoLoad performance indicators give a detailed view of the most resource-hungry SQL
queries (Oracle and DB2 only).
• Such information helps optimizing SQL queries and validating the use of cache, memory,
sorts, indexes…
Advanced Results

Analysis: Network Contention
• In case of a network contention, if the time gap between response times and TTFB:
- Increases, that often indicates a network problem.
- Remains the same, that often indicates a server problem.
Advanced Results

Objectives
• Use the NeoLoad features to analyze a test result.
Scenario
1. Open the project called ‘Training_Project’ and select the test results (‘Charge_Prod_1’
and ‘Charge_Prod_2’).
2. Analyze both results by looking at the response times and each layer of the JpetStore
architecture.
Exercise 22

For Advanced Users
• Servers
• Logical Actions
• Variables
• Mobile Support
 The NeoLoad License
AGENDA

A Combination Of
The NeoLoad License
Number of
VUs
Protocol
Modules
Monitoring
Modules

FREE STANDARD PROFESSIONAL ENTERPRISE ULTIMATE
Suggested Usage
Developers and
testers running
small tests
One tester testing
one application at a
time
Teams testing one
application at a time
Organizations
testing multiple
applications
concurrently
Corporate CoEs and
teams needing to
run very large tests
Virtual Users 50 50 – 1,000,000 50 – 1,000,000 50 – 1,000,000 1,000,000+
Protocols All Included
HTTP/S Included,
Others Optional
HTTP/S Included,
Others Optional
HTTP/S Included,
Others Optional
All Included
Command Line
Access     
Continuous
Integration Plugin     
Monitoring  Optional   
Collaboration   
Integration &
Customization
• APM
• Data Exchange
API
• Custom Action
Extensions
Optional  
Shared License  
The NeoLoad License

Available Modules
The NeoLoad License
O/S Web Server DB Server App Server Network Protocol Integration
Windows IIS MS SQL Server .NET SNMP SOAP
CA / dynaTrace /
AppDynamics
Linux Apache Oracle JBoss Flex
Solaris Nginx My SQL Tomcat Silverlight
IBM AIX DB2 WebLogic Oracle Forms
HP-UX PostgreSQL WebSphere GWT
Data Exchange API
VMWare MongoDB Oracle AS Java
RSTAT JOnAS Oracle Siebel
Adobe LCDS Push
GlassFish RTMP
Custom Actions
SAP NetWeaver
Generic JMX
Kaazing

Standard
• To be used on a single machine
• May be rent (a minimum of 8 days)
• Testing days must be consumed after
the license activation within ‘10 x days’
of the license term
Shared
• Installed on the Neotys Team Server.
• Minimum split is 5VUs.
• Modules available for each split.
• Cannot be rent.
The NeoLoad License

The No Key Mode
• Features
- Record, design and check new or
existing VUs.
- Set up already existing monitors.
- Analyze existing test results.
- Perfect to design before running a test.
• Limitations
- Tests cannot be run.
- No collaboration.
- No ‘Monitoring Data Import’.
The NeoLoad License

Thank you for attending this training
Other courses are available at www.neotys.com

Neoload

More Related Content

What's hot (20)

Similar to Neoload (20)

More from Kumar Gupta (11)

Recently uploaded (20)

Neoload