Table of Contents

Introduction.......................................................................................................................................................1

Why Robot Framework....................................................................................................................................2

High-level architecture.....................................................................................................................................3

Getting more information................................................................................................................................4

Copyright and Licence.....................................................................................................................................5

Installation.........................................................................................................................................................6
Precondition...........................................................................................................................................6
Installation.............................................................................................................................................6
Where files are installed.........................................................................................................................6

Verifying installation........................................................................................................................................7

Uninstallation....................................................................................................................................................8

Creating Test Data............................................................................................................................................9

Structure.................................................................................................................................................9
Supported File Formats..........................................................................................................................9
Test Data Tables....................................................................................................................................9
Parsing Data Rules...............................................................................................................................10
Parsing...........................................................................................................................................10
Escaping........................................................................................................................................10
Whitespaces...................................................................................................................................10
Dividing test data to several rows.................................................................................................11

Creating Test Cases........................................................................................................................................12

Test Case Syntax..................................................................................................................................12
Basic Syntax..................................................................................................................................12
Settings in the Test Case Table.....................................................................................................12
Using Arguments...........................................................................................................................12
Test Case Name and Documentation...................................................................................................13
Tagging Test Cases..............................................................................................................................13
Test Setup and Teardown.....................................................................................................................14

Using Test Libraries.......................................................................................................................................16

Using Custom Libraries.......................................................................................................................16
Standard Libraries................................................................................................................................16

Variables..........................................................................................................................................................18
Introduction..........................................................................................................................................18
Variable Types.....................................................................................................................................18
Scalar Variables.............................................................................................................................18
List Variables................................................................................................................................19
Environment Variables..................................................................................................................19
Creating Variables...............................................................................................................................19
Variable Table...............................................................................................................................19
Variable File..................................................................................................................................20
Command Line..............................................................................................................................20
Return Values from Keywords......................................................................................................20
Using built-in Set Test/Suite/Global Variable keywords..............................................................21

i
 Table of Contents
Variables
Built-in Variables.................................................................................................................................21
Operating System Variables..........................................................................................................21
Number Variables..........................................................................................................................21
Boolean and None Variables.........................................................................................................22
Space and Empty Variables...........................................................................................................22
Automatic Variables......................................................................................................................22
Variable Priorities and Scopes.............................................................................................................23
Variable Priorities..........................................................................................................................23
Variable Scopes.............................................................................................................................24
Advanced Variable Features................................................................................................................24
Extended variable syntax...............................................................................................................24
Variables inside variables..............................................................................................................26

Creating User Keywords................................................................................................................................27

Basic Syntax........................................................................................................................................27
Settings in the Keyword Table.............................................................................................................27
User Keyword Arguments...................................................................................................................27
Positional arguments.....................................................................................................................27
Return Values................................................................................................................................28

Resource and Variable Files..........................................................................................................................29

Resource Files......................................................................................................................................29
How to use.....................................................................................................................................29
Resource File Structure.................................................................................................................29
Documenting Resource Files.........................................................................................................29
Variable Files.......................................................................................................................................30
How to include Variable Files.......................................................................................................30
Creating Variables Directly...........................................................................................................31
Basic Syntax...........................................................................................................................31
Using Objects as values..........................................................................................................31
Creating Variables Dynamically.............................................................................................31
Selecting which Variables to Include.....................................................................................32
Getting Variables from a Special Function...................................................................................32

Advanced Features..........................................................................................................................................34
Handling keywords with same names..................................................................................................34
Keyword scopes............................................................................................................................34
Specifying a keyword explicitly....................................................................................................34
Specifying library search order.....................................................................................................34
Timeouts..............................................................................................................................................35
Test Case Timeout.........................................................................................................................35
User Keyword Timeout.................................................................................................................36
For loops..............................................................................................................................................36
Normal for loop.............................................................................................................................36
Using several loop variables..........................................................................................................37
For in range...................................................................................................................................37
Exiting for loop.............................................................................................................................38
Repeating single keyword.............................................................................................................38
Conditional execution..........................................................................................................................39

Executing Test Cases......................................................................................................................................40

Basic Usage..........................................................................................................................................40
Starting the Test Execution...........................................................................................................40

ii
 Table of Contents
Executing Test Cases
Synopsis..................................................................................................................................40
Runner Scripts.........................................................................................................................40
Specifying Test Data...............................................................................................................40
Using Command Line Options......................................................................................................41
Using Options.........................................................................................................................41
Option Values Patterns...........................................................................................................41
Test Results...................................................................................................................................41
Command Line Output...........................................................................................................41
Generated Output Files...........................................................................................................41
Return Codes...........................................................................................................................42
Help and Version Info...................................................................................................................42
Debugging Execution Problems....................................................................................................42
Test Execution.....................................................................................................................................42
Execution Flow..............................................................................................................................42
Execution Suites and Tests.....................................................................................................43
Setups and Teardowns............................................................................................................43
Execution Order......................................................................................................................43
Continue on Failure.......................................................................................................................44
Stopping Test Execution Gracefully.............................................................................................44
Handling Teardowns...............................................................................................................44
Post-processing Outputs......................................................................................................................44
Using the rebot Tool......................................................................................................................45
Creating Different Reports and Logs............................................................................................45
Combining Outputs.......................................................................................................................45
Configuring Execution.........................................................................................................................45
Selecting Test Cases......................................................................................................................45
By Test Suite and Test Case Name.........................................................................................45
By Tag Names........................................................................................................................46
When no tests match selection................................................................................................46
Setting Criticality..........................................................................................................................47
Setting Metadata............................................................................................................................47
Setting the name......................................................................................................................47
Setting the documentation......................................................................................................47
Setting free metadata..............................................................................................................47
Setting tags..............................................................................................................................47
Setting Variables...........................................................................................................................47
Randomizing Execution Order......................................................................................................48
Controlling Console Output..........................................................................................................48
Console Width........................................................................................................................48
Console Colors........................................................................................................................48
Created Outputs...................................................................................................................................48
Different Output Files...................................................................................................................48
Output Directory.....................................................................................................................49
Output File..............................................................................................................................49
Log File...................................................................................................................................49
Report File..............................................................................................................................49
Debug File...............................................................................................................................49
Setting Titles...........................................................................................................................49
Setting Background Colors.....................................................................................................50
Log Levels.....................................................................................................................................50
System Log....................................................................................................................................51

iii
 Table of Contents
Extending Robot Framework \ Creating New Libraries.............................................................................52

Appendices.......................................................................................................................................................53
Test Data Settings................................................................................................................................53
Setting Table..................................................................................................................................53
Test Case Table.............................................................................................................................53
Keyword Table..............................................................................................................................53
Command Line Options.......................................................................................................................54
Test Execution Command Line Options.......................................................................................54
Output Post-processing Command Line Options..........................................................................55

Document Licence...........................................................................................................................................58

iv
Introduction
Robot Framework is a Python-based, extensible keyword-driven test automation framework for end-to-end
acceptance testing and acceptance-test-driven development (ATDD). It can be used for testing distributed,
heterogeneous applications, where verification requires touching several technologies and interfaces.

Introduction 1
Why Robot Framework
• Enables easy-to-use tabular syntax for creating test cases in a uniform way.
• Provides ability to create reusable higher-level keywords from the existing keywords.
• Provides easy-to-read result reports and logs in HTML format.
• Is platform and application independent.
• Provides a simple library API for creating customized test libraries which can be implemented
natively with either Python or Java.
• Provides a command line interface and XML based output files for integration into existing build
infrastructure (continuous integration systems).
• Provides support for Selenium for web testing, Java GUI testing, running processes, Telnet, SSH, and
so on.
• Supports creating data-driven test cases.
• Has built-in support for variables, practical particularly for testing in different environments.
• Provides tagging to categorize and select test cases to be executed.
• Enables easy integration with source control: test suites are just files and directories that can be
versioned with the production code.
• Provides test-case and test-suite -level setup and teardown.
• The modular architecture supports creating tests even for applications with several diverse interfaces.

Why Robot Framework 2

High-level architecture
Robot Framework is a generic, application and technology independent framework. It has a highly modular
architecture illustrated in the diagram below.

Robot Framework architecture

The test data is in simple, easy-to-edit tabular format. When Robot Framework is started, it processes the test
data, executes test cases and generates logs and reports. The core framework does not know anything about
the target under test, and the interaction with it is handled by test libraries. Libraries can either use application
interfaces directly or use lower level test tools as drivers.

High-level architecture 3
Getting more information
Project Pages Robot Framework http://code.google.com/p/robotframework/ Robot Framework SSH Library
http://code.google.com/p/robotframework-sshlibrary/ Robot Framework RIDE Editor
http://code.google.com/p/robotframework-ride/ Paramiko SSH Python Module
http://www.lag.net/paramiko/ Pycrypto Cryptography Python Toolkit
https://www.dlitz.net/software/pycrypto/ Contact Dimosthenes Fioretos dfiore -at- noc -dot- edunet -dot- gr

Getting more information 4

Copyright and Licence
Each respective tool has its own copyright. Robot Framework related: Apache Pycrypto: Public Domain
Paramiko: LGPLv2.1 Cream Test: GPLv3

Copyright and Licence 5

Installation
Precondition
Robot Framework runs on Python, and you need to have it installed to be able to use it. However, a recent
Python version is required,so installing python26 from the epel repository is a necessary step.

Installation
There are two main ways with which you can install robot framework and any relevant usefull software.The
first is by downloading and installing them by hand.The other is by installing the robot_testing repository
residing at yum.gridctb.uoa.gr and using the yum tool.Both ways,should be preceded by the installation of
python26 from the epel repository. The second and most easy way,requires the following steps:

yum install python26

wget http://yum.gridctb.uoa.gr/repository/robot_testing.repo -O /etc/yum.repos.d/robot_testing.re
yum install robotframework pycrypto paramiko SSHLibrary
yum install wxPython-common-gtk2-unicode wxPython2.8-gtk2-unicode robotframework-ride

The other available packages are:

• robotframework-ride: a wysiwyg editor/IDE for robot framework test data.

• cream_test: a test for the CREAM service,using the robot framework.
• wxPython-common: library needed by wxPython2.8
• wxPython2.8-gtk2-unicode: library needed by RIDE editor

Where files are installed

Paramiko:

• under /usr/lib/python2.6/site-packages/paramiko/

Pycrypto:

• under /usr/lib64/python2.6/site-packages/Crypto/
• /usr/lib64/python2.6/site-packages/pycrypto-2.3-py2.6.egg-info

Robot Framework:

• /usr/bin/jybot
• /usr/bin/pybot
• /usr/bin/rebot
• /usr/lib/python2.6/site-packages/robotframework-trunk_20110727-py2.6.egg-info
• and under /usr/lib/python2.6/site-packages/robot/

SSHLibrary:

• under /usr/lib/python2.6/site-packages/SSHLibrary/

RIDE:

• under /usr/lib/python2.6/site-packages/robotide/
• /usr/bin/ride.py

Installation 6
Verifying installation
For robot framework,execute:

pybot --version

If everything is ok,it should output the version of robot framework. For the RIDE editor,execute:

ride.py

If everything is ok,the editor environment should appear. For the ssh library,create the following file:

| *Setting* | *Value* |
| Library | SSHLibrary |

| *Test Case* | *Action* | *Argument* | *Argument*

| Test1 | Open Connection | put ssh host here |
| | Login | put ssh user here | put ssh passwo

Then save it -lets assume the name is test.txt- and execute:

pybot test.txt

The output should be something like:

==============================================================================
Test
==============================================================================
Test1 | PASS |
------------------------------------------------------------------------------
Test | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output: /home/dfiore/robot/output.xml
Log: /home/dfiore/robot/log.html
Report: /home/dfiore/robot/report.html

Verifying installation 7
Uninstallation
All the modules presented can be uninstalled by manually deleting the files the installed (the list is available
in a previous section of this document). Other than that,if the installation was done by the tools provided by
the OS,they can by uninstalled by the same means.For example:

yum remove robotframework

Uninstallation 8
Creating Test Data
Structure
The hierarchical structure for arranging test cases is built as follows:

• Test cases are created in test case files.

• A test case file automatically creates a test suite containing the test cases in that file.
• A directory containing test case files forms a higher-level test suite. Such a test suite directory has
suites created from test case files as its sub test suites.
• A test suite directory can also contain other test suite directories, and this hierarchical structure can be
as deeply nested as needed.
• Test suite directories can have a special initialization file.

In addition to this, there are:

• Test libraries containing the lowest-level keywords.

• Resource files with variables and higher-level user keywords.
• Variable files to provide more flexible ways to create variables than resource files.

Supported File Formats

Robot Framework test data is defined in tabular format, using either the hypertext markup language (HTML),
tab-separated values (TSV), plain text, or reStructuredText (reST) formats. Robot Framework selects a parser
for the test data based on the file extension. The extension is case-insensitive, and the recognized extensions
are .html, .htm and .xhtml for HTML, .tsv for TSV, .txt for plain text, and .rst or .rest for reStructuredText.For
ease of use and generally easier development and maintainability,txt and html files are mostly used -note that
the RIDE editor can save the test case files in either txt,html or tsv-.Test case files can be edited with any
editor.Still,the provided RIDE editor is recomended for graphical environments. For the HTML format,the
test data is defined in separate tables (see the example below). Robot Framework recognizes these test data
tables based on the text in their first cell. Everything outside recognized tables is ignored. For the plain text
format,test data tables are recognized from one or more asterisks ( ), followed by a normal table name and an
optional closing asterisks. Everything before the first recognized table is ignored similarly as data outside
tables in HTML data. For a separator between cells you can use either two or more spaces or a pipe character
surrounded with spaces ( | ). Also,everything before the first table is ignored.Example:

Test Case Action Argument Argument

Simple Test Should Contain Hello, world! world
This test case checks whether the second argument ("world") is contained within the first argument ("Hello,
world!").

Test Data Tables

Test data is structured in four types of tables listed below. These test data tables are identified by the first cell
of the table, and the last column in the table below lists different aliases that can be used as a table name.

Table name Used for A

Setting table 1) Importing test libraries, resource files and variable files S
2) Defining metadata for test suites and test cases

Variable table Defining variables that can be used elsewhere in the test data V
Test case table Creating test cases from available keywords T
Keyword table Creating user keywords from existing lower-level keywords K

Creating Test Data 9

 RobotFrameworkAdvancedGuide < EMI < TWiki

Parsing Data Rules

Parsing
When Robot Framework parses the test data, it ignores:

• All tables that do not start with a recognized table name in the first cell.
• Everything else on the first row of a table apart from the first cell.
• Data outside tables in HTML and data before the first table in TXT.
• All empty rows, which means these kinds of rows can be used to make the tables more readable.
• All empty cells at the end of rows; you must add a backslash (\) to prevent such cells from being
ignored.
• All single backslashes (\); they are used as an escape character.
• All characters following a hash mark (#), if it is the first character of a cell; this means that hash
marks can be used to enter comments in the test data.
• All formatting in the HTML test data.

Escaping
When Robot Framework ignores some data, this data is not available in any resulting reports and,
additionally, most tools used with Robot Framework also ignore them. To add information that is visible in
Robot Framework outputs, or available to, for example, RIDE, place it to the documentation or other metadata
of test cases or suites, or log with the Log or Comment keywords available from the BuiltIn library. The
escape character for the parser is the backslash (\). The escape character can be used as follows:

• To escape special characters so that their literal values are used:

• \${notvar} means a literal string ${notvar} that looks like a variable
• \\ means a single backslash (for example, C:\\Temp)
• \# means a literal hash (#) mark, even at the beginning of a cell
• To affect the parsing of whitespaces.
• To prevent ignoring empty cells at the end of a row (this requires \ to be in the appropriate cell).
Another possibility is using built-in variable ${EMPTY}.

Note that these escaping rules are applied only to arguments to keywords and values to settings. They are not
used, for example, with keyword and test case names.

Whitespaces
Robot Framework handles whitespace, such as spaces, newlines and tabs, the same way as they are handled in
HTML. This means that Robot Framework:

• Removes leading and trailing whitespace in all cells.

• Changes multiple consecutive spaces into single spaces.
• Converts all newlines and tabs into spaces.

To prevent Robot Framework from parsing data according to these rules, a backslash can be used:

• Before leading spaces, for example \ some text.

• Between consecutive spaces, for example text \ \ more text.
• After trailing spaces, for example some text \ \.
• As \n to create a newline, for example first line\n2nd line.
• As \t to create a tab character, for example text\tmore text.
• As \r to create a carriage return, for example text\rmore text.

Parsing Data Rules 10

 RobotFrameworkAdvancedGuide < EMI < TWiki
Another, and often clearer, possibility for representing leading, trailing, or consecutive spaces is using built-in
variable ${SPACE}. The extended variable syntax even allows syntax like ${SPACE * 8} which makes
handling consecutive spaces very simple.

Dividing test data to several rows

If there is more data than readily fits a row, it possible to use ellipsis (...) to continue the previous line. In test
case and user keyword tables, the ellipsis must be preceded by at least one empty cell. In settings and variable
tables, it can be placed directly under the setting or variable name. In all tables, all empty cells before the
ellipsis are ignored. Additionally, values of settings that take only one value (mainly documentations) can be
split to several columns. These values will be then catenated together with spaces when the test data is parsed.
Example:

Test Case Action Argument Argument Argument Argument Argument Argument

Example Do Something value1 value2 value3 value4 value5 value6
Can be turned to:
Test Case Action Argument Argument
Example Do Something value1 value2
... value3 value4
... value5 value6

Whitespaces 11
Creating Test Cases
Test Case Syntax
Basic Syntax
Test cases are constructed in test case tables from the available keywords. Keywords can be imported from
test libraries or resource files, or created in the keyword table of the test case file itself. The first column in the
test case table contains test case names. A test case starts from the row with something in this column and
continues to the next test case name or to the end of the table. It is an error to have something between the
table headers and the first test. The second column normally has keyword names. An exception to this rule is
setting variables from keyword return values, when the second and possibly also the subsequent columns
contain variable names and a keyword name is located after them. In either case, columns after the keyword
name contain possible arguments to the specified keyword. Example:

Setting Value
Library /path/to/cream/library
Test Case Action Argument Argument Argument
Submit Job Test Create Proxy dteam p4sSw0rD
${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
${final_status}= Get Final Status ${job_id}
Should Be Equal ${final_status} DONE-OK
Destroy Proxy
This test imports a library,and then uses the keywords defined within to submit a job to a cream ce and it
checks its final status.

Settings in the Test Case Table

Test cases can also have their own settings. Setting names are always in the second column, where keywords
normally are, and their values are in the subsequent columns. Setting names have square brackets around them
to distinguish them from keywords. The available settings are listed below and explained later in this section.
[Documentation] Used for specifying a test case documentation. [Tags] Used for tagging test cases. [Setup],
[Teardown] Specify test setup and teardown. Have also synonyms [Precondition] and [Postcondition],
respectively. [Timeout] Used for setting a test case timeout. Timeouts are discussed in their own section.
Following is the last example,enhanced with the use of settings.

Setting Value
Library /path/to/cream/library
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce
[Tags] job_submit
Create Proxy dteam p4sSw0rD
${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
${final_status}= Get Final Status ${job_id}
Should Be Equal ${final_status} DONE-OK
Destroy Proxy

Using Arguments
The earlier examples have already demonstrated keywords taking different arguments, and this section
discusses this important functionality more thoroughly. How to actually implement user keywords and library

Creating Test Cases 12

 RobotFrameworkAdvancedGuide < EMI < TWiki
keywords with different arguments is discussed in separate sections. Keywords can accept zero or more
arguments, and some arguments may have default values. What arguments a keyword accepts depends on its
implementation, and typically the best place to search this information is keyword's
documentation.Arguments can either be:

• Required: An argument must always be passed to the keyword.An example of this could be a logging
keyword.
• Have default value: The argument may or may not be given.In the first case,it has the explicit value
given.In the latter case,it has its default value.An example of this could be a keyword creating a
temporary file name.
• Variable number of arguments: The arguments provided may be from zero (in case of a default value
being used at the same time),to any number.An example of this could be a keyword which deletes
files. Important Note: The variable arguments must always be the last ones!
• Named: Arguments can be given their values,by naming them.In this way,the arguments can be given
in any order.
• Embedded: Arguments may finally be incorporated inside user level keywords,thus being able to call
a user keyword and in this way implicitely assign values to variables.

Test Case Name and Documentation

The test case name comes directly from the Test Case table: it is exactly what is entered into the test case
column. Test cases in one test suite should have unique names. Pertaining to this, you can also use the
automatic variable ${TEST_NAME} within the test itself to refer to the test name. It is available whenever a
test is being executed, including all user keywords, as well as the test setup and the test teardown. The
[Documentation] setting allows you to set a free documentation for a test case. That text is shown in the
command line output, as well as the resulting test logs and test reports. If the documentation is long, it can be
split into several cells that are catenated together with spaces. It is possible to use simple HTML formatting
and variables can be used to make the documentation dynamic. Possible non-existing variables in
documentation are just left unchanged. It is important that test cases have clear and descriptive names, and in
that case they normally do not need any documentation. If the logic of the test case needs documenting, it is
often a sign that keywords in the test case need better names and they are to be enhanced, instead of adding
extra documentation.

Tagging Test Cases

Using tags in Robot Framework is a simple, yet powerful mechanism for classifying test cases. Tags are free
text and they can be used at least for the following purposes:

• Tags are shown in test reports, logs and, of course, in the test data, so they provide metadata to test
cases.
• Statistics about test cases (total, passed, failed are automatically collected based on tags).
• With tags, you can include or exclude test cases to be executed.
• With tags, you can specify which test cases are considered critical.

In this section it is only explained how to set tags for test cases, and different ways to do it are listed below.
These approaches can naturally be used together.

• Force Tags in the Setting table

♦ All test cases in a test case file with this setting always get specified tags. If it is used in the
test suite initialization file, all test cases in sub test suites get these tags.
• Default Tags in the Setting table
♦ Test cases that do not have a [Tags] setting of their own get these tags.
• [Tags] in the Test Case table

Using Arguments 13
 RobotFrameworkAdvancedGuide < EMI < TWiki
♦ A test case always gets these tags. Additionally, it does not get the possible tags specified
with Default Tags, so it is possible to override the Default Tags by using empty value. It is
also possible to use value NONE to override default tags.
• --settag command line option
♦ All executed test cases get tags set with this option in addition to tags they got elsewhere.
• Set Tags and Remove Tags keywords

These BuiltIn keywords can be used to modify tags during the test execution. Tags are free text, but they are
normalized so that they are converted to lowercase and all spaces are removed. If a test case gets the same tag
several times, other occurrences than the first one are removed. Tags can be created using variables, assuming
that those variables exist. Example:

Setting Value Value Value

Force Tags req-42
Default Tags owner-john smoke
Variable Value Value Value
${HOST} 10.0.1.42
Test Case Action Argument Argument
No own tags [Documentation] This test has tags owner-john, smoke, req-42
No Operation

With own tags [Documentation] This test has tags not_ready, owner-mrx, req-42
[Tags] owner-mrx not_ready
No Operation

Own tags with variables [Documentation] This test has tags host-10.0.1.42, req-42
[Tags] host-${HOST}
No Operation

Empty own tags [Documentation] This test has tags req-42

[Tags]
No Operation

Set Tags and Remove Tags Keywords [Documentation] This test has tags mytag, owner-john
Set Tags mytag
Remove Tags smoke req-*

Test Setup and Teardown

Robot Framework has similar test setup and teardown functionality as many other test automation
frameworks. In short, a test setup is something that is executed before a test case, and a test teardown is
executed after a test case. In Robot Framework setups and teardowns are just normal keywords with possible
arguments. Setup and teardown are always a single keyword. If they need to take care of multiple separate
tasks, it is possible to create higher-level user keywords for that purpose. An alternative solution is executing
multiple keywords using the BuiltIn keyword Run Keywords. The test teardown is special in two ways. First
of all, it is executed also when a test case fails, so it can be used for clean-up activities that must be done
regardless of the test case status. All the keywords in the teardown are also executed even if one of them fails.
This continue on failure functionality can be used also with normal keywords, but inside teardowns it is on by
default. The easiest way to specify a setup or a teardown for test cases in a test case file is using the Test Setup
and Test Teardown settings in the Setting table. Individual test cases can also have their own setup or
teardown. They are defined with the [Setup] or [Teardown] settings in the test case table and they override
possible Test Setup and Test Teardown settings. Having no keyword after a [Setup] or [Teardown] setting

Tagging Test Cases 14

 RobotFrameworkAdvancedGuide < EMI < TWiki
means having no setup or teardown. Following is the previous cream job submission example,enhanced with
the use of setups/teardowns

Setting Value Value Value

Library /path/to/cream/library
Test Setup Create Proxy dteam p4sSw0rD
Test Teardown Destroy Proxy
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce
[Tags] job_submit
${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
${final_status}= Get Final Status ${job_id}
Should Be Equal ${final_status} DONE-OK
Our sample test,now uses setup and teardown facilities. In the case of the proxy creation,in case we had any
more tests in this testsuite,we wouldn't have to create the proxy again and again for each seperate test case. In
the case of proxy destruction,we also ensure that in case of test failure,the proxy is always destroyed.

Test Setup and Teardown 15

Using Test Libraries
Using Custom Libraries
Test libraries contain those lowest-level keywords, often called library keywords, which actually interact with
the system under test. All test cases always use keywords from some library, often through higher-level user
keywords. This section explains how to take test libraries into use and how to use the keywords they provide.
Creating test libraries is described in a separate section. Libraries can be imported,as allready has been seen in
previous examples,in the settings table of a test.They can also be imported with the builtin keyword "Import
Library" used within a test case -this can be usefull for example,when a library isn't availlable until test
execution- . Using the full path name to the library is the safest and most reliable way to import it,but it can
also be imported by its name if it is availavle in PYTHONPATH.All the libraries dependencies -if any- must
be made available in the same way -i.e. by their absolute or relative name inside the imported library- .
Libraries can be imported with arguments,in this way:

Setting Value Value Value

Library /path/to/cream/library arg1 arg2
It is possible to use default values,variable number or named arguments,exactly like keyword arguments.
Libraries can be imported with a custom name,in this way:
Setting Value Value Value
Library /path/to/cream/library WITH NAME libname
This is usefull in the following cases:

• There is a need to import the same library several times with different arguments. This is not possible
otherwise.
• The library name is inconveniently long. This can happen, for example, if a Java library has a long
package name.
• You want to use variables to import different libraries in different environments, but refer to them
with the same name.
• The library name is misleading or otherwise poor. In this case, changing the actual name is, of course,
a better solution.

Standard Libraries
Some test libraries are distributed with Robot Framework and these libraries are called standard libraries.
These are the available standard libraries:

• BuiltIn
• OperatingSystem
• Telnet
• Collections
• String
• Dialogs
• Screenshot
• Remote

The BuiltIn library is special, because it is taken into use automatically and thus its keywords are always
available. Other standard libraries need to be imported in the same way as any other libraries, but there is no
need to install them. Additionally, they work when running tests both with Python and Jython (with the
Screenshot library as an exception). New standard libraries can, and will, be added in the future. If you have
an idea for a new standard library, or even have one that could be incorporated immediately, you can contact
Robot Framework developers. In general, a library is a good candidate to be added into standard libraries if it

Using Test Libraries 16

 RobotFrameworkAdvancedGuide < EMI < TWiki
is generic, works on both Python and Jython without any external dependencies, and is adequately tested and
documented.

BuiltIn library The BuiltIn library provides a set of generic keywords needed often. It is imported
automatically and thus always available. The provided keywords can be used, for example, for verifications
(e.g. Should Be Equal, Should Contain), conversions (e.g. Convert To Integer) and for various other purposes
(e.g. Log, Sleep, Run Keyword If, Set Global Variable). For more information, see the BuiltIn library
documentation.

OperatingSystem library The OperatingSystem library enables various operating system related tasks to be
performed in the system where Robot Framework is running. It can, among other things, execute commands
(e.g. Run), create and remove files and directories (e.g. Create File, Remove Directory), check whether files
or directories exists or contain something (e.g. File Should Exist, Directory Should Be Empty) and manipulate
environment variables (e.g. Set Environment Variable). For more information, see the OperatingSystem
library documentation.

Telnet library The Telnet library makes it possible to connect to Telnet servers and execute commands on the
opened connections. For more information, see the Telnet library documentation.

Collections library The Collections library provides a set of keywords for handling Python lists and
dictionaries. This library has keywords, for example, for modifying and getting values from lists and
dictionaries (e.g. Append To List, Get From Dictionary) and for verifying their contents (e.g. Lists Should Be
Equal, Dictionary Should Contain Value). For more information, see the Collections library documentation.

String library The String library enables manipulating strings (e.g. Replace String With Regexp, Split To
Lines) and verifying their contents (e.g. Should Be String). For more information, see the String library
documentation.

Dialogs library The Dialogs library provides means for pausing the test execution and getting input from
users. The dialogs are slightly different depending on are tests run on Python or Jython but they provide the
same functionality. For more information, see the Dialogs library documentation.

Screenshot library The Screenshot library has keywords to capture and store screenshots of the whole desktop.
This library is implemented with Java AWT APIs, so it can be used only when running Robot Framework on
Jython. For more information, see the Screenshot library documentation.

Remote library The Remote library is totally different than the other standard libraries. It does not have any
keywords of its own but it works as a proxy between Robot Framework and actual test library
implementations. These libraries can be running on other machines than the core framework and can even be
implemented using languages not supported by Robot Framework natively. See separate Remote library
interface section for more information about the concept.

Standard Libraries 17
Variables
Introduction
Variables are an integral feature of Robot Framework, and they can be used in most places in test data. Most
commonly, they are used in arguments for keywords in test case tables and keyword tables, but also all
settings allow variables in their values. A normal keyword name cannot be specified with a variable, but the
BuiltIn keyword Run Keyword can be used to get the same effect. Robot Framework itself has two kinds of
variables, scalars and lists, and they have the syntaxes ${SCALAR} and @{LIST}, respectively. In addition
to this, environment variables can be used directly with the syntax %{VARIABLE}. If a nonexistent variable
is used in the test data, the keyword using it fails. If the same syntax that is used for variables is needed as a
literal string, it must be escaped with a backslash as in \${NAME}.

Variable Types
Robot Framework variables, similarly as keywords, are case-insensitive, and also spaces and underscores are
ignored. However, it is recommended to use all capital letters with global variables (for example, ${PATH} or
${TWO_WORDS}) and small letters with variables that are only available in certain test cases or user
keywords (for example, ${my_var} or ${myVar}). Much more importantly, though, cases should be used
consistently. Unlike in some programming languages where similar variable syntax is used, curly braces ({
and }) are mandatory in Robot Framework test data. Basically, variable names can have any characters
between the curly braces. However, using only alphabetic characters from a to z, numbers, underscore and
space is recommended, and it is even a requirement for using the extended variable syntax.

Scalar Variables
When scalar variables are used in the test data, they are replaced with the value they are assigned to. While
scalar variables are most commonly used for simple strings, you can assign any objects, including lists, to
them. The scalar variable syntax, for example ${NAME}, should be familiar to most users, as it is also used,
for example, in shell scripts and Perl programming language. The example below illustrates the usage of
scalar variables.It is the well known example we have allready used,which submits a job to a cream ce.

Setting Value Value Value

Library /path/to/cream/library
Test Setup Create Proxy dteam p4sSw0rD
Test Teardown Destroy Proxy
Variable Value
${ce} cream_endpoint.cern.ch:8443/queue
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce
[Tags] job_submit
${job_id}= Submit Job /path/to/jdl ${ce}
${final_status}= Get Final Status ${job_id}
Should Be Equal ${final_status} DONE-OK
This is the same test as before,submiting a job to a cream ce and then checking its final status. The pros of
using variables should be well known by anyone reading this page . When a scalar variable is used as the
only value in a test data cell, the scalar variable is replaced with the value it has. The value may be any object.
When a scalar variable is used in a test data cell with anything else (constant strings or other variables), its
value is first converted into a Unicode string and then catenated to whatever is in that cell. Converting the
value into a string means that the object's method unicode (in Python, with str as a fallback) is called.Variable
values are used as-is without conversions when passing arguments to keywords using the named arguments

Variables 18
 RobotFrameworkAdvancedGuide < EMI < TWiki
syntax like argname=${var}. Note: Converting variables to Unicode obviously fails if the variable cannot be
represented as Unicode. This can happen, for example, if you try to use byte sequences as arguments to
keywords so that you catenate the values together like ${byte1}${byte2}. A workaround is creating a variable
that contains the whole value and using it alone in the cell (e.g. ${bytes}) because then the value is used as-is.

List Variables
List variables are compound variables that can have several values assigned to them. In short, they are always
lists and can contain an unlimited number of entries (also empty lists are possible). The main benefit of list
variables is that they allow you to assign a name for a larger data set. While list variables normally contain
only strings, other content is also possible. When you use a list variable in test data, then the elements of the
list are inserted as new cells in the test data. Thus, if the list variable contains two elements, the cell
containing the list variable is turned into two cells with the content of the list variable. Note that cells with list
variables should not contain other content. The list variable syntax, @{NAME}, is borrowed from Perl.
Assuming that the list variable @{LST} is set to the value
['/path/to/jdl','cream_endpoint.cern.ch:8443/queue'], the following two test cases are equivalent:

Test Case Action JDL Path CREAM CE

Strings Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
Strings Submit Job @{LST}
It is also possible to access a certain value from the list variable with the syntax @{NAME}[i], where i is the
index of the selected value. Indexes start from zero, and trying to access a value with too large an index causes
an error. List items accessed in this manner can be used similarly as scalar variables:
Test Case Action JDL Path CREAM CE
Strings Submit Job @{LST}[0] @{LST}[1]

Environment Variables
Robot Framework allows using environment variables in the test data using the syntax
%{ENV_VAR_NAME}. They are limited to string values. Environment variables set in the operating system
before the test execution are available during it, and it is possible to create new ones with the keyword Set
Environment Variable or delete existing ones with the keyword Delete Environment Variable, both available
in the OperatingSystem library. Because environment variables are global, environment variables set in one
test case can be used in other test cases executed after it. However, changes to environment variables are not
effective after the test execution.

Creating Variables
Variables can spring into existence from different sources.Those include variable tables in test case files and
resource files,variable files,return values from keywords and some builtin special purpose keywords.

Variable Table
The simplest possible variable assignment is setting a string into a scalar variable. This is done by giving the
variable name (including ${}) in the first column of the Variable table and the value in the second one. If the
second column is empty, an empty string is set as a value. Also an already defined variable can be used in the
value.Example:

Variable Value Value

${test_name} cream_test
${version} 1.2
${test_descr} ${name} ${version}

Scalar Variables 19
 RobotFrameworkAdvancedGuide < EMI < TWiki

@{lst} val1 val2

@{lst2} @{lst} val3
@{empty}
@{many} val1 val2
... val3 val4

Variable File
Variable files are the most powerful mechanism for creating different kind of variables. It is possible to assign
variables to any object using them, and they also enable creating variables dynamically. The variable file
syntax and taking variable files into use is explained in a later section.

Command Line
Variables can be set from the command line either individually with the --variable (-v) option or using a
variable file with the --variablefile (-V) option. Variables set from the command line are globally available for
all executed test data files, and they also override possible variables with the same names in the Variable table
and in variable files imported in the test data. The syntax for setting individual variables is --variable
name:value, where name is the name of the variable without ${} and value is its value. Several variables can
be set by using this option several times. Only scalar variables can be set using this syntax and they can only
get string values. Many special characters are difficult to represent in the command line, but they can be
escaped with the --escape option. --variable EXAMPLE:value --variable HOST:localhost:7272 --variable
USER:robot --variable ESCAPED:Qquotes_and_spacesQ --escape quot:Q --escape space:_ In the examples
above, variables are set so that:

• ${EXAMPLE} gets the value value

• ${HOST} and ${USER} get the values localhost:7272 and robot
• ${ESCAPED} gets the value "quotes and spaces"
• The basic syntax for taking variable files into use from the command line is --variablefile
path/to/variables.py, and Taking variable files into use section has more details. What variables
actually are created depends on what variables there are in the referenced variable file.

If both variable files and individual variables are given from the command line, the latter have higher priority.

Return Values from Keywords

Return values from keywords can also be set into variables. This allows communication between different
keywords even in different test libraries. The syntax for a simple case is illustrated in the example below:
Assigning values from keywords to variables

Test Case Action Argument Argument

Returning ${x}= Get X arg
Log We got ${x}!
In the example above, the value returned by the Get X keyword is first set into the variable ${x} and then
used by the Log keyword. This syntax works in all cases where a keywords returns something, and the
variable is set to whatever value returned by the keyword. Having the equals sign = after the variable name is
not obligatory, but recommended, because it makes the assignment more explicit. If a keyword returns a list, it
is also possible to assign the return value into several scalar variables and/or one list variable. Assigning
multiple values at once
Test Case Action Argument Argument Argument
Return Multiple ${scalar}= Get 3
${a} ${b} ${c} = Get 3
${first} @{rest} = Get 3

Variable Table 20
 RobotFrameworkAdvancedGuide < EMI < TWiki

@{list} = Get 3
Assuming that the keyword Get 3 returns a list [1, 2, 3], the following variables are created: ${scalar} with the
value [1, 2, 3] ${a}, ${b} and ${c} with the values 1, 2, and 3, respectively ${first} with the value 1, and
@{rest} with the value [2, 3] @{list} with the value [1, 2, 3] Variables set in this manner are otherwise
similar to any other variables, but they are available only within the scope of the test case or keyword where
they are created. Thus it is not possible, for example, to set a variable in one test case and use it in another.
This is because, in general, automated test cases should not depend on each other, and accidentally setting a
variable that is used elsewhere could cause hard-to-debug errors. If there is a genuine need for setting a
variable in one test case and using it in another, it is possible to use built-in keywords as explained in the next
section.

Using built-in Set Test/Suite/Global Variable keywords

The BuiltIn library has keywords Set Test Variable, Set Suite Variable and Set Global Variable which can be
used for setting variables dynamically during the test execution. If a variable already exists within the new
scope, its value will be overwritten, and otherwise a new variable is created. Variables set with Set Test
Variable keyword are available everywhere within the scope of the currently executed test case. For example,
if you set a variable in a user keyword, it is available both in the test case level and also in all other user
keywords used in the current test. Other test cases will not see variables set with this keyword. Variables set
with Set Suite Variable keyword are available everywhere within the scope of the currently executed test
suite. Setting variables with this keyword thus has the same effect as creating them using the Variable table in
the test data file or importing them from variable files. Other test suites, including possible child test suites,
will not see variables set with this keyword. Variables set with Set Global Variable keyword are globally
available in all test cases and suites executed after setting them. Setting variables with this keyword thus has
the same effect as creating from the command line using the options --variable or --variablefile. Because this
keyword can change variables everywhere, it should be used with care.

Built-in Variables
Robot Framework provides some built-in variables that are available automatically.

Operating System Variables

Built-in variables related to the operating system ease making the test data operating-system-agnostic.
Available operating-system-related built-in variables: Variable Explanation ${CURDIR} An absolute path to
the directory where the test data file is located. This variable is case-sensitive. ${TEMPDIR} An absolute
path to the system temporary directory. In UNIX-like systems this is typically /tmp, and in Windows
c:\Documents and Settings\\Local Settings\Temp. ${EXECDIR} An absolute path to the directory where test
execution was started from. ${/} The system directory path separator. / in UNIX-like systems, \ in Windows.
${:} The system path element separator. : in UNIX-like systems and ; in Windows. Using
operating-system-related built-in variables

Test Case Action Argument Argument

Example Create File ${CURDIR}${/}input.data Some text here
Set Environment CLASSPATH ${TEMPDIR}${:}${TEMPDIR}${/}foo.jar
Variable

Number Variables
The variable syntax can be used for creating both integers and floating point numbers, as illustrated in the
example below. This is useful when a keyword expects to get an actual number, and not a string that just looks
like a number, as an argument. Example:

Return Values from Keywords 21

 RobotFrameworkAdvancedGuide < EMI < TWiki

Test Case Action Argument Argument Comment

Example 1A Connect example.com 80 # Connect gets two strings as arguments
Example 1B Connect example.com ${80} # Connect gets a string and an integer
Example 2 Do X ${3.14} ${-1e-4} # Do X gets floating point numbers 3.14 and -0.0001
It is possible to create integers also from binary, octal, and hexadecimal values using 0b, 0o and 0x prefixes,
respectively. The syntax is case insetive. Example:
Test Case Action Argument Argument
Example Should Be Equal ${0b1011} ${11}
Should Be Equal ${0o10} ${8}
Should Be Equal ${0xff} ${255}
Should Be Equal ${0B1010} ${0XA}

Boolean and None Variables

Also Boolean values and Python None can be created using the variable syntax similarly as numbers.
Example:

Test Case Action Argument Argument Comment

Boolean Set Status ${true} # Set Status gets Boolean true as an argument
Create Y something ${false} # Create Y gets a string and Boolean false
None Do XYZ ${None} # Do XYZ gets Python None as an argument
Null ${ret} = Get Value arg # Checking that Get Value returns python none
Should Be Equal ${ret} ${None}
These variables are case-insensitive, so for example ${True} and ${true} are equivalent. Additionally,
${None} and ${null} are synonyms, because when running tests on the Jython interpreter, Jython
automatically converts None and null to the correct format when necessary.

Space and Empty Variables

It is possible to create spaces and empty strings using variables ${SPACE} and ${EMPTY}, respectively.
These variables are useful, for example, when there would otherwise be a need to escape spaces or empty cells
with a backslash. If more than one space is needed, it is possible to use the extended variable syntax like
${SPACE * 5}. In the following example, Should Be Equal keyword gets identical arguments but those using
variables are easier to understand than those using backslashes. Example:

Test Case Action Argument Argument

One Space Should Be Equal ${SPACE} \\
Four Spaces Should Be Equal ${SPACE * 4} \\\\\
Ten Spaces Should Be Equal ${SPACE * 10} \\\\\\\\\\\
Quoted Space Should Be Equal "${SPACE}" ""
Quoted Spaces Should Be Equal "${SPACE * 2}" "\"
Empty Should Be Equal ${EMPTY} \

Automatic Variables
Some automatic variables can also be used in the test data. These variables can have different values during
the test execution and some of them are not even available all the time. Available automatic variables

Variable Explanation Available

${TEST NAME} The name of the current test case. Test case
@{TEST TAGS} Contains the tags of the current test case in alphabetical order. Test case

Number Variables 22
 RobotFrameworkAdvancedGuide < EMI < TWiki

${TEST STATUS} The status of the current test case, either PASS or FAIL. Test teardown
${TEST MESSAGE} The possible error message of the current test case. Test teardown
${PREV TEST The name of the previous test case, or an empty string if no tests have Everywhere
NAME} been executed yet.
${PREV TEST The status of the previous test case: either PASS, FAIL or an empty Everywhere
STATUS} string when no tests have been executed.
${PREV TEST The possible error message of the previous test case. Everywhere
MESSAGE}
${SUITE NAME} The full name of the current test suite. Everywhere
${SUITE SOURCE} An absolute path to the suite file or directory. Everywhere
${SUITE STATUS} The status of the current test case, either PASS or FAIL. Suite
teardown
${SUITE MESSAGE} The full message of the current test suite, including statistics. Suite
teardown
${OUTPUT FILE} An absolute path to the output file. Everywhere
${LOG FILE} An absolute path to the log file or string NONE when no log file is Everywhere
created.
${REPORT FILE} An absolute path to the report file or string NONE when no report is Everywhere
created.
${DEBUG FILE} An absolute path to the debug file or string NONE when no debug file Everywhere
is created.
${OUTPUT DIR} An absolute path to the output directory. Everywhere

Variable Priorities and Scopes

Variables coming from different sources have different priorities and are available in different scopes.

Variable Priorities
Variable priorities

• Variables from the command line

♦ Variables set in the command line have the highest priority of all variables that can be set
before the actual test execution starts. They override possible variables created in Variable
tables in test case files, as well as in resource and variable files imported in the test data.
♦ Individually set variables (--variable option) override the variables set using variable files
(--variablefile option). If you specify same individual variable multiple times, the one
specified last will override earlier ones. This allows setting default values for variables in a
start-up script and overriding them from the command line. Notice, though, that if multiple
variable files have same variables, the ones in the file specified first have the highest priority.
• Variable table in a test case file
♦ Variables created using the Variable table in a test case file are available for all the test cases
in that file. These variables override possible variables with same names in imported resource
and variable files. Variables created in the variable tables are available in all other tables in
the file where they are created. This means that they can be used also in the Setting table, for
example, for importing more variables from resource and variable files.
• Imported resource and variable files
♦ Variables imported from the resource and variable files have the lowest priority of all
variables created in the test data. Variables from resource files and variable files have the
same priority. If several resource and/or variable file have same variables, the ones in the file
imported first are taken into use.
♦ If a resource file imports resource files or variable files, variables in its own Variable table

Automatic Variables 23
 RobotFrameworkAdvancedGuide < EMI < TWiki
have a higher priority than variables it imports. All these variables are available for files that
import this resource file. Note that variables imported from resource and variable files are not
available in the Variable table of the file that imports them. This is due to the Variable table
being processed before the Setting table where the resource files and variable files are
imported.
• Variables set during test execution
♦ Variables set during the test execution either using return values from keywords or built-in
keywords Set Test/Suite/Global Variable always override possible existing variables in the
scope where they are set. In a sense they thus have the highest priority, but on the other hand
they do not affect variables outside the scope they are defined.
• Built-in variables
♦ Built-in variables like ${TEMPDIR} and ${TEST_NAME} have the highest priority of all
variables. They cannot be overridden using Variable table or from command line, but even
they can be reset during the test execution. An exception to this rule are number variables,
which are resolved dynamically if no variable is found otherwise. They can thus be
overridden, but that is generally a bad idea. Additionally ${CURDIR} is special because it is
replaced already during the test data processing time.

Variable Scopes
Depending on where and how they are created, variables can have a global, test suite, test case or user
keyword scope.

• Global scope
♦ Global variables are available everywhere in the test data. These variables are normally set
from the command line with the --variable and --variablefile options, but it is also possible to
create new global variables or change the existing ones with the BuiltIn keyword Set Global
Variable anywhere in the test data. Additionally also built-in variables are global.
♦ It is recommended to use capital letters with all global variables.
• Test suite scope
♦ Variables with the test suite scope are available anywhere in the test suite where they are
defined or imported. They can be created in Variable tables, imported from resource and
variable files, or set during the test execution using the BuiltIn keyword Set Suite Variable.
♦ The test suite scope is not recursive, which means that variables available in a higher-level
test suite are not available in lower-level suites. If necessary, resource and variable files can
be used for sharing variables.
♦ Since these variables can be considered global in the test suite where they are used, it is
recommended to use capital letters also with them.
• Test case scope
♦ Variables created in test cases from the return values of keywords have a test case scope and
they are available only in that test case. Another possibility to create them is using the BuiltIn
keyword Set Test Variable anywhere in that particular test case. Test case variables are local
and should use lower-case letters.
• User keyword scope
♦ User keywords get their own variables from arguments passed to them and return values from
the keywords they use. Also these variables are local and should use lower-case letters.

Advanced Variable Features

Extended variable syntax
Extended variable syntax can be used with objects set into scalar variables. It allows accessing the attributes
of the object (for example, ${obj.name} or ${obj.some_attr}), and even calling its methods (for example,

Variable Priorities 24
 RobotFrameworkAdvancedGuide < EMI < TWiki
${obj.get_name()} or ${obj.getSomething('arg')}). Extended variable syntax is a powerful feature, but it
should be used with care. Accessing attributes is normally not a problem, on the contrary, as one variable with
an object having several attributes is often better than having several variables. On the other hand, calling
methods, especially when they are used with arguments, can make the test data complicated. If that happens, it
is recommended to move the code into a test library. The most common usages of extended variable syntax
are illustrated in the example below. First assume that we have the following variable file and test case:

class MyObject:
def __init__(self, name):
self.name = name
def greet(self, who):
return '%s says hello to %s' % (self.name, who)
def __str__(self):
return self.name
OBJECT = MyObject('Robot')
DICTIONARY = { 1: 'one', 2: 'two', 3: 'three'}

Test Case Action Argument Argument

Example KW 1 ${OBJECT.name}
KW 2 ${OBJECT.greet('Fit')}
KW 3 ${DICTIONARY[2]}
When this test data is executed, the keywords get the arguments as explained below: KW 1 gets string Robot
KW 2 gets string Robot says hello to Fit KW 3 gets string two The extended variable syntax is evaluated in
the following order:

• The variable is searched using the full variable name. The extended variable syntax is evaluated only
if no matching variable is found.
• The real name of the base variable is created. The body of the name consists of all the characters after
${ until the first occurrence of a non-alphanumeric character or a space (for example, OBJECT in
${OBJECT.name} and DICTIONARY in ${DICTIONARY[2]}).
• A variable matching the body is searched. If there is no match, an exception is raised and the test case
fails.
• The expression inside the curly brackets is evaluated as a Python expression, so that the base variable
name is replaced with its value. If the evaluation fails because of an invalid syntax or that the queried
attribute does not exist, an exception is raised and the test fails.
• The whole extended variable is replaced with the value returned from the evaluation.

Many standard Python objects, including strings and numbers, have methods that can be used with the
extended variable syntax either explicitly or implicitly. Sometimes this can be really useful and reduce the
need for setting temporary variables, but it is also easy to overuse it and create really cryptic test data.
Following examples show few pretty good usages. Example:

Test Case Action Argument Argument

String ${string} = Set Variable abc
Log ${string.upper()} # Logs 'ABC'
Log ${string * 2} # Logs 'abcabc'
Number ${number} = Set Variable ${-2}
Log ${number * 10} # Logs -20
Log ${number.__abs__()} # Logs 2
Note that even though abs(number) is recommended over number.__abs__() in normal Python code, using
${abs(number)} does not work. This is because the variable name must be in the beginning of the extended
syntax. Using xxx methods in the test data like this is already a bit questionable, and it is normally better to
move this kind of logic into test libraries.

Extended variable syntax 25

 RobotFrameworkAdvancedGuide < EMI < TWiki

Variables inside variables

Variables are allowed also inside variables, and when this syntax is used, variables are resolved from the
inside out. For example, if you have a variable ${var${x}}, then ${x} is resolved first. If it has the value
name, the final value is then the value of the variable ${varname}. There can be several nested variables, but
resolving the outermost fails, if any of them does not exist. In the example below, Do X gets the value
${JOHN HOME} or ${JANE HOME}, depending on if Get Name returns john or jane. If it returns something
else, resolving ${${name} HOME} fails. Example:

Variable Value
${JOHN HOME} /home/john
${JANE HOME} /home/jane
Test Case Action Argument
Example ${name} = Get Name
Do X ${${name} HOME}

Variables inside variables 26

Creating User Keywords
Basic Syntax
In many ways, the overall user keyword syntax is identical to the test case syntax. User keywords are created
in keyword tables which differ from test case tables only by the name that is used to identify them. User
keyword names are in the first column similarly as test cases names. Also user keywords are created from
keywords, either from keywords in test libraries or other user keywords. Keyword names are normally in the
second column, but when setting variables from keyword return values, they are in the subsequent columns.
Example:

Keyword Action Argument Argument

Open Login Page Open Browser http://host/login.html
Title Should Be Login Page

Title Should Start With [Arguments] ${expected}

${title} = Get Title
Should Start With ${title} ${expected}
Most user keywords take some arguments. This important feature is used already in the second example
above, and it is explained in detail later in this section, similarly as user keyword return values. User
keywords can be created in test case files, resource files, and test suite initialization files. Keywords created in
resource files are available for files using them, whereas other keywords are only available in the files where
they are created.

Settings in the Keyword Table

User keywords can have similar settings as test cases, and they have the same square bracket syntax
separating them from keyword names. All available settings are listed below and explained later in this
section.

• [Documentation] Used for setting a user keyword documentation.

• [Arguments] Specifies user keyword arguments.
• [Return] Specifies user keyword return values.
• [Teardown] Specify keyword teardown.
• [Timeout] Sets the possible user keyword timeout. Timeouts are discussed in a section of their own.

User Keyword Arguments

Most user keywords need to take some arguments. The syntax for specifying them is probably the most
complicated feature normally needed with Robot Framework, but even that is relatively easy, particularly in
most common cases. Arguments are normally specified with the [Arguments] setting, and argument names
use the same syntax as variables, for example ${arg}.

Positional arguments
The simplest way to specify arguments (apart from not having them at all) is using only positional arguments.
In most cases, this is all that is needed. The syntax is such that first the [Arguments] setting is given and then
argument names are defined in the subsequent cells. Each argument is in its own cell, using the same syntax
as with variables. The keyword must be used with as many arguments as there are argument names in its
signature. The actual argument names do not matter to the framework, but from users' perspective they should
should be as descriptive as possible. It is recommended to use lower-case letters in variable names, either as

Creating User Keywords 27

 RobotFrameworkAdvancedGuide < EMI < TWiki
${my_arg}, ${my arg} or ${myArg}.

Keyword Action Argument Argument Argument

One Argument [Arguments] ${arg_name}
Log Got argument ${arg_name}

Three Arguments [Arguments] ${arg1} ${arg2} ${arg3}

Log 1st argument: ${arg1}
Log 2nd argument: ${arg2}
Log 3rd argument: ${arg3}
Default values and variable numbers of arguments can be used as well.Example:
Keyword Action Argument Argument Argument
Default Arguments [Arguments] ${arg1}=val1 ${arg2} ${arg3}
Do something ${arg1} ${arg2} ${arg3}

Var Arguments [Arguments] @{varargs}

Do something @{varargs}

Combi Arguments [Arguments] ${req} ${opt}=1924 @{rest}

Do something ${req} ${opt} @{rest}

Return Values
Similarly as library keywords, also user keywords can return values. Return values are defined with the
[Return] setting. The values can then be assigned to variables in test cases or other user keywords. In a typical
case, a user keyword returns one value and it can be set to a scalar variable. This is done by having the return
value in the next cell after the [Return] setting. User keywords can also return several values, which can then
be assigned into several scalar variables at once, to a list variable, or to scalar variables and a list variable.
Several values can be returned simply by specifying those values in different cells after the [Return] setting.
Example:

Keyword Action Argument Argument Argument

Return One Value [Arguments] ${arg}
Do Something ${arg}
${value} = Get Some Value
[Return] ${value}

Return Three Values [Return] foo bar zap

Positional arguments 28
Resource and Variable Files
User keywords and variables in test case files and test suite initialization files can only be used in files where
they are created, but resource files provide a mechanism for sharing them. Since the resource file structure is
very close to test case files, it is easy to create them. Variable files provide a powerful mechanism for creating
and sharing variables. For example, they allow values other than strings and enable creating variables
dynamically. Their flexibility comes from the fact that they are created using Python code, which also makes
them somewhat more complicated than Variable tables.

Resource Files
How to use
Resource files are imported using the Resource setting in the Settings table. The path to the resource file is
given in the cell after the setting name. If the path is given in an absolute format, it is used directly. In other
cases, the resource file is first searched relatively to the directory where the importing file is located. If the file
is not found there, it is then searched from the directories in PYTHONPATH. The path can contain variables,
and it is recommended to use them to make paths system-independent (for example,
${RESOURCES}/login_resources.html or ${RESOURCE_PATH}). Additionally, slashes ("/") in the path are
automatically changed to backslashes ("\") on Windows. Example:

Setting Value
Resource myresources.html
Resource ../data/resources.html
Resource ${RESOURCES}/common.tsv
The user keywords and variables defined in a resource file are available in the file that takes that resource file
into use. Similarly available are also all keywords and variables from the libraries, resource files and variable
files imported by the said resource file.

Resource File Structure

The higher-level structure of resource files is the same as that of test case files otherwise, but, of course, they
cannot contain Test Case tables. Additionally, the Setting table in resource files can contain only import
settings (Library, Resource, Variables) and Documentation. The Variable table and Keyword table are used
exactly the same way as in test case files. If several resource files have a user keyword with the same name,
they must be used so that the keyword name is prefixed with the resource file name without the extension (for
example, myresources.Some Keyword and common.Some Keyword). Moreover, if several resource files
contain the same variable, the one that is imported first is taken into use.

Documenting Resource Files

Keywords created in a resource file can be documented using [Documentation] setting. Also the resource file
itself can have Documentation in the Setting table similarly as test suites. Both libdoc.py and RIDE use these
documentations, and they are naturally available for anyone opening resource files. The first line of the
documentation of a keyword is logged when it is run, but otherwise resource file documentations are ignored
during the test execution. Example Resource File:

Setting Value
Documentation An example resource file
Library SeleniumLibrary
Resource ${RESOURCES}/common.html
Variable Value

Resource and Variable Files 29

 RobotFrameworkAdvancedGuide < EMI < TWiki

${HOST} localhost:7272
${LOGIN_URL} http://${HOST}/
${WELCOME_URL} http://${HOST}/welcome.html
${BROWSER} Firefox
Keyword Action Argument Argument
Open Login Page [Documentation] Opens browser to login page
Open Browser ${LOGIN_URL} ${BROWSER}
Title Should Be Login Page

Input Name [Arguments] ${name}

Input Text username_field ${name}

Input Password [Arguments] ${password}

Input Text password_field ${password}

Variable Files
Variable files contain variables that can be used in test data. Variables can also be created using variable
tables or set from the command line, but variable files allow creating them dynamically and their variables can
contain any objects. Variable files are created using Python code, and technically they are Python modules.
There are two different possibilities for creating variables: Creating variables directly Variables are specified
as module attributes. In simple cases, the syntax is so simple that no real programming is needed. For
example, MY_VAR = 'my value' creates a variable ${MY_VAR} with the specified text as the value. Getting
variables from a special function Alternatively, variable files can have a special get_variables (or
getVariables) method that returns variables as either a Python dictionary or a Java Map. This method can even
take arguments, which makes this approach quite flexible.

How to include Variable Files

All test data files can import variables using the Variables setting in the Setting table, in the same way as
resource files are imported using the Resource setting. Similarly to resource files, the path to the imported
variable file is considered relative to the directory where the importing file is, and if not found, it is searched
from the directories in PYTHONPATH. The path can also contain variables, and slashes are converted to
backslashes on Windows. If an argument file takes arguments, they are specified in the cells after the path and
also they can contain variables. Example:

Setting Value Value Value

Variables myvariables.py
Variables ../data/variables.py
Variables ${RESOURCES}/common.py
Variables taking_arguments.py arg1 ${ARG2}
All variables from a variable file are available in the test data file that imports it. If several variable files are
imported and they contain a variable with the same name, the one in the earliest imported file is taken into
use. Additionally, variables created in Variable tables and set from the command line override variables from
variable files. Another way to take variable files into use is using the command line option --variablefile.
Variable files are referenced using a path to them, and possible arguments are joined to the path with a colon
(:): --variablefile myvariables.py --variablefile path/variables.py --variablefile /absolute/path/common.py
--variablefile taking_arguments.py:arg1:arg2 Variables in these files are globally available in all test data
files, similarly as individual variables set with the --variable option. If both --variablefile and --variable
options are used and there are variables with same names, those that are set individually with --variable option
take precedence.

Documenting Resource Files 30

 RobotFrameworkAdvancedGuide < EMI < TWiki

Creating Variables Directly

Basic Syntax

When variable files are taken into use, they are imported as Python modules and all their global attributes that
do not start with an underscore ( ) are considered to be variables. Because variable names are case-insensitive,
both lower- and upper-case names are possible, but in general, capital letters are recommended for global
variables and attributes.

Example:

VARIABLE = "An example string" ANOTHER_VARIABLE = "This is pretty easy!" INTEGER = 42

STRINGS = ["one", "two", "kolme", "four"] NUMBERS = [1, INTEGER, 3.14] In the example above,
variables ${VARIABLE}, ${ANOTHER_VARIABLE}, and so on, are created. The first two variables are
strings, the third one is an integer and the last two are lists. All these variables are scalar variables, even the
ones containing lists as values. To create list variables, the variable name must be prefixed with LIST_ (note
the two underscores). Example: LIST__STRINGS = ["list", "of", "strings"] LIST__MIXED = ["first value",
-1.1, None, True] The variables in both the examples above could be created also using the Variable table
below.

Variable Value Value Value *Value

${VARIABLE} An example string
${ANOTHER_VARIABLE} This is pretty easy!
${INTEGER} ${42}
${STRINGS} one two three four
${NUMBERS} ${1} ${INTEGER} ${3.14}
@{STRINGS} list of strings
@{MIXED} first value ${-1.1} ${None} ${True}

Using Objects as values

Variables in variable files are not limited to having only strings or other base types as values like variable
tables. Instead, their variables can contain any objects. In the example below, the variable ${TELEPHONES}
is a python dictionary containing two pairs of values: TELEPHONES = {'jack': 4098, 'sape': 4139}

Creating Variables Dynamically

Because variable files are created using a real programming language, they can have dynamic logic for setting
variables.

import os
import random
import time
USER = os.getlogin() # current login name
RANDOM_INT = random.randint(0, 10) # random integer in range [0,10]
CURRENT_TIME = time.asctime() # timestamp like 'Thu Apr 6 12:45:21 2006'
if time.localtime()[3] > 12:
AFTERNOON = True
else:
AFTERNOON = False

The example above uses standard Python libraries to set different variables, but you can use your own code to
construct the values. The example below illustrates the concept, but similarly, your code could read the data
from a database, from an external file or even ask it from the user.

import math

Creating Variables Directly 31

 RobotFrameworkAdvancedGuide < EMI < TWiki
def get_area(diameter):
radius = diameter / 2
area = math.pi * radius * radius
return area
AREA1 = get_area(1)
AREA2 = get_area(2)

Selecting which Variables to Include

When Robot Framework processes variable files, all their attributes that do not start with an underscore are
expected to be variables. This means that even functions or classes created in the variable file or imported
from elsewhere are considered variables. For example, the last example would contain the variables ${math}
and ${get_area} in addition to ${AREA1} and ${AREA2}. Normally the extra variables do not cause
problems, but they could override some other variables and cause hard-to-debug errors. One possibility to
ignore other attributes is prefixing them with an underscore:

import math as _math

def _get_area(diameter):
radius = diameter / 2.0
area = _math.pi * radius * radius
return area
AREA1 = _get_area(1)
AREA2 = _get_area(2)

If there is a large number of other attributes, instead of prefixing them all, it is often easier to use a special
attribute all and give it a list of attribute names to be processed as variables.

import math
__all__ = ['AREA1', 'AREA2']
def get_area(diameter):
radius = diameter / 2.0
area = math.pi * radius * radius
return area
AREA1 = get_area(1)
AREA2 = get_area(2)

Note that the all attribute is also, and originally, used by Python to decide which attributes to import when
using the syntax "from modulename import *".

Getting Variables from a Special Function

An alternative syntax for getting variables is having a special get_variables function (also camelCase syntax
getVariables is possible) in the variable file. In this case, Robot Framework calls that function and it returns
variables as a Python dictionary or its subclass or a Java Map, with variable names as keys and variable values
as values. Variables are considered to be scalars, unless prefixed with LIST__, and values can contain
anything. The example below is identical to the first examples of creating variables directly.

def get_variables():
variables = { "VARIABLE ": "An example string",
"ANOTHER_VARIABLE": "This is pretty easy!",
"INTEGER": 42,
"STRINGS": ["one", "two", "kolme", "four"],
"NUMBERS": [1, 42, 3.14],
"LIST__STRINGS": ["list", "of", "strings"],
"LIST__MIXED": ["first value", -1.1, None, True] }
return variables

get_variables can also take arguments, which facilitates changing what variables actually are created.
Arguments to the function are set just as any other arguments for a Python function. When taking variable
files into use in the test data, arguments are specified in cells after the path to the variable file, and in the

Creating Variables Dynamically 32

 RobotFrameworkAdvancedGuide < EMI < TWiki

command line they are separated from the path with a colon. The dummy example below shows how to use
arguments with variable files. In a more realistic example, the argument could be a path to an external text file
or database where to read variables from.

variables1 = { 'scalar': 'Scalar variable',

'LIST__list': ['List','variable'] }
variables2 = { 'scalar' : 'Some other value',
'LIST__list': ['Some','other','value'],
'extra': 'variables1 does not have this at all' }
def get_variables(arg):
if arg == 'one':
return variables1
else:
return variables2

The most notable benefit of using get_variables instead of defining variables directly as global attributes of
the variable file is the ability to use arguments. On the other hand, the main drawback is that this approach
always requires some actual programming.

Getting Variables from a Special Function 33

Advanced Features
Handling keywords with same names
Keywords that are used with Robot Framework are either library keywords or user keywords. The former
come from standard libraries or external libraries, and the latter are either created in the same file where they
are used or then imported from resource files. When many keywords are in use, it is quite common that some
of them have the same name, and this section describes how to handle possible conflicts in these situations.

Keyword scopes
When only a keyword name is used and there are several keywords with that name, Robot Framework
attempts to determine which keyword has the highest priority based on its scope. The keyword's scope is
determined on the basis of how the keyword in question is created:

• Created as a user keyword in the same file where it is used. These keywords have the highest priority
and they are always used, even if there are other keywords with the same name elsewhere.
• Created in a resource file and imported either directly or indirectly from another resource file. This is
the second-highest priority.
• Created in an external test library. These keywords are used, if there are no user keywords with the
same name. However, if there is a keyword with the same name in the standard library, a warning is
displayed.
• Created in a standard library. These keywords have the lowest priority.

Specifying a keyword explicitly

Scopes alone are not a sufficient solution, because there can be keywords with the same name in several
libraries or resources, and thus, they provide a mechanism to use only the keyword of the highest priority. In
such cases, it is possible to use the full name of the keyword, where the keyword name is prefixed with the
name of the resource or library and a dot is a delimiter. With library keywords, the long format means only
using the format LibraryName.Keyword Name. For example, the keyword Run from the OperatingSystem
library could be used as OperatingSystem.Run, even if there was another Run keyword somewhere else. If the
library is in a module or package, the full module or package name must be used (for example,
com.company.Library.Some Keyword). If a custom name is given to a library using the WITH NAME syntax,
the specified name must be used also in the full keyword name. Resource files are specified in the full
keyword name, similarly as library names. The name of the resource is derived from the basename of the
resource file without the file extension. For example, the keyword Example in a resource file
myresources.html can be used as myresources.Example. Note that this syntax does not work, if several
resource files have the same basename. In such cases, either the files or the keywords must be renamed. The
full name of the keyword is case-, space- and underscore-insensitive, similarly as normal keyword names.

Specifying library search order

It is also possible to use the keyword Set Library Search Order from BuiltIn library to specify the priority
order between test libraries that are in use. If multiple keywords with the same name are found, libraries
implementing them are gone through in the library search order. The first library containing the keyword is
selected and the keyword from that library is used. If keyword is not found, test execution fails. There is no
need to use the long format LibraryName.Keyword Name notation when library search order is set. This
makes it easier to to use multiple instances of the same library as Set Library Search Order keyword can be
used to switch between the libraries. This enables using same user keywords instead of duplicating those for
all the library instances. Library search order is valid in the suite level meaning it is available only in the suite
where it was set.

Advanced Features 34
 RobotFrameworkAdvancedGuide < EMI < TWiki

Timeouts
Warning: Using timeouts might cause test to be executed slower if the Python interpreter is used. The slowing
down is caused by Python's threading implementation. Keywords may be problematic in situations where they
take exceptionally long to execute or just hang endlessly. Robot Framework allows you to set timeouts both
for test cases and user keywords, and if a test or keyword is not finished within the specified time, the
keyword that is currently being executed is forcefully stopped. Stopping keywords in this manner may leave
the library or system under test to an unstable state, and timeouts are recommended only when there is no
safer option available. In general, libraries should be implemented so that keywords cannot hang or that they
have their own timeout mechanism, if necessary.

Test Case Timeout

The test case timeout can be set either by using the Test Timeout setting in the Setting table or the [Timeout]
setting in the Test Case table. Test Timeout in the Setting table defines a default test timeout value for all the
test cases in the test suite, whereas [Timeout] in the Test Case table applies a timeout to an individual test case
and overrides the possible default value. Using an empty [Timeout] means that the test has no timeout even
when Test Timeout is used. It is also possible to use value NONE for this purpose. Regardless of where the
test timeout is defined, the first cell after the setting name contains the duration of the timeout. The duration
must be given in Robot Framework's time format, that is, either directly in seconds or in a format like 1
minute 30 seconds. It must be noted that there is always some overhead by the framework, and timeouts
shorter than one second are thus not recommended. The default error message displayed when a test timeout
occurs is Test timeout <time> exceeded. It is also possible to use custom error messages, and these messages
are written into the cells after the timeout duration. The message can be split into multiple cells, similarly as
documentations. Both the timeout value and the error message may contain variables. If there is a timeout, the
keyword running is stopped at the expiration of the timeout and the test case fails. However, keywords
executed as test teardown are not interrupted if a test timeout occurs, because they are normally engaged in
important clean-up activities. If necessary, it is possible to interrupt also these keywords with user keyword
timeouts. Test Case timeout examples:

Setting Value Value

Test Timeout 2 minutes
Test Case Action Argument Argument Argument
Default [Documentation] Timeout from the Setting table is
Timeout used
Some Keyword argument

Override [Documentation] Override default, use 10 seconds

timeout
[Timeout] 10
Some Keyword argument

Custom [Documentation] Override default and use custom

Message message
[Timeout] 1min 10s This is my custom It continues
error. here.
Some Keyword argument

Variables [Documentation] It is possible to use variables too

[Timeout] ${TIMEOUT}
Some Keyword argument

Timeouts 35
 RobotFrameworkAdvancedGuide < EMI < TWiki

No Timeout [Documentation] Empty timeout means no timeout Test Timeout has been
even when used
[Timeout]
Some Keyword argument

No Timeout 2 [Documentation] Empty timeout using NONE

[Timeout] NONE
Some Keyword argument

User Keyword Timeout

A timeout can be set for a user keyword using the [Timeout] setting in the Keyword table. The syntax for
setting it, including how timeout values and possible custom messages are given, is identical to the syntax
used with test case timeouts. If no custom message is provided, the default error message Keyword timeout
exceeded is used if a timeout occurs. Example:

Keyword Action Argument Argument

Timed Keyword [Documentation] Set only the timeout value and not the custom message.
[Timeout] 1 minute 42 seconds
Do Something
Do Something Else

Timed-out Wrapper [Arguments] @{args}

[Documentation] This keyword is a wrapper that adds a timeout to another keyword.
[Timeout] 2 minutes Original Keyword didn't finish in 2
minutes
Original Keyword @{args}
A user keyword timeout is applicable during the execution of that user keyword. If the total time of the whole
keyword is longer than the timeout value, the currently executed keyword is stopped. User keyword timeouts
are applicable also during a test case teardown, whereas test timeouts are not. If both the test case and some of
its keywords (or several nested keywords) have a timeout, the active timeout is the one with the least time left.
Warning: Using timeouts might cause test to be executed slower if the Python interpreter is used. The slowing
down is caused by Python's threading implementation.

For loops
Repeating same actions several times is quite a common need in test automation. With Robot Framework, test
libraries can have any kind of loop constructs, and most of the time loops should be implemented in them.
Robot Framework also has its own For loop syntax, which is useful, for example, when there is a need to
repeat keywords from different libraries. For loops can be used with both test cases and user keywords.
Except for really simple cases, user keywords are better, because they hide the complexity introduced by for
loops. The basic for loop syntax, FOR item IN sequence, is derived from Python, but similar syntax is
possible also in shell scripts or Perl.

Normal for loop

In a normal For loop, one variable is assigned from a list of values, one value per iteration. The syntax starts
with :FOR, where colon is required to separate the syntax from normal keywords. The next cell contains the
loop variable, the subsequent cell must have IN, and the final cells contain values over which to iterate. The
keywords used in the For loop are on the next rows and they must be indented one cell to the right. The For
loop ends when the indentation returns back to normal or the table ends. Having nested For loops directly is
not supported, but it is possible to use a user keyword inside a For loop and have another For loop there.

Test Case Timeout 36

 RobotFrameworkAdvancedGuide < EMI < TWiki
Example:

Test Case Action Argument Argument Argument Arguments

Example 1 :FOR ${animal} IN cat dog
Log ${animal}
Log 2nd keyword
Log Outside loop

Example 2 :FOR ${var} IN one two

... three four five six
... seven
Log ${var}
The For loop in Example 1 above is executed twice, so that first the loop variable ${animal} has the value cat
and then dog. The loop consists of two Log keywords. In the second example, loop values are split into
several rows and the loop is run altogether seven times. For loops are most useful and also clearest when they
are used with list variables. This is illustrated by the example below, where @{ELEMENTS} contains an
arbitrarily long list of element names and keyword Start Element is used with all of them. Example:
Test Case Action Argument Argument Argument Arguments
Example :FOR ${element} IN @{ELEMENTS}
Start Element ${element}

Using several loop variables

It is also possible to use several loop variables. The syntax is the same as with the normal For loop, but all
loop variables are listed in the cells between :FOR and IN. There can be any number of loop variables, but the
number of values must be evenly dividable by the number of variables. This syntax naturally works both with
and without list variables. In the former case, it is often possible to organize loop values below loop variables,
as in the first part of the example below:

Test Case Action Argument Argument Argument Arguments

Example :FOR ${index} ${english} ${greek} IN
... 1 cat gata
... 2 dog skilos
... 3 horse alogo
Do X ${english}
Y Should Be ${greek} ${index}
:FOR ${name} ${id} IN @{EMPLOYERS}
Create ${name} ${id}

For in range
Earlier For loops always iterated over a sequence, and this is also the most common use case. Sometimes it is
still convenient to have a For loop that is executed a certain number of times, and Robot Framework has a
special FOR index IN RANGE limit syntax for this purpose. This syntax is derived from the similar Python
idiom. Similarly as other For loops, the For in range loop starts with :FOR and the loop variable is in the next
cell. In this format there can be only one loop variable and it contains the current loop index. The next cell
must contain IN RANGE and the subsequent cells loop limits. In the simplest case, only the upper limit of the
loop is specified. In this case, loop indexes start from zero and increase by one until, but excluding, the limit.
It is also possible to give both the start and end limits. Then indexes start from the start limit, but increase
similarly as in the simple case. Finally, it is possible to give also the step value that specifies the increment to
use. If the step is negative, it is used as decrement. It is also possible to use simple arithmetics such as
addition and subtraction with the range limits. This is especially useful when the limits are specified with

Normal for loop 37

 RobotFrameworkAdvancedGuide < EMI < TWiki
variables. Example:

Test Case Action Argument Argument Arg Arg Arg

Only upper limit [Documentation] Loops over values from 0 to 9
:FOR ${index} IN RANGE 10
Log ${index}

Start and end [Documentation] Loops over values from 1 to 10

:FOR ${index} IN RANGE 1 11
Log ${index}

Also step given [Documentation] Loops over values 5, 15, and 25

:FOR ${index} IN RANGE 5 26 10
Log ${index}

Negative step [Documentation] Loops over values 13, 3, and -7

:FOR ${index} IN RANGE 13 -13 -10
Log ${index}

Arithmetics [Documentation] Arithmetics with variable

:FOR ${index} IN RANGE ${var}+1
Log ${index}

Exiting for loop

Normally for loop is executed until all the elements of the loop have been looped through or an error occurs
and the test case or keyword fails. However sometimes execution of a for loop needs to be stopped before all
elements of the loop have been gone through. BuiltIn keyword Exit For Loop can be used to exit the enclosing
for loop. Exit For Loop keyword can be used directly in a for loop or in a keyword that the for loop uses. In
both cases the test execution continues after the for loop. If executed outside of a for loop, the test fails.
Example:

Test Case Action Argument Argument Argument

Exit Example :FOR ${var} IN @{SOME
LIST}
Run Keyword If '${var}' == 'EXIT' Exit For
Loop
Do Something ${var}
Exiting a for loop can also be initiated from a keyword in a test library by raising an exception with
ROBOT_EXIT_FOR_LOOP attribute. Please see Stopping test execution for examples how to do this in
Python and Java libraries.

Repeating single keyword

For loops can be excessive in situations where there is only a need to repeat a single keyword. In these cases it
is often easier to use BuiltIn keyword Repeat Keyword. This keyword takes a keyword and how many times
to repeat it as arguments. The times to repeat the keyword can have an optional postfix times or x to make the
syntax easier to read. Example:

Test Case Action Argument Argument Argument Argument

Example Repeat Keyword 5 Some Keyword arg1 arg2
Repeat Keyword 42 times My Keyword

For in range 38
 RobotFrameworkAdvancedGuide < EMI < TWiki

Repeat Keyword ${var} Another KW argument

Conditional execution
In general, it is not recommended to have conditional logic in test cases, or even in user keywords, because it
can make them hard to understand and maintain. Instead, this kind of logic should be in test libraries, where it
can be implemented using natural programming language constructs. However, some conditional logic can be
useful at times, and even though Robot Framework does not have an actual if/else construct, there are several
ways to get the same effect:

• The name of the keyword used as a setup or a teardown of both test cases and test suites can be
specified using a variable. This facilitates changing them, for example, from the command line.
• The BuiltIn keyword Run Keyword takes a keyword to actually execute as an argument, and it can
thus be a variable. The value of the variable can, for example, be got dynamically from an earlier
keyword or given from the command line.
• The BuiltIn keywords Run Keyword If and Run Keyword Unless execute a named keyword only if a
certain expression is true or false, respectively. They are ideally suited to creating simple if/else
constructs. For an example, see the documentation of the former.
• Another BuiltIn keyword, Set Variable If, can be used to set variables dynamically based on a given
expression.

There are several BuiltIn keywords that allow executing a named keyword only if a test case or test suite has
failed or passed.

Repeating single keyword 39

Executing Test Cases
Basic Usage
Robot Framework test cases are executed from the command line, and the end result is, by default, an output
file in XML format and an HTML report and log. After the execution, output files can be combined and
otherwise post-processed with the rebot tool.

Starting the Test Execution

Synopsis

pybot [options] datasources

jybot [options] datasources
interpreter /path/to/robot/runner.py [options] datasources
java -jar robotframework-

Runner Scripts

Test execution is normally started with the pybot or jybot commands. These commands are otherwise
identical, but the former executes tests using the Python interpreter and the latter uses Jython. Which one to
use depends on the needed test libraries. Some libraries use modules or syntax available only on Python,
others use Java-based tools that require Jython, and some work on both. If you can use either pybot or jybot,
the former is recommended, as Python is somewhat faster than Jython. It is also possible to get Robot
Framework as a single jar file. In this case, Jython is bundled inside the jar, and the above discussion about
Jython applies. Normal arguments to Robot Framework are given after the jar file name. Another possibility
for starting the test execution is running the runner.py script under the installed robot module directly. This
method allows selecting the interpreter and setting command line options to it freely. The most common use
case is altering the options controlling JVM maximum memory consumption. Regardless of the runner script,
the path (or paths) to the test data to be executed is given as an argument. Additionally, different command
line options can be used to alter the test execution or generated outputs in some way.

Specifying Test Data

Robot Framework test cases are created in files and directories, and they are executed by giving the path to the
file or directory in question to the selected runner script. The path can be absolute or, more commonly,
relative to the directory where tests are executed from. The given file or directory creates the top-level test
suite, which gets its name, unless overridden with the --name option, from the file or directory name.
Different execution possibilities are illustrated in the examples below. Note that in these examples, as well as
in other examples in this section, only the pybot command is used, but jybot or a custom runner script could
be used similarly.

pybot test_cases.html
pybot path/to/my_tests/
pybot c:\robot\tests.txt

It is also possible to give paths to several test case files or directories at once, separated with spaces. In this
case, Robot Framework creates the top-level test suite automatically, and the specified files and directories
become its child test suites. The name of the created test suite is got from child suite names by catenating
them together with an ampersand (&) and spaces. For example, the name of the top-level suite in the first
example below is My Tests & Your Tests. These automatically created names are often quite long and
complicated. In most cases, it is thus better to use the --name option for overriding it, as in the second
example below:

Executing Test Cases 40

 RobotFrameworkAdvancedGuide < EMI < TWiki
pybot my_tests.html your_tests.html
pybot --name Example path/to/tests/pattern_*.html

Using Command Line Options

Robot Framework provides a number of command line options that can be used to control how test cases are
executed and what outputs are generated. This section explains the option syntax, and what options actually
exist. How they can be used is discussed elsewhere in this chapter.

Using Options

When options are used, they must always be given between the runner script and the data sources. For
example:

pybot -L debug my_tests.txt

pybot --include lib_name --variable HOST:10.0.0.42 path/to/tests/

Option Values Patterns

Many of the options take arguments as simple patterns. This means that * and ? can be used as special
characters, so that the former matches any string (even an empty string) and the latter matches any single
character. For example, --include prefix-* matches all tags starting with prefix-, and --include a??? matches
any tag that is four characters long and starts with a character a.

Test Results
Command Line Output

The most visible output from test execution is the output displayed in the command line. All executed test
suites and test cases, as well as their statuses, are shown there in real time. The example below shows the
output from executing a simple test suite with only two test cases:

==============================================================================
Example test suite
==============================================================================
First test :: Possible test documentation | PASS |
------------------------------------------------------------------------------
Second test | FAIL |
Error message is displayed here
==============================================================================
Example test suite | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Output: /path/to/output.xml
Report: /path/to/report.html
Log: /path/to/log.html

Generated Output Files

The command line output is very limited, and separate output files are normally needed for investigating the
test results. As the example above shows, three output files are generated by default. The first one is in XML
format and contains all the information about test execution. The second is a higher-level report and the third
is a more detailed log file. These files and other possible output files are discussed in more detail in the
section Different output files.

Specifying Test Data 41

 RobotFrameworkAdvancedGuide < EMI < TWiki
Return Codes

Runner scripts communicate the overall test execution status to the system running them using return codes.
When the execution starts successfully and no critical test fail, the return code is zero. All possible return
codes are explained in the table below.

RC Explanation
0 All critical tests passed.
1-249 Returned number of critical tests failed.
250 250 or more critical failures.
251 Help or version information printed.
252 Invalid test data or command line options.
253 Test execution stopped by user.
255 Unexpected internal error.
The return code can be set to 0 even if there are critical failures using the --NoStatusRC command line option.
This might be useful, for example, if post-processing of the results is needed before the overall status of test
execution can be determined.

Help and Version Info

Both when executing test cases with pybot or jybot and when post-processing reports with rebot, it is possible
to get command line help with the option --help and its short version -h. These help texts have a short general
overview and briefly explain the available command line options. All runner scripts also support getting the
version information with the option --version. This information also contains Python or Jython version and the
platform type.

Debugging Execution Problems

A test case can fail because the system under test does not work correctly, in which case the test has found a
bug, or because the test itself is buggy. The error message explaining the failure is shown on the command
line output and in the report file, and sometimes the error message alone is enough to pinpoint the problem.
More often that not, however, log files are needed because they have also other log messages and they show
which keyword actually failed. When a failure is caused by the tested application, the error message and log
messages ought to be enough to understand what caused it. If that is not the case, the test library does not
provide enough information and needs to be enhanced. In this situation running the same test manually, if
possible, may also reveal more information about the issue. Failures caused by test cases themselves or by
keywords they use can sometimes be hard to debug. If the error message, for example, tells that a keyword is
used with wrong number of arguments fixing the problem is obviously easy, but if a keyword is missing or
fails in unexpected way finding the root cause can be harder. The first place to look for more information is
the execution errors section in the log file. For example, an error about a failed test library import may well
explain why a test has failed due to a missing keyword. If the log file does not provide enough information by
default, it is possible to execute tests with a lower log level. For example tracebacks showing where in the
code the failure occurred are logged using the DEBUG level, and this information is invaluable when the
problem is in an individual keyword. If the log file still does not have enough information, it is a good idea to
enable the syslog and see what information it provides. It is also possible to add some keywords to the test
cases to see what is going on. Especially BuiltIn keywords Log and Log Variables are useful.

Test Execution
Execution Flow

Return Codes 42
 RobotFrameworkAdvancedGuide < EMI < TWiki
Execution Suites and Tests

Test cases are always executed within a test suite. A test suite created from a test case file has tests directly,
whereas suites created from directories have child test suites which either have tests or their own child suites.
By default all the tests in an executed suite are run, but it is possible to select tests using options --test, --suite,
--include and --exclude. Suites containing no tests are ignored. The execution starts from the top-level test
suite. If the suite has tests they are executed one-by-one, and if it has suites they are executed recursively in
depth-first order. When an individual test case is executed, the keywords it contains are run in a sequence.
Normally the execution of the current test ends if any of the keywords fails, but it is also possible to continue
after failures. The exact execution order and how possible setups and teardowns affect the execution are
discussed in the following sections.

Setups and Teardowns

Setups and teardowns can be defined on test case and test suite level.

• Suite Setup
♦ If a test suite has a setup, it is executed before its tests and child suites. If the suite setup
passes, test execution continues normally. If it fails, all the test cases the suite and its child
suites are marked failed with a message Setup of the parent suite failed.. The tests and
possible suite setups and teardowns in the child test suites are not executed.
♦ Suite setups are often used for setting up the test environment. Because tests are not run if the
suite setup fails, it is easy to use suite setups for verifying that the environment is in state in
which the tests can be executed.
• Suite Teardown
♦ If a test suite has a teardown, it is executed after all its test cases and child suites. Suite
teardowns are executed regardless of the test status and even if the matching suite setup fails.
If the suite teardown fails, all tests in the suite are marked failed afterwards in reports and
logs.
♦ Suite teardowns are mostly used for cleaning up the test environment after the execution. To
ensure that all these tasks are done, all the keywords used in the teardown are executed even
if some of them fail.
• Test Setup
♦ Possible test setup is executed before the keywords of the test case. If the setup fails, the
keywords are not executed. The main use for test setups is setting up the environment for that
particular test case.
• Test Teardown
♦ Possible test teardown is executed after the test case has been executed. It is executed
regardless of the test status and also if the setup of the test has failed.
♦ Similarly as suite teardown, test teardowns are used mainly for cleanup activities. Also they
are executed fully even if some of their keywords fail.

Execution Order

Test cases in a test suite are executed in the same order as they are defined in the test case file. Test suites
inside a higher level test suite are executed in case-insensitive alphabetical order based on the file or directory
name. If multiple files and/or directories are given from the command line, they are executed in the order they
are given. If there is a need to use certain test suite execution order inside a directory, it is possible to add
prefixes like 01 and 02 into file and directory names. Such prefixes are not included in the generated test suite
name if they are separated from the base name of the suite with two underscores:

01__my_suite.html -> My Suite

02__another_suite.html -> Another Suite

Execution Suites and Tests 43

 RobotFrameworkAdvancedGuide < EMI < TWiki
If the alphabetical ordering of test suites inside suites is problematic, a good workaround is giving them
separately in the required order. This easily leads to overly long start-up commands, but a shell script wrapper
would remedy this. It is also possible to randomize the execution order using the --runmode option.

Continue on Failure
Normally test cases are stopped immediately when any of their keywords fail. This behavior shortens test
execution time and prevents subsequent keywords hanging or otherwise causing problems if the system under
test is in unstable state. This has the drawback that often subsequent keywords would give more information
about the state of the system. One way to handle failures so that test execution is not terminated immediately
is using BuiltIn keywords Run Keyword And Ignore Error and Run Keyword And Expect Error. Using these
keywords for this purpose often adds extra complexity to test cases though.

Stopping Test Execution Gracefully

Sometimes there is a need to stop the test execution before all the tests have finished, but so that logs and
reports are created. Different ways how to accomplish this are explained below. In all these cases the
remaining test cases are marked failed.

• Pressing ctrl-c
♦ The execution is stopped when Ctrl-C is pressed in the console where the tests are running.
When running the tests on Python, the execution is stopped immediately, but with Jython it
ends only after the currently executing keyword ends.
♦ If Ctrl-C is pressed again, the execution ends immediately and reports and logs are not
created.
• Using Signals
♦ It is possible to terminate test execution using signals INT and TERM. These signals can be
sent from the command line using the kill command, and sending signals can also be easily
automated.
♦ Signals have the same limitation on Jython as pressing Ctrl-C. Similarly also the second
signal stops the execution forcefully.
• Using Keywords
♦ The execution can be stopped also by the executed keywords. There is a separate Fatal Error
BuiltIn keyword for this purpose, and custom keywords can use fatal exceptions when they
fail.
• ExitOnFailure mode
♦ If option --runmode is used with value ExitOnFailure (case-insensitive), the execution of tests
stops immediately if a critical test fails and the remaining tests are marked as failed.

Handling Teardowns

By default teardowns for tests and suites that have been started are executed even if test execution is stopped
using one of the methods above. This allows clean-up activities to be run regardless how execution ends.
Teardowns are skipped when the execution is stopped if the --runmode SkipTeardownOnExit command line
option is used. This can be useful if clean-up takes a lot of time.

Post-processing Outputs
XML output files that are generated during the test execution can be post-processed afterwards by the rebot
tool, which is an integral part of Robot Framework. It is used automatically when test reports and logs are
generated during the test execution, but there are also good grounds for using it separately after the execution.

Execution Order 44
 RobotFrameworkAdvancedGuide < EMI < TWiki

Using the rebot Tool

The basic syntax for using rebot is exactly the same as when starting test execution, return codes are the same,
and also most of the command line options are identical. The main difference is that arguments to rebot are
XML output files instead of test data files or directories.

Creating Different Reports and Logs

You can use rebot for creating the same reports and logs that are created automatically during the test
execution. Of course, it is not sensible to create the exactly same files, but, for example, having one report
with all test cases and another with only some subset of tests can be useful. Another common usage is creating
only the output file when running tests (log and report generation can be disabled with options --log NONE
and --report NONE) and generating logs and reports later. Tests can, for example, be executed on different
environments, output files collected to a central place, and reports and logs created there.

Combining Outputs
The most important feature of rebot is its ability to combine outputs from different test execution rounds. This
capability allows, for example, running the same test cases on different environments and generating an
overall report from all outputs. Combining outputs is extremely easy, all that needs to be done is giving
several output files as arguments:

rebot output1.xml output2.xml

rebot outputs/*.xml

When outputs are combined, a new top-level test suite is created so that test suites in the given output files are
its child suites. This works the same way when multiple test data files or directories are executed, and also in
this case the name of the top-level test suite is created by joining child suite names with an ampersand (&) and
spaces. These automatically generated names are not that good, and it is often a good idea to use --name to
give a more meaningful name:

rebot --name Browser_Compatibility firefox.xml opera.xml safari.xml ie.xml

Configuring Execution
Selecting Test Cases
Robot Framework offers several command line options for selecting which test cases to execute. The same
options also work when post-processing outputs with the rebot tool.

By Test Suite and Test Case Name

Test suites and test cases can be selected by their names with the command line options --suite (-s) and --test
(-t), respectively. Both of these options can be used several times to select several test suites or cases.
Arguments to these options are case- and space-insensitive, and there can also be simple patterns matching
multiple names. If both the --suite and --test options are used, only test cases in matching suites with matching
names are selected.

--test Example
--test mytest --test yourtest
--test example*
--test mysuite.mytest
--test *.suite.mytest
--suite example-??
--suite mysuite --test mytest --test your*

Using the rebot Tool 45

 RobotFrameworkAdvancedGuide < EMI < TWiki

Using the --suite option is more or less the same as executing only the appropriate test case file or directory.
One major benefit is the possibility to select the suite based on its parent suite. The syntax for this is
specifying both the parent and child suite names separated with a dot. In this case, the possible setup and
teardown of the parent suite are executed.

--suite parent.child
--suite myhouse.myhousemusic --test jack*

By Tag Names

It is possible to include and exclude test cases by tag names with the --include (-i) and --exclude (-e) options,
respectively. When the former is used, only test cases having a matching tag are selected, and with the latter,
test cases having a matching tag are not. If both are used, only tests with a tag matching the former option, and
not with a tag matching the latter, are selected.

--include example
--exclude not_ready
--include regression --exclude long_lasting

Both --include and --exclude can be used several times to match multiple tags, and their arguments can be
simple patterns. In these cases, the rules for selecting test cases apply, so that test cases with a tag matching
any include patterns are selected, and tests with a tag matching exclude patterns are not. It is also possible to
select only test cases that have two or more specified tags by separating the tags either with & or AND
(case-sensitive). Only tests with a certain tag, but without any others, can be selected by separating these tags
with NOT (case-sensitive). If any of the patterns between multiple NOT is matching, the test case is not
selected.

--include req-*
--include regressionANDiter-42
--include tag1&tag2&tag3&tag4
--exclude regressionNOTowner-*
--include tag1NOTtag2NOTtag3&tag4 (includes tests which have `tag1`, but not tests which additio

Selecting test cases by tags is a very flexible mechanism and allows many interesting possibilities:

• A subset of tests to be executed before other tests can be tagged with smoke and executed with
--include smoke.
• Unfinished test can be committed to version control with the tag not_ready and excluded from the test
execution with --exclude not_ready.
• Tests can be tagged with iter-, where specifies the number of the current iteration, and after executing
all test cases, a separate report containing only the tests for a certain iteration can be generated (for
example, rebot --include iter-42 output.xml).

When no tests match selection

By default when no tests match the selection criteria the execution fails with an error like:

[ ERROR ] Suite 'Example' with includes 'xxx' contains no test cases.

Because no outputs are generated, this behavior can be problematic if tests are executed and results processed
automatically. Luckily a command line option --RunEmptySuite can be used to force the suite to be executed
also in this case. As a result normal outputs are created but show zero executed tests. The same option can be
used also to alter the behavior when an empty directory or a test case file containing no tests is executed.

By Test Suite and Test Case Name 46

 RobotFrameworkAdvancedGuide < EMI < TWiki

Setting Criticality
The final result of test execution is determined on the basis of critical tests. If a single critical test fails, the
whole test run is considered failed. On the other hand, non-critical test cases can fail and the overall status is
still passed. By default, all test cases are critical, but this can be changed with the --critical (-c) and
--noncritical (-n) options. These options specify which test cases are consider critical based on tags, similarly
as --include and --exclude are used to select test cases by tag names. If only --critical is used, test cases with a
matching tag are critical. If only --noncritical is used, tests without a matching tag are critical. Finally, if both
are used, only test with a critical tag but without a non-critical tag are considered critical. Both of these
options accept simple patterns and can be given several times:

--critical regression
--noncritical not_ready
--critical iter-* --critical req-* --noncritical req-6??

The most common use case for setting criticality is having test cases that are not ready or test features still
under development in the test execution. Of course, these tests could be excluded from the test execution
altogether with the --exclude option, but including them as non-critical tests enables you to see when they start
to pass. Criticality set when the tests are executed is not stored,thus if you want to keep the same criticality
when post-processing the outputs with rebot,you need to use --critical and/or --noncritical also with it.

Setting Metadata
Setting the name

When Robot Framework parses test data, test suite names are created from file and directory names. The
name of the top-level test suite can, however, be overridden with the command line option --name (-N).
Underscores in the given name are converted to spaces automatically, and words in the name capitalized.

Setting the documentation

In addition to defining documentation in the test data, documentation of the top-level suite can be given from
the command line with the option --doc (-D). Underscores in the given documentation are converted to spaces,
and it may contain simple HTML formatting.

Setting free metadata

Free test suite metadata may also be given from the command line with the option --metadata (-M). The
argument must be in the format name:value, where name the name of the metadata to set and value is its
value. Underscores in the name and value are converted to spaces, and the latter may contain simple HTML
formatting. This option may be used several times to set multiple metadata.

Setting tags

The command line option --settag (-G) can be used to set the given tag to all executed test cases. This option
may be used several times to set multiple tags.

Setting Variables
Variables can be set from the command line either individually using the --variable (-v) option or through
variable files with the --variablefile (-V) option. Variables and variable files are explained in earlier parts of
this guide, but the following examples illustrate how to use these options:

--variable name:value
--variable OS:Linux --variable IP:10.0.0.42

Setting Criticality 47
 RobotFrameworkAdvancedGuide < EMI < TWiki
--variablefile path/to/variables.py
--variablefile myvars.py:possible:arguments:here

Randomizing Execution Order

The --runmode option can also be used to randomize the test execution order. This is done using the different
values explained below.

• random:test
♦ Test cases inside each test suite are executed in random order.
• random:suite
♦ All test suites are executed in a random order, but test cases inside suites are run in the order
they are defined.
• random:all
♦ Both test cases and test suites are executed in a random order.

Example:

pybot --runmode random:test my_test.txt

Controlling Console Output

Console Width

The width of the test execution output in the console can be set using the option --monitorwidth (-W). The
default width is 78 characters. The handy $COLUMNS variable can be used like: --monitorwidth
$COLUMNS.

Console Colors

The --monitorcolors (-C) option is used to control whether colors should be used in the console output. This
option has following case-insensitive values:

• auto
♦ Colors are enabled when output is written into the console but not when it is redirected into a
file. This is the default and there is rarely need to change it.
• on
♦ Colors are used also when output is redirected into a file.
• off
♦ Colors are disabled.

Created Outputs
Several output files are created when tests are executed, and all of them are somehow related to test results.
This section discusses what outputs are created, how to configure where they are created, and how to fine-tune
their contents.

Different Output Files

This section explains what different output files can be created and how to configure where they are created.
Output files are configured using command line options, which get the path to the output file in question as an
argument. A special value NONE (case-insensitive) can be used to disable creating a certain output file.

Setting Variables 48
 RobotFrameworkAdvancedGuide < EMI < TWiki
Output Directory

All output files can be set using an absolute path, in which case they are created to the specified place, but in
other cases, the path is considered relative to the output directory. The default output directory is the directory
where the execution is started from, but it can be altered with the --outputdir (-d) option. The path set with this
option is, again, relative to the execution directory, but can naturally be given also as an absolute path.
Regardless of how a path to an individual output file is obtained, its parent directory is created automatically,
if it does not exist already.

Output File

Output files contain all the test execution results in XML format. Log and report files are generated based on
output files, and output files can also be combined and otherwise post-processed after the test execution. The
command line option --output (-o) determines the path where the output file is created relative to the output
directory. The default name for the output file, when tests are run, is output.xml. When post-processing
outputs, new output files are not created unless --output (-o) option is explicitly used. It is possible to disable
creation of the output file also when running tests with special value NONE. In this case also other output
files, except for the debug file, are disabled.

Log File

Log files contain details about the executed test cases in HTML format. They have a hierarchical structure
showing test suite, test case and keyword details. Log files are needed nearly every time when test results are
to be investigated in detail. Even though log files also have statistics, reports are better for getting an
higher-level overview. The command line option --log (-l) determines where log files are created. Unless the
special value NONE is used, log files are always created and their default name is log.html.

Report File

Report files contain an overview of the test execution results in HTML format. They have statistics based on
tags and executed test suites, as well as a list of all executed test cases. When both reports and logs are
generated, the report has links to the log file for easy navigation to more detailed information. It is easy to see
the overall test execution status from report, because its background color is green, if all critical tests pass, and
bright red otherwise. The command line option --report (-r) determines where report files are created.
Similarly as log files, reports are always created unless NONE is used as a value, and their default name is
report.html.

Debug File

Debug files are plain text files that are written during the test execution. All messages got from test libraries
are written to them, as well as information about started and ended test suites, test cases and keywords. Debug
files can be used for monitoring the test execution. This can be done using, for example, with the tail -f
command. Debug files are not created unless the command line option --debugfile (-b) is used explicitly.

Setting Titles

The default titles for logs and reports are generated by prefixing the name of the top-level test suite with Test
Log or Test Report. Custom titles can be given from the command line using the options --logtitle and
--reporttitle, respectively. Underscores in the given titles are converted to spaces automatically.Example:

pybot --logtitle Cream_Test_Log --reporttitle Cream_Test_Report --include cream cream_tests/

Output Directory 49
 RobotFrameworkAdvancedGuide < EMI < TWiki
Setting Background Colors

By default the report file has a green background when all the critical tests pass and a red background
otherwise. These colors can be customized by using the --reportbackground command line option, which takes
two or three colors separated with a colon as an argument:

--reportbackground blue:red
--reportbackground green:yellow:red
--reportbackground #00E:#E00

If you specify two colors, the first one will be used instead of the default green color and the second instead of
the default red. This allows, for example, using blue instead of green to make backgrounds easier to separate
for color blind people. If you specify three colors, the first one will be used when all the test succeed, the
second when only non-critical tests have failed, and the last when there are critical failures. This feature thus
allows using a separate background color, for example yellow, when non-critical tests have failed. The
specified colors are used as a value for the body element's background CSS property. The value is used as-is
and can be a HTML color name (e.g. red), a hexadecimal value (e.g. #F00 or #FF0000), or an RGB value (e.g.
rgb(255,0,0)). The default green and red colors are specified using hexadecimal values #9F6 and #F33,
respectively.

Log Levels
Messages in log files can have different log levels. Some of the messages are written by Robot Framework
itself, but also executed keywords can log information using different levels. The available log levels are:

• FAIL
♦ Used when a keyword fails. Can be used only by Robot Framework itself.
• WARN
♦ Used to display warnings. They shown also in the console and in the Test Execution Errors
section in log files, but they do not affect the test case status.
• INFO
♦ The default level for normal messages. By default, messages below this level are not shown
in the log file.
• DEBUG
♦ Used for debugging purposes. Useful, for example, for logging what libraries are doing
internally. When a keyword fails, a traceback showing where in the code the failure occurred
is logged using this level automatically.
• TRACE
♦ More detailed debugging level. The keyword arguments and return values are automatically
logged using this level.

By default, log messages below the INFO level are not logged, but this threshold can be changed from the
command line using the --loglevel (-L) option. This option takes any of the available log levels as an
argument, and that level becomes the new threshold level. A special value NONE can also be used to disable
logging altogether. It is possible to use the --loglevel option also when post-processing outputs with rebot.
This allows, for example, running tests initially with the TRACE level, and generating smaller log files for
normal viewing later with the INFO level. By default all the messages included during execution will be
included also with rebot. Messages ignored during the execution cannot be recovered. Another possibility to
change the log level is using the BuiltIn keyword Set Log Level in the test data. It takes the same arguments
as the --loglevel option, and it also returns the old level so that it can be restored later, for example, in a test
teardown.

Setting Background Colors 50

 RobotFrameworkAdvancedGuide < EMI < TWiki

System Log
Robot Framework has its own plain-text system log where it writes information about:

• Processed and skipped test data files

• Imported test libraries, resource files and variable files
• Executed test suites and test cases
• Created outputs

Normally users never need this information, but it can be useful when investigating problems with test
libraries or Robot Framework itself. A system log is not created by default, but it can be enabled by setting the
environment variable ROBOT_SYSLOG_FILE so that it contains a path to the selected file. A system log has
the same log levels as a normal log file, with the exception that instead of FAIL it has the ERROR level. The
threshold level to use can be altered using the ROBOT_SYSLOG_LEVEL environment variable like shown
in the example below. Possible unexpected errors and warnings are written into the system log in addition to
the console and the normal log file.

#!/bin/bash

export ROBOT_SYSLOG_FILE=/tmp/syslog.txt
export ROBOT_SYSLOG_LEVEL=DEBUG

pybot --name Syslog_example path/to/tests

System Log 51
Extending Robot Framework \ Creating New
Libraries
TODO

Extending Robot Framework \ Creating New Libraries 52

Appendices
Test Data Settings
Setting Table
The Setting table is used to import test libraries, resource files and variable files and to define metadata for
test suites and test cases. It can be included in test case files and resource files. Note that in a resource file, a
Setting table can only include settings for importing libraries, resources, and variables.

Name Description
Library Used for taking test libraries into use.
Resource Used for taking resource files into use.
Variables Used for taking variable files into use.
Documentation Used for specifying a test suite or resource file documentation.
Metadata Used for setting free test suite metadata.
Suite Setup Used for specifying the suite setup.
Suite Teardown Used for specifying the suite teardown.
Suite Precondition A synonym for Suite Setup.
Suite Postcondition A synonym for Suite Teardown.
Force Tags Used for specifying forced values for tags when tagging test cases.
Default Tags Used for specifying default values for tags when tagging test cases.
Test Setup Used for specifying a default test setup.
Test Teardown Used for specifying a default test teardown.
Test Precondition A synonym for Test Setup.
Test Postcondition A synonym for Test Teardown.
Test Template Used for specifying a default template keyword for test cases.
Test Timeout Used for specifying a default test case timeout.

Test Case Table

The settings in the Test Case table are always specific to the test case for which they are defined. Some of
these settings override the default values defined in the Settings table.

Name Description
[Documentation] Used for specifying a test case documentation.
[Tags] Used for tagging test cases.
[Setup] Used for specifying a test setup.
[Teardown] Used for specifying a test teardown.
[Precondition] A synonym for [Setup].
[Postcondition] A synonym for [Teardown].
[Template] Used for specifying a template keyword.
[Timeout] Used for specifying a test case timeout.

Keyword Table
Settings in the Keyword table are specific to the user keyword for which they are defined.

Name Description
[Documentation] Used for specifying a user keyword documentation.

Appendices 53
 RobotFrameworkAdvancedGuide < EMI < TWiki

[Arguments] Used for specifying user keyword arguments.

[Return] Used for specifying user keyword return values.
[Timeout] Used for specifying a user keyword timeout.

Command Line Options

Test Execution Command Line Options
• -N, --name name
♦ Sets the name of the top-level test suite.
• -D, --doc document
♦ Sets the documentation of the top-level test suite.
• -M, --metadata name:value
♦ Sets free metadata for the top level test suite.
• -G, --settag tag
♦ Sets the tag(s) to all executed test cases.
• -t, --test name
♦ Selects the test cases by name.
• -s, --suite name
♦ Selects the test suites by name.
• -i, --include tag
♦ Selects the test cases by tag.
• -e, --exclude tag
♦ Selects the test cases by tag.
• -c, --critical tag
♦ Tests that have the given tag are considered critical.
• -n, --noncritical tag
♦ Tests that have the given tag are not critical.
• -v, --variable name:value
♦ Sets individual variables.
• -V, --variablefile path:args
♦ Sets variables using variable files.
• -d, --outputdir dir
♦ Defines where to create output files.
• -o, --output file
♦ Sets the path to the generated output file.
• -l, --log file
♦ Sets the path to the generated log file.
• -r, --report file
♦ Sets the path to the generated report file.
• -S, --summary file
♦ Summary files are not supported in Robot Framework 2.6 or newer.
• -x, --xunitfile file
♦ Sets the path to the generated XUnit compatible result file.
• -b, --debugfile file
♦ A debug file that is written during execution.
• -T, --timestampoutputs
♦ Adds a timestamp to all output files.
• --splitlog
♦ Split log file into smaller pieces that open in browser transparently.
• --splitoutputs level
♦ Splitting outputs is not supported in Robot Framework 2.6 or newer.
• --logtitle title

Keyword Table 54
 RobotFrameworkAdvancedGuide < EMI < TWiki

♦ Sets a title for the generated test log.

• --reporttitle title
♦ Sets a title for the generated test report.
• --reportbackground colors
♦ Sets background colors of the generated report.
• --summarytitle title
♦ Summary files are not supported in Robot Framework 2.6 or newer.
• -L, --loglevel level
♦ Sets the threshold level for logging.
• --suitestatlevel level
♦ Defines how many levels to show in the Statistics by Suite table in outputs.
• --tagstatinclude tag
♦ Includes only these tags in the Statistics by Tag table.
• --tagstatexclude tag
♦ Excludes these tags from the Statistics by Tag table.
• --tagstatcombine tags:title
♦ Creates combined statistics based on tags.
• --tagdoc pattern:doc
♦ Adds documentation to the specified tags.
• --tagstatlink pattern:link:title
♦ Adds external links to the Statistics by Tag table.
• --listener name:args
♦ Sets a listener for monitoring test execution.
• --warnonskippedfiles
♦ Show a warning when an invalid file is skipped.
• --nostatusrc
♦ Sets the return code to zero regardless of failures in test cases. Error codes are returned
normally.
• --runemptysuite
♦ Executes tests also if the top level test suite is empty.
• --runmode mode
♦ Sets the execution mode for this test run. Valid modes are ExitOnFailure,
SkipTeardownOnExit, DryRun, and Random:.
• -W, --monitorwidth chars
♦ Sets the width of the console output.
• -C, --monitorcolors on|off|force
♦ Specifies are colors used in the console.
• -P, --pythonpath path
♦ Additional locations where to search test libraries from when they are imported.
• -E, --escape what:with
♦ Escapes characters that are problematic in the console.
• -A, --argumentfile path
♦ A text file to read more arguments from.
• -h, --help
♦ Prints usage instructions.
• --version
♦ Prints the version information.

Output Post-processing Command Line Options

• -N, --name name
♦ Sets the name of the top level test suite.
• -D, --doc document
♦ Sets the documentation of the top-level test suite.

Test Execution Command Line Options 55

 RobotFrameworkAdvancedGuide < EMI < TWiki

• -M, --metadata name:value

♦ Sets free metadata for the top-level test suite.
• -G, --settag tag
♦ Sets the tag(s) to all processed test cases.
• -t, -test name
♦ Selects the test cases by name.
• -s, --suite name
♦ Selects the test suites by name.
• -i, --include tag
♦ Selects the test cases by tag.
• -e, --exclude tag
♦ Selects the test cases by tag.
• -c, --critical tag
♦ Tests that have the given tag are considered critical.
• -n, --noncritical tag
♦ Tests that have the given tag are not critical.
• -d, --outputdir dir
♦ Defines where to create output files.
• -o, --output file
♦ Sets the path to the generated output file.
• -l, --log file
♦ Sets the path to the generated log file.
• -r, --report file
♦ Sets the path to the generated report file.
• -S, --summary file
♦ Summary files are not supported in Robot Framework 2.6 or newer.
• -x, --xunitfile file
♦ Sets the path to the generated XUnit compatible result file.
• -T, --timestampoutputs
♦ Adds a timestamp to all output files.
• --splitlog
♦ Split log file into smaller pieces that open in browser transparently.
• --splitoutputs level
♦ Splitting outputs is not supported in Robot Framework 2.6 or newer.
• --logtitle title
♦ Sets a title for the generated test log.
• --reporttitle title
♦ Sets a title for the generated test report.
• --reportbackground colors
♦ Sets background colors of the generated report.
• --summarytitle title
♦ Summary files are not supported in Robot Framework 2.6 or newer.
• -L, --loglevel level
♦ Sets the threshold level to select log messages.
• --suitestatlevel level
♦ Defines how many levels to show in the Statistics by Suite table in outputs.
• --tagstatinclude tag
♦ Includes only these tags in the Statistics by Tag table.
• --tagstatexclude tag
♦ Excludes these tags from the Statistics by Tag table.
• --tagstatcombine tags:title
♦ Creates combined statistics based on tags.
• --tagdoc pattern:doc
♦ Adds documentation to the specified tags.

Output Post-processing Command Line Options 56

 RobotFrameworkAdvancedGuide < EMI < TWiki

• --tagstatlink pattern:link:title
♦ Adds external links to the Statistics by Tag table. table in outputs.
• --removekeywords all|passed
♦ Removes keyword data from the generated outputs.
• --starttime timestamp
♦ Sets the starting time of test execution when creating reports.
• --endtime timestamp
♦ Sets the ending time of test execution when creating reports.
• --nostatusrc
♦ Sets the return code to zero regardless of failures in test cases. Error codes are returned
normally.
• -E, --escape what:with
♦ Escapes characters that are problematic in the console.
• -A, --argumentfile path
♦ A text file to read more arguments from.
• -h, --help
♦ Prints usage instructions.
• --version
♦ Prints the version information.

Output Post-processing Command Line Options 57

Document Licence
This page is based on the work presented here:
http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.6.1
Copyright © Nokia Siemens Networks 2008-2011 Licensed under the Creative Commons Attribution 3.0
Unported license http://creativecommons.org/licenses/by/3.0/

-- DimosthenesFioretos - 04-Oct-2011

-- DimosthenesFioretos - 17-Oct-2011

This topic: EMI > RobotFrameworkAdvancedGuide

Topic revision: r1 - 2011-10-17 - unknown

Copyright &© 2008-2023 by the contributing authors. All material on this

collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback

Document Licence 58