Journal of Behavioral and Experimental Finance 23 (2019) 12–22

Contents lists available at ScienceDirect

Journal of Behavioral and Experimental Finance

journal homepage: www.elsevier.com/locate/jbef

Full length article

Simulation-based learning using the RIT market simulator and RIT

decision cases
∗,1
Kevin Mak a,b , Thomas H. McCurdy c,d ,
a
Real-time Analysis and Investment Lab (RAIL), United States
b
Lecturer in Management, Graduate School of Business, Stanford University, United States
c
Academic Director, BMO Financial Group Finance Research and Trading Lab (FRTL), Canada
d
Professor of Finance and Bonham Chair in International Finance, Rotman School of Management, University of Toronto, Canada

article info a b s t r a c t

Article history: RIT is a custom-designed product consisting of a market simulator platform and learning-by-doing
Received 1 December 2018 decision cases that simulate risks and opportunities for various financial securities, investment and
Received in revised form 5 May 2019 risk management strategies. Using the market simulator with custom designed cases, linked in real-
Accepted 6 May 2019
time to decision support models applying the relevant theory, participants learn how to make good
Available online 15 May 2019
real-time decisions in complex environments for which there is material uncertainty. The market
aggregates participants’ decisions and provides immediate feedback concerning the success of their
strategies, allowing them to adapt their strategies after each replication of the case. Besides supporting
teaching and training, the decision cases facilitate competitions and events at many different levels.
Given the high degree of flexibility and customization that is available to the market administrator,
low-friction engagement for participants, and high-resolution data logging, the RIT product is also a
valuable resource for investigating research questions across a wide-range of topics.
© 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction and motivation 2 The impact of simulations on university-level learning has

been investigated most thoroughly in aviation and medical and
Uncertainty about the future is pervasive. Decision-making health professional education/training. For example, a system-
under uncertainty is an important skill. Simulation-based learn- atic review of medical simulations by Issenberg et al. (2005)
ing provides participants with a hands-on approach to practice concludes that simulations benefit learning when they: are in-
tegrated into curricula and capture a variety of (clinical) con-
decision making in a controlled environment where they can
ditions; provide clear goals and outcomes; provide a range of
immediately observe the outcomes of their decisions. By being
difficulty levels and adapt to multiple learning strategies; require
able to analyze the consequences of their decisions in different
repetitive practice and feedback; promote active participation in
situations, students are able to learn how to make good decisions
learning; and provide a safe environment for errors. In addi-
for complex problems given varying degrees of uncertainty about tion, a systematic review and meta-analysis provided by Cook
the future. The simulation-based tools enable students to apply et al. (2011) concludes that ‘‘in comparison with no intervention,
and develop what they have learned in the classroom in order to technology-enhanced simulation training in health professions
solve problems they would find in the workplace, but in a ‘safe’ education is consistently associated with large effects for out-
and controlled environment. comes of knowledge, skills, and behaviors and moderate effects
for patient-related outcomes’’.
∗ Correspondence to: Rotman School of Management, University of Salas et al. (2009) discuss advantages associated with
Toronto, 105 St. George Street, Toronto ON M5S 3E6, Canada. simulation-based training for management education, including:
E-mail addresses: [email protected] (K. Mak), providing a more complex and realistic learning environment
[email protected] (T.H. McCurdy). than other training strategies while still allowing reality to be
1 The authors are grateful to Jasper Chan, Marco Salerno, Eric Kang and
simplified and manageable; providing a (relatively) risk-free en-
FRTL Assistants for their contributions to the RIT product. We are also grateful vironment for learning and experimentation; leading to learning
for many helpful suggestions for this paper from two anonymous reviewers,
Stefan Palan (Editor), and R. Woodhouse. The RIT webpages can be accessed at
in reduced time; is an ideal method for training infrequently
http://rit.rotman.utoronto.ca. engaged but critical skills; is more engaging than other train-
2 This section draws on McCurdy and Woodhouse (2002) and Woodhouse ing methods; and can impart complex applied competencies.
and McCurdy (2014). Woodhouse and McCurdy (2014) discuss educational benefits of

https://doi.org/10.1016/j.jbef.2019.05.003
2214-6350/© 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22 13

interactive simulations, for example: facilitating deep learning; The RIT package has built-in ‘Application Program Interfaces’
clarifying troublesome and threshold concepts; providing moti- (APIs) that can be turned on for any case so that participants
vation and opportunities for practice; and allowing immediate can practice writing simple scripts to implement some decisions
feedback and assessment. automatically. Automated systems can be efficient but fragile
As highlighted in later sections of this paper, all of the above which is why the pilot’s role in monitoring the cockpit instru-
advantages of simulation-based learning and training are ments and skill in responding quickly to problems is so crucial.
achieved using the RIT Decision Cases implemented on the RIT That is why sequencing the RIT cases such that participants
(Rotman Interactive Trader) market simulator platform (an in- make the decisions manually first (practicing at lower speeds and
dustry strength order-driven market). The RIT Decision Cases incrementally) so that they fully understand the scope of po-
simulate risks and opportunities for a broad range of tasks/jobs. tential outcomes before writing an algorithm and implementing
Students can use their classroom lessons to try out decisions. The strategies automatically.
simulated market aggregates the decisions of all participants and The remaining sections of this paper are organized as follows:
provides immediate feedback on the success of their strategies Section 2 summarizes the structure of the RIT Server Application;
allowing them to adapt their strategies after each replication. Section 3 reviews some features of the RIT Client Application as
Multiple replications of the case simulation allow students to well as available real-time links with support models; Section 4
practice finding a robust strategy for the types of risks and oppor- discusses some examples of the RIT Decision Cases; Section 5 pro-
tunities they face for a particular task. The RIT product contains vides some examples of applications for research, competitions
many performance feedback tools, which facilitate discussion, and events; and Section 6 concludes.
accelerate learning, and provide an opportunity for instructors
to provide incentives for participants to focus on the learning 2. Structure and features of the RIT server
objectives of each case. As a result, students learn how to de-
velop and use their knowledge and skills in complex, real-time 2.1. Server structure
environments for which uncertainty is material.
There are few clinics or teaching practice settings to enable The RIT package is currently available for the Microsoft Win-
students in the social sciences to obtain experience acquiring dows OS. The RIT Server application operates a matching engine
skills using simulation. Why is simulation-based learning more that continuously accepts and matches limit and market orders
entrenched in aviation and medical training? Perhaps because from market participants and computer-generated AI (Artificial
the occurrence of errors in those settings are usually more easily Intelligence) order flow. These orders are segregated based on
observed and the implications of making errors in those settings their association with different securities, for example: multiple
can be catastrophic. stocks, options, futures or bonds. Further segmentation can be
In social science applications, such as finance and economics, achieved by creating different marketplaces for the same security,
the risks and uncertainties associated with decisions are perhaps for example: a single stock trading on two different exchanges.
even more complex since they are also affected by model un- Each security market and marketplace can be individually pa-
certainty, parameter uncertainty, and signal extraction issues for rameterized to create differences in fees, tick sizes and market
the varying signal-to-noise ratios associated with most decisions. permissions to name a few. In terms of robustness, the matching
Recall the literature on detecting skill versus luck associated with engine can accept and clear over 50,000 orders per second using
portfolio managers’ performance. For example, Fama and French a modern personal computer.
(2010) use bootstrap simulations from long histories of fund re- The matching engine operates as a continuous double auc-
turns in order to confidently identify skill. This research example tion and follows a price–time priority ruleset. Order types are
highlights the necessity of using simulations to separate signal limited to market and limit orders, and limit orders are treated
from noise in many empirical finance applications; an important (partially or completely) as marketable if the limit-order price
skill for both students and practitioners. crosses existing orders on the other side of the order book. Mar-
Simulations can train participants to better identify the im- ket clearing prices are established on a per marketplace basis,
plications of parameter and model uncertainty and acquire skills so there is no aggregate order book generated by the server. This
to manage those risks. There is also pedagogical value associated creates requirements for participants to efficiently route their or-
with sequencing skill acquisition from mastering single skills ders (in the case of a security trading on multiple market places),
(dealing with one risk) and then including additional risks as we as poor execution will cause price distortions and create arbitrage
sequence through a set of cases on a particular topic until they opportunities.3 The RIT markets also allow for ‘‘upstairs market’’
can manage the capstone case in which several risks can interact. trading where participants can directly submit negotiated trades
Here again, the analogy with a flight simulator is useful since that fit within market price constraints (i.e. blocks must trade
airplane accidents often happen when risks cascade so pilots have within 5% of National Best Bid Offer). All back-office functionality
to be trained to deal with those situations. They do so by practic- (clearing and settlement) is done at the time of the trade by the
ing acquiring skills one at a time and then combining them; and server since all funds and securities are virtual.
they often unintentionally crash the plane in simulation during The RIT server currently allows equities, fixed income se-
this learning-by-doing. They do not learn or practice with a new curities, physical commodities, foreign exchange, options and
airplane full of people. The RIT Decision Cases train participants futures to be transacted. Of course, synthetic products can also be
to practice dealing with market or economy dynamics, includ- created by combining existing instruments. All of these securities
ing potential crises and other complex events, as well as the can be denoted in any currency.
endogenous uncertainty introduced by the market participants Information can be distributed to market participants pub-
themselves. They practice these skills before proceeding to advise licly or privately in plain text or HTML format. Information
clients about risk and opportunities associated with the assigned content and arrival times can be customized (and/or random-
task(s). In addition, management problems themselves are dy- ized) to create asymmetry across market participants. Partici-
namic and often novel. Simulation-based learning can contribute pants can also be queried and required to provide information,
to learning ‘how-to-learn’, a skill that is conducive to life-long
learning and innovation. 3 Participants can build their own aggregate order book using the RTD (real-
Another useful analogy with a flight simulator is the automa- time data) links to the alternative markets. A start-up decision-support model
tion required to deal with ‘big data’ and fast decision making. does this for the market microstructure cases.
14 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22

feedback, and/or answer questions in real-time while the markets As opposed to liquidity AI Order Flow, one can also utilize
are running. Informed AI Order Flow. In this case, N orders are generated per
All information and actions available to market participants minute, each order has a price that is Normally distributed around
can be controlled by the server on an aggregate or individual the current midmarket price. These orders are programmed to
level. For example, limit-order book ‘‘level 2’’ or time-and-sales be informed in the following sense. The current midmarket price
data can be selectively hidden from market participants. Individ- is compared to a predetermined P* ‘‘informationally efficient’’
ual markets can be halted and resumed, slowed down or sped price, and the direction of a programmed order is determined
up, and securities can pay dividends, be settled, expire or be based on whether the current midmarket price is below or above
liquidated. the P* price. If the market is currently below the P* price, the
programmed order will be a buy order, otherwise it is a sell order.
2.2. Defining participant roles All AI orders are generated as limit orders, however, when a buy
order is generated with a price that is above the current best ask
2.2.1. Optional AI programmed order flow price, the matching engine treats it as a marketable limit order.
One of the most innovative features of the RIT platform is As a result, when the market is currently below the P* price,
the AI Order Flow which can be programmed as uninformed roughly 50% of orders are limit orders to buy, adding liquidity to
(liquidity or noise order flow) or, alternatively, as informed or- the market, and 50% are marketable limit orders to buy, removing
der flow. N orders are generated per minute, each order has a liquidity from the market. Vice versa if the market is above the
price that is Normally distributed around the midpoint4 of the
current P* price.
bid and ask prices at the top of the limit-order book (hereafter
The AI Informed Order Flow provides many customizable
referred to as the midmarket price). The Normally distributed
benefits, including:
prices are rounded to the nearest tick, based on a specified tick-
size (i.e. nearest penny). The order is determined to be a buy order (1) Allowing the market to be informationally efficient even
or a sell order based on an equal probability draw. All orders when human agents act irrationally.
are generated as limit orders, however, when a buy order is (2) Creating competitive forces against human traders; this
generated with a price that is above the current best ask price, the can be useful when there are a relatively small number of
matching engine treats it as a marketable limit order. As a result:
humans participating in the market.
roughly 25% of orders are limit orders to buy, adding liquidity
(3) Creating uncertainty for human participants when they are
to the market; 25% are marketable limit orders to buy, removing
trading with the AI order flow. The human trader does
liquidity from the market; 25% of orders are limit orders to sell;
not know, ex-ante, whether or not she is trading with an
and 25% of orders are marketable limit orders to sell.
informed or uninformed programmed participant (both are
Some benefits of the liquidity or noise order flow are as
labeled with the same trader ID ‘ANON’ in the limit-order
follows:
book).
(1) In class settings with few participants (n<10), human
traders typically lack the trading frequency to facilitate an The result of allowing both uninformed and informed AI order
orderly marketplace. The AI liquidity order flow can act as flow in the market is that the human participants have the po-
price-agnostic market makers who absorb market orders tential to generate excess profits, but cannot do so with impunity
or trade against limit orders and hold temporary inventory because they may be trading against informed traders at any
positions until they can be transferred from one human time. They also cannot easily observe or ‘‘follow’’ the actions from
participant to another. the informed AI order flow because it is difficult to distinguish
(2) The previous point is particularly important for partici- noise from informed trades. This creates a semi-informationally
pants who are practicing in preparation for a class or event. efficient market where market participants’ actions, both pro-
The RIT cases can be set to run 24/7 with remote access grammed and human, are constantly being obfuscated. Put an-
so those practicing may be accessing the market at times other way, a very realistic marketplace environment in which to
when there are few human participants. practice and learn.
(3) The AI order flow allows an instructor to predetermine a Consider the following example of how the Informed and
certain level of market liquidity, instead of relying solely Uninformed AI Order Flow generated by the computerized agents
on human participants. This can be used to illustrate the can interact with and affect the market prices. Suppose that there
difference between very liquid or illiquid markets and can is a stock that is initialized with information that it will pay a
be changed dynamically. liquidating dividend of V as its only cash flow and that the risk
(4) In class settings with few or many participants, the AI order free rate is zero. The P* price is parameterized to begin at V so that
flow parameterized as noise traders provides opportunities the informed computerized traders are aware that the fair value
for informed human traders to generate trading profits, of the asset is V. At ts , the case is programmed to announce to all
even if the human traders hold the same information. human traders that the liquidating dividend has been increased
Without noise traders, human participants who possess by B to V+B. The P* price is programmed to increase by B for t >
the same information will never trade with one another. ts . In other words, as soon as the news is released, P* increases by
This is the most important feature of noise orders because B. Informed AI Order Flow will now observe the market price of
it allows the administrator to use the trading results to P being considerably below P*, and react by placing a significant
distinguish skill. The trading results require some mar- number of buy orders, thus causing observed market prices to
ket participants to systematically lose money in order to increase. Uninformed AI Order Flow will continue to trade in a
provide profits to the informed and skilled participants. manner that is completely ignorant of P*, i.e. they will randomly
(5) Finally, for participants at the introductory level, the mar- buy and sell.
ket activity generated by the AI order flow creates an To make the computerized traders partially-informed, the case
environment that is less intimidating to enter. could adjust P* by V+B+e where e is a noisy error term, centered
on zero. The computerized traders will now react to the news in
4 If no midpoint exists, the last traded price, or the parameterized starting a correct, but imprecise manner. They may under or over react to
price is used instead. the result, even though on average they are correct.
 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22 15

2.2.2. Human participants 2.3.2. Real-time identification of participants and monitoring

Human participants can be defined with both common and Fig. 1 illustrates one example configuration of the RIT Server
role-specific permissions that determine the information that interface. Note that modules can be opened or closed quickly as
they receive (both news and price data), the securities (and/or one sequences through the various roles of the instructor: loading
physical infrastructure) to which they have access, and the trad- a decision case, setting parameters, checking market participants’
ing capital that is available to them. links, adjusting market speed, monitoring the market and partic-
The most common application of human participants’ roles ipants’ performance, providing feedback, saving results, posting
is to create homogeneous roles (that is, all humans have the results, etc.
exact same parameters) and allow for their individual analyses, All trade, position, and profit & loss data are available in
utility, and skill to create dispersion. In theory, if all participants real-time to the instructor. This is crucial because it allows the
have the same utility function, information and skill, no trades instructor to identify and monitor students who are struggling
would occur. However, even with homogeneous roles and com- (or excelling) and address their needs in an immediate manner.
mon information, heterogeneous skill levels can result in trading These data can also be broadcast to participants so that they can
activity. see their performance relative to their peers and calibrate their
To generate an even more dynamic marketplace, we can strategies accordingly. Displays in the ‘Monitoring’ module allow
force dispersion in the participants’ actions by setting up a non- one to match real names with ‘Trader Id’ or optionally hide the
homogeneous environment. The administrator can generate cap-
real names when posting results.
ital differences, trading constraints, and information asymmetry
With respect to saving the results, all trade, position, and
to further accentuate disagreement and incentivize more trading
profit and loss (P&L) data can be exported to a high-frequency
to occur. Note that, since participants are not exogenous to the
excel file that timestamps all participant actions. Most commonly,
market, endogenous uncertainty can be generated by trader
the aggregated P&L data are exported and used to illustrate per-
activity and the resultant behavioral effects studied by behavioral
formance to a class of participants. However, the entire trading
economics and finance can occur in relevant situations.
histories can be rebuilt and replayed for the class using the saved
detailed data.
2.3. Administering the market After every replication of a case, individual-specific perfor-
mance reports are distributed to all participants. These reports
2.3.1. Adaptive control of the market provide detailed time-series charts of their actions relative to the
With respect to running the market, the market begins in an market dynamics, as well as tabular summaries of their profit &
offline-state while the market administrator changes parameters loss resulting from those actions. The PDF format of these per-
or case-specific variables and checks that all participants are formance reports is designed to be easy to read and interpret; as
linked to the market. On their RIT Client App, participants will be compared to the high-frequency data (saved in Excel) which are
able to observe which case is loaded and set up their monitoring complete but more cumbersome to generate meaningful reports
and decision interface accordingly (see further descriptions in that can be consumed easily after each replication of the case.
Section 3 below). There are many benefits for learning associated with timely
Once the market is started, orders can be submitted and po- and useful feedback. Creating an engaging and challenging sim-
tentially matched, markets clear, and the results – including ulation that focuses on the most important components of a
information flows – are revealed to participants through their required decision provides the basis of useful hands-on learning.
RIT Client modules. While online, the market administrator can
Equally important is timely feedback concerning the outcomes of
pause and subsequently resume the market. In addition, the
decisions. In the RIT cases, the market aggregates participants’
administrator can adjust the speed of the market in real time.
decisions and provides immediate feedback.
Lastly, the server can be run 24/7 on an ‘‘automatic reset’’ mode
Being able to report the sequence of their actions in a way that
where a simulation replication is run, and then, after a pause,
allows participants to assess whether or not their decisions were
the market automatically resets by reloading the case with new
correct is extremely important for the learning process. Simply
random seeds and runs again. This allows for a server to be
showing aggregate P&L data is an extremely blunt tool to under-
left on in an un-administered state but participants can login
stand whether a participant truly understood the subject matter.
and ‘‘practice’’ at their convenience. The practice mode allows
Making volumes of high-frequency data available to participants
for students to have considerably more exposure (contact-time)
with a fully functioning market. The enables students to ‘‘learn at creates the potential for deep and thoughtful analytics; but those
their own pace’’ either prior to, or after, the traditional classroom data are often too detailed to be useful between case replications
experience. which sometimes results in the feedback being ignored. The RIT
There are many implications for learning associated with performance reports strike a balance by presenting time series
adaptive control of the market. While seemingly trivial on the plots of the relevant decisions (for example, Fig. 5) and tabular
surface, the ability to pause an active market is crucial for class- summary data of results (for example, Fig. 6) in a way that can
room learning. This feature allows the instructor to completely be easily interpreted and discussed. As a result, participants can
halt the market and take time (and have the undivided attention learn from their mistakes and adjust their strategies in follow-up
of the students) to discuss current market observations without replications of the case.
being concerned with data and parameters changing. An ex- Examples of non-P&L data that can be reported using the
tension of this is the ability to slow down a case. This allows granular and high-frequency data would be: Value-At-Risk for a
participants to be more thoughtful about their decision making given student’s portfolio if the objective is to manage the risk
because the pressure of a ‘‘fast moving market’’ is alleviated. In of a portfolio, Options pay-off graphs displaying the possible
the authors’ experience, it is common to run the market at half outcomes for a student who is building a hedge, or arbitrage
(50%) speed and then progress to 100% speed as students master vs. speculative profits in a case where the objective is generate
the skills required by the simulation. arbitrage profits.
16 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22

Fig. 1. Sample configuration of the RIT server interface.

3. Structure and features of the RIT client and decision-support (1) Useful modules vary across specific RIT Decision Cases.
models For example, modules relevant for physical assets, such as
pipelines or refineries, are unlikely to be relevant for cases
3.1. Client features that only include decisions about financial securities.
(2) A customizable RIT Client interface makes the learning
The RIT Server Application is analogous to a flight simulator
curve more gradual for students. By starting with a handful
machine that can be programmed to deliver practice for the
of information and decision-input modules available at any
pilot to acquire skills. The RIT Client Application is analogous to
the cockpit displays that allow monitoring of the environment given time, the relevant information that they need is
and provides instruments for inputting decisions. An important readily at their disposal instead of being hidden across a
feature of the RIT Client is the fact that the case information and plethora of data. This allows them to focus on processing
support files are dynamically integrated into the user interface. the data and making decisions, instead of spending time
This means that the student can readily access help documents, finding the data.
case documentation, decision-support models, or other attach- (3) Individuals learn at different paces and analyze data in
ments any time a case is running. This seems trivial, but it ensures different ways. Some may want to add many extra modules
that the student is never in a situation where they are participat- showing very minute details, whereas others may prefer to
ing in a simulation ‘‘without access to the required information’’. keep the displayed data to a minimum. The ‘‘one size fits
In addition, having these documents integrated into the software
all’’ model of displaying data does not match the needs of
is far more efficient than requiring the students to download
a diverse user base.
them from an external website, Learning Management System, or
network drive. (4) Professional industry-standard applications typically use a
The RIT Client features a customizable modular decision modular layout due to the variation in roles and personal
space where the student can open, move, and resize RIT Client preferences of those using the same software application.
modules5 anywhere on their screen. Each module serves a dif- The RIT Client also being modular is simply realistic and
ferent purpose, for example, the ‘Charting’ module provides real- introduces students to what they can expect if they enter
time charts of variables such as security prices, combinations of the securities trading industry.
security prices such as spreads, P&L, etc.; and the ‘Time and Sales’
module provides all transactions that have occurred for a specific The limit-order book can displayed either in ladder or book
security. Allowing the student to show, hide, and move these format or both. Fast order entry can be implemented with pre-
modules serves multiple learning and efficiency purposes. Fig. 2 set volume and price offset (improvement) by clicking on orders
illustrates just one of many possible configurations of the RIT
in the book. This feature is available for limit orders, market
Client interface. Multiple screens allows participants to configure
orders and sweep orders, and is particularly useful for tasks like
the RIT Client interface on one screen with a linked real-time
decision-support model on another screen. market making. Alternatively, individual order entry screens can
be toggled to be market or limit orders. As described below,
5 These RIT Client modules include: Portfolio, Order Entry, Market Depth, orders can also be implemented automatically using algorithms
Trade Blotter, Transactions Log, Assets, News, Trader Info, Time & Sales, Charting, which access the simulated market using alternative application
Chat, Kill, Case Files, and Help Files. program interfaces (APIs).
 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22 17

Fig. 2. Sample configuration of the RIT client interface.

3.2. Data interfacing between the RIT client and decision-support option reserved for students with significant coding experience.
models Most commonly, students would use Python or Java to submit
trades via REST. These languages require a significant amount of
The RIT Client has three alternative data interfaces, and these coding overhead that Excel would typically automatically handle
are also scaled in terms of difficulty to facilitate a gentle but for the student. For context, programming a simple two-stock ar-
robust learning curve. Real-Time Data (RTD) links are available bitrage algorithm in VBA requires about 12 lines of code, whereas
in order to drag-and-drop nearly all quantitative data elements using REST would typically require many more lines of code.
directly from the RIT Client into Excel. This will create a data Nevertheless, for robust modeling, the flexibility and speed of the
link that updates in real-time, and students can then use Excel REST interface is unmatched.
formulas that they are already familiar with to manipulate the
data. Students can also learn how to use the RTD Function Syntax 3.3. Some implications for learning associated with the RIT client
to construct dynamic RTD links so that they can quickly and
efficiently extract large amounts of data from the RIT Client for The RIT Client was designed with one primary principle: make
analytical purposes. This method of data analysis is extremely the ‘software’ part of the learning process as frictionless as possi-
accessible given that it is based on drag-and-drop functionality ble. The goal is to make the software simple and straightforward
so that the student can focus on the content and learning objec-
augmented by Excel formulas. The RTD Links are uni-directional
tives of each RIT Decision Case. In other words, the goal is for
and can only facilitate data being exported from the RIT Client
the student to think about their strategy, such as, ‘‘should I buy
into Excel (i.e. trades cannot be submitted from Excel to the RIT
or sell this security’’ and not ‘‘what screens do I need to look
client via RTD). Fig. 3 provides an example of a decision support
at, and then what buttons do I need to push, in order to buy
template for a particular RIT Decision Case utilizing RTD links
or sell this security’’. Further, as users become more experienced
to the simulated market. Participants can build out the start-
and obtain a level of proficiency with the RIT Client, they should
up templates by adding their own programming, for example, to
not feel like the software is ‘‘holding them back’’, that is, they
include flashing signals for actions when a security is mispriced
should be able to design and customize their client interface so
in the market.
that information and decision input options are available to them
Students who master the RTD functions can then use an Excel in whatever manner best suits their learning and analysis style.
VBA COM Object which creates custom VBA functions that can The challenge was designing software that was user and learn-
be utilized to submit data or trade requests to the RIT Client. ing friendly for someone who had 2 h experience with it, while
As a result, this data link is bi-directional. Initializing the VBA making it robust enough for someone who had 200 h of experi-
code (functions or subroutines) and submitting trade requests ence using it. This design philosophy also applies to the student’s
is considerably more difficult than simply requesting data via journey through data integration (RTD, VBA API, REST API), as
RTD, so it provides students with a sequential challenge to their well as case design structure and sequencing of cases for skill
learning. Since the VBA API is built into Excel, students can still acquisition.
rely on Excel’s graphical user interface and cell-based functions to
perform most of the calculations, thus making the programming 4. RIT decision cases
requirements less onerous.
In addition, a REST (Representational State Transfer) API, 4.1. Summary of available cases
built on industry-standard protocols, is available. It can accept
queries from nearly any programming language. The REST API is The RIT market simulator package has more than 50 different
a very robust bi-directional messaging, trade and data integration RIT Decision Cases available for instructors to choose from when
18 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22

Fig. 3. Sample decision-support template with RTD links to the simulated market.

deciding what materials are most relevant to their learning ob- As summarized in the summary of available cases in Sec-
jectives. These cases span the spectrum of securities including: tion 4.1 above, many of the RIT Decision Cases feature learning
bonds (corporate and sovereign), commodities (electricity, natu- objectives that use financial securities, such as derivatives, for
ral gas, oil, wheat, etc.), equities, ETFs, foreign currency, futures strategies focused on risk management or statistical arbitrage
and options. rather than speculation. For example, cases that focus on real
Using these securities, the cases mimic the decision space economy risks, agricultural crop hedging, commodity strategies
faced by professionals in many different roles including, but not (such as those related to oil, gas and electricity production, distri-
limited to: Algorithmic Market Making and Arbitrage, Algorith- bution and marketing objectives) and pension plan management
mic Smart-Order Routers, Credit Analysts & Traders, Dealers, Eq- are used to practice using financial securities to achieve those
uity Analysts & Traders, Pension Plan and other Portfolio Man- objectives.
agers, Rates Analysts & Traders, Risk Managers including market For more advanced cases, the application of the relevant asset
risk, liquidity risk, etc., Commodity Speculators, Hedgers and pricing models becomes considerably more complex, taking into
account multiple asset correlations, risks, etc. An example, is the
Arbitragers.
RIT Fixed Income 7 Case which combines dynamic yield curves
with changing credit risks for corporate bonds. Less skilled par-
4.2. Case design and calibration
ticipants may only be able to apply simple models to individual
securities, whereas more experienced or more skilled participants
Some cases are designed in such a way that students should
may be able to identify more opportunities and manage more
apply specific asset pricing models to determine the fair price of
risks. Nevertheless, both can participate in the same markets,
the asset which they can then compare to the relevant market learning from their own level and from each other. This highlights
quotes, buying under-priced and selling over-priced securities. the value of the cases for individualized learning-by-doing.
The inputs into their model are stochastic, so that each time the Inputs to markets are stochastic (exogenous uncertainty) and
case is run, the results and correct actions will be different. In behavioral affects, including heterogeneous participants and
other words, the methodology of applying the solution to each model uncertainty, can generate endogenous uncertainty. There-
case replication is the same, even the though the explicit solution fore, a participant’s performance metrics such as P&L will be a
will be different. function not only of that trader’s skills, such as her informed
These cases are calibrated such that the AI order flow (and trades, but also will be affected by market ‘noise’. While the noise
other, outside factors) purposely cause market mispricing, which on a single replication may be large enough to overwhelm the
generates opportunities for astute market participants to sub- signal generated by informed trades, cases are intended to be
mit informed trades and accrue profits. Ultimately those with run over multiple replications such that the noise component
more accurate, better calibrated (more informed) asset pricing becomes a smaller and smaller component of the participant’s re-
models, as well as better decision and execution skills, will gen- sults. In other words, the signal versus noise has been calibrated
erally be able generate higher returns from their decisions. This such that on average across multiple replications those with the
may not be the case for any particular replication, but we have best strategy will have the best performance.6
calibrated the signal versus noise such that on average across
multiple replications those with the best strategy will have the 6 For example, in the RIT LT3 Case, the stochastic liquidity spread associated
best performance. with tender offers is calibrated jointly with the market liquidity such that
 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22 19

4.3. Case sequencing order routing (ALGO 3 Case), etc. In fact, API order entry can
be turned on for any RIT case, if appropriate, once the learning
The RIT Decision Cases are intended to be deployed in a objectives associated with manual decision and execution have
sequential manner, where each case builds upon the skills learned been mastered.
in the previous case. Following the same philosophy as the RIT
Client software, data integration tools, and case design, the step- 4.4. Feedback & performance evaluation: Implications for learning
by-step sequences facilitate a gentle learning curve where par-
ticipants can focus on mastering a sequence of learning objec- The final step in a participant’s learning-by-doing journey
tives before being presented with more challenging content that is their ability to easily understand, ex-post, the decisions that
combines exposure to multiple risks and opportunities. they made and the outcome of those decisions. Reliable, detailed,
timely and easy to digest feedback and reporting is a crucial
An Example Case Sequence
ingredient for this objective.
Since the RIT market simulator package uses simulated mar-
As discussed in Sections 2.3 and 2.3.2 above, the market ad-
kets to aggregate participants’ decisions, a common case se-
ministrator can observe the results of all market participants
quence with which to begin is a Market Microstructure series. The
in real time. The market administrator can also export a mi-
following is an example sequence.
crosecond timestamped report showing all actions taken by all
(1) AT1 — This purpose of this case is to teach students how participants, and also save the entire set of case parameters so
limit-order books work. Participants familiarize themselves that the exact same parameterized simulation could be run in the
with market orders, limit orders and the concept of market future.
liquidity. A wealth of real-time information (including positions, P&L,
(2) AT2 — This case introduces the idea of short sales (negative etc.) is available to market participants on the RIT Client, allowing
positions in securities), and creates a market that is less liq- them to monitor their progress as they are participating in the
uid, so that students need to utilize limit orders. The price markets. Fig. 4 provides one example that computes Value-at-
paths of the securities are not predictable over the day, Risk in real time to guide participants’ risk taking decisions while
so the objective of the agency trader is to fill customers’ managing a portfolio.
orders using volume-weighted or time-weighted average Following a case replication, market participants are given a
price (VWAP or TWAP) order-entry strategies. detailed trading report in PDF format that visually shows their
(3) LT1 — This case teaches participants about the bid–ask performance, as well as their trading actions through time, and a
spread, and the profits earned by market making (and full attribution breakdown of their profit and loss across securi-
implicitly, the costs of using market orders and paying the ties, as in Figs. 5 and 6.
bid–ask spread). In addition to the performance reporting options that are
(4) LT2 — This case requires participants to generate profits included in the RIT platform, some of the RIT cases include an
via the liquidity spread associated with a large block of instructor tool that generates custom-scripted reports to display
shares. They must utilize their order execution (market and very specific results tailor made to the case objectives. These
limit orders) skills to ensure they do not cause significant instructor tools have been designed to decompose participants’
adverse price movements as they unwind their blocks. performance (for example, Fig. 7) to illustrate their results as
(5) LT3 — Having mastered the skills practiced in the previous they relate to the case objective versus results associated with
cases in this sequence, this case focuses on using links to a other strategies, such as, front-running or unwarranted specula-
decision-support model that aggregates information in the tion. These tools also allow instructors to penalize participants
limit-order books in order to quantify liquidity risk. The for inappropriate strategies or outcomes, for example: exceeding
liability trader (LT) must make a decision as to whether a chief risk officer’s loss limit associated with their regulatory
the liquidity spread offered by the institution is adequate Value-at-Risk measure; or excess risk exposure associated with
to compensate them for the liquidity risk and market risk their delta hedging strategy. This feedback encourages partici-
they will face while unwinding the large block order. As pants to focus on the learning objectives of the case, and con-
such, this case combines model building and decision skills sequently to acquire the skills more quickly, allowing one to
with their trade execution skills. progress more effectively through the case sequence associated
(6) LT4 — This case applies the skills acquired in the LT3 case with a particular topic.
to multiple exchanges. As such, participants need to build
a ‘global’ order book and unwind the block trades across 5. Examples of applications to research and events
two exchanges. Given how difficult it is to manage liquidity
and market risk in real time by manual order entry across Besides supporting teaching and training,7 the RIT Decision
more than one exchange trading the same security, while Cases facilitate competitions and events at many different levels.
satisfying regulatory requirements such as fills at the NBBO Given the high degree of flexibility and customization that is
(National Best Bid Offer), participants soon appreciate the available to the market administrator, low-friction engagement
value of building algorithms, such as a SOR (Smart Order for participants, and high-resolution data logging, the RIT product
Router), for automated order entry. is also a valuable resource for investigating research questions
across a wide-range of topics.
Having acquired the learning objectives and practiced the In particular, the RIT platform has contributed to research pub-
skills in the above sequence of cases, one can then move to the lications in diverse subject areas including behavioral finance, ex-
sequence of ALGO cases to learn how to translate their strategy perimental economics and trading. For example, Patterson (2014)
to code for situations that require very fast decisions, such as, and Nofsinger et al. (2018) studied behavioral effects in markets
arbitrage (ALGO 1 Case), market making (ALGO 2 Case), smart by utilizing the biometric readings of market participants. On the

participants who are managing the liquidity and market risk well will make 7 In addition to applications for students, many financial institutions have
a reasonable return on their bank’s capital across a relatively small number of utilized the RIT software as a training tool for their employees, either on site
replications of the case. or as participants in academic executive programs.
20 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22

Fig. 4. Sample decision-support and monitoring template for a RIT value-at-risk case.

Fig. 5. Time-series of trades covering a tender offer in the liability trading 3 case.

other hand, Gould et al. (2010), Brousseau et al. (2014) and Glik- Morgan used a purpose-built web multi-user webpage to sim-
stein and Kryzanowski (2017) have explored market efficiency, ulate their ‘‘market’’. A replica of their simulation parameters
price discovery and volatility. The RIT platform and RIT Decision has been programmed for the RIT platform and applied to asset
Cases have also been referenced by Latuszynska (2015) and Palan bubbles. The clock games experiment can be carried out with
(2015) in their surveys of experimental software available to significantly less overhead using the RIT application. In addition,
the user experience for test subjects would be better since the RIT
researchers.
platform has been optimized to reduce end-user frictions.
In addition to testing specific research hypotheses, the RIT
The RIT package can provide a strictly controlled environment
application can also be deployed to illustrate and extend existing and precise measurement of inputs and outputs which are funda-
experiments. For example, Brunnermeier and Morgan (2010) test mental for applications to research questions. The ‘requirements
experimentally the gains from waiting versus the risk of being for experimental asset market software’ discussed in Palan (2015)
preempted. This is a classic example of how asset bubbles can are all features of the RIT package. These include his ‘general re-
form, and persist, given certain circumstances. Brunnermeier and quirements’, for example: ‘complete, time-stamped data record’;
 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22 21

Fig. 6. Tabular performance report for the liability trading 3 case.

Fig. 7. Sample custom report decomposing participants’ P & L.

‘customizability and extensibility’; and ‘reliability’; as well as his fair values (determined by the news). Total market liquidity can
‘requirements regarding the market mechanism’ [Palan (2015), be measured based on the sum of trades from all participants. In
section 1.2 and Table 1] including, for example: ‘user-friendly subsequent iterations of the simulation, traders are made aware
interface’; ‘choice between single and multi-unit trading with or that some traders will receive the ‘‘inside information’’ prior to
without wealth transfer’; ‘possibility to trade in multiple markets the rest of the population and their information will be perfect.
or over the counter’; ‘parameter specification’; ‘designated trader The proportion of traders receiving insider information can in-
roles’; ‘short selling’; ‘order validation’; ‘order types and priority’; crease over many iterations. Observing the volume of trade over
and ‘algorithmic trading’. the iterations with various treatments would provide insight into
For example, given the access to a REST API to support
the ways markets may or may not be harmed by insider infor-
decision-support modeling in RIT Decision Cases, combined with
mation. This is parameterized in the simulation by generating
the ‘complete time-stamped data record’ at a millisecond fre-
the final payout at the start of the case (but not revealing it),
quency, one could use the RIT package to test various market
then generating news releases that are distributed based on that
microstructure features or design such as endogenous speed
bumps. final payout with noise terms. Although the payout is ‘‘set’’ at the
Another example of how an experiment could be designed beginning of the case, that information is not revealed until the
and carried out, would be a straightforward case involving the end, resulting in an experience where the payout is revealed over
effects of ‘‘insider information’’ on market dynamics. Participants time.
can be introduced to a case where asymmetric noisy information Probably the most visible application of the RIT platform
is released to all traders and that information is linked to final and RIT Decision Cases has been for special events or compe-
asset prices. Participants are happy to compete, in an attempt to titions. The Rotman International Trading Competition (http://
generate profits when asset prices deviate from their perceived ritc.rotman.utoronto.ca) attracts talented competitors every year
22 K. Mak and T.H. McCurdy / Journal of Behavioral and Experimental Finance 23 (2019) 12–22

from universities around the world; as do the regional competi- Brunnermeier, M.K., Morgan, J., 2010. Clock games: Theory and experiments.
tions, such as the Rotman European Trading Competition (RETC) Games Econom. Behav. 68 (2), 352–550.
in Rome and the Rotman UNIST Trading Competition (RUTC) Cook, D.A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J.H., Want, A.T., Erwin, P.J.,
Hamstra, S.J., 2011. Technology-enhanced simulation for health professions
in South Korea. Furthermore, the adaptability of the RIT Deci-
education: a systematic review and meta-analysis. JAMA 306 (9), 978–988.
sion Cases for participants with different backgrounds and levels
Fama, E.F., French, K.R., 2010. Luck versus skill in the cross-section of mutual
of expertise, has resulted in frequent applications for recruit- fund returns. J. Finance 65 (5), 1915–1947.
ing and university advancement, student extra-curricular clubs Glikstein, E.D., Kryzanowski, L., 2017. Defeating the VPIN: Slipping into the noise.
and events, as well as course-based competitions for decision J. Trading 12 (3), 37–47.
performance components of their course marks. Gould, S., Valenzuela, A., Kachersky, L., Holowczak, R., 2010. The behavioral
dimensions of trading: Proximal and distal influences on performance. In:
6. Concluding comments Campbell, M.C., Inman, J., Pieters, R. (Eds.), Advances in Consumer Research
Vol. 37, 166-169. Association for Consumer Research, MN.
Issenberg, S.B., McGaghie, W.C., Petrusa, E.R., Lee, G.D., Scalese, R.J., 2005.
As summarized above, the RIT product contains many per-
Features and uses of high-fidelity medical simulations that lead to effective
formance feedback tools and reports which facilitate discussion, learning: A BEME systematic review. Med. Teach. 27 (1), 10–28.
provide an opportunity for participants to adapt and improve Latuszynska, M., 2015. Experimental research in economics and computer
their strategies after each replication, and allow instructors to simulation. In: Selected Issues in Experimental Economics, Proceedings
use incentives to encourage participants to focus on the learning of the 2015 Computational Methods in Experimental Economics (CMEE)
objectives of the case and consequently to accelerate learning. Conference. pp. 151–169.
In addition to providing case participants and instructors with McCurdy, T.H., Woodhouse, R.A., 2002. Instructional innovation: experiential
learning. Rotman Management Spring/Summer.
the tools that they need to understand what is happening in real
Nofsinger, J.R., Patterson, F.M., Shank, C.A., 2018. Decision-making, financial risk
time, as well as upon completion of each replication of the sim- aversion, and behavioral biases: The role of testosterone and stress. Econ.
ulated decision cases, it is even more important that the results Human Biol. 29, 1–16.
are reflective of skill. The RIT cases have been calibrated, through Palan, S., 2015. GIMS—Software for asset market experiments. J. Behav. Exp.
extensive testing with a broad spectrum of participants, to deliver Financ. 5, 1–14.
the mix of signal versus noise such that those participants who Patterson, F.M., 2014. The Relation of Steroid Hormones and Personality Factors
are pursuing good strategies will see their skill revealed in the To Financial Performance and Risk-Taking Behavior. PhD Dissertation. Florida
results when averaged across multiple replications of a case. International University.
Salas, E., Wildman, J.L., Piccolo, R.F., 2009. Using simulation-based training to
Achieving the learning objectives associated with each RIT
enhance management education. Acad. Manage. Learn. Educ. 8 (4), 559–573.
Decision Case reflects the culmination of careful design balance Woodhouse, R.A., McCurdy, T.H., 2014. Innovation in management education:
being applied at every level, from the trading interface, to the adding value with simulation-based learning. Manuscript.
case calibration, case sequencing, and finally to the classroom or
lab delivery, feedback and practice.

References

Brousseau, C., Gendron, M., Belanger, P., Coupland, J., 2014. Does fair value
accounting contribute to market price volatility? An experimental approach.
Account. Financ. 54, 1033–1061.