mt5se: An Open Source Framework for Building

Autonomous Trading Robots

Paulo André Lima de Castro
Autonomous Computational Systems Lab - LABSCA
Aeronautics Institute of Technology (ITA - Instituto Tecnológico de Aeronáutica)
São José dos Campos-SP, Brazil
[email protected],[email protected]

Abstract—Autonomous trading robots have been studied in This kind of simulation using real-time data feed from real
artificial intelligence area for quite some time. Many AI tech- markets is very important to identify possible overfitting
arXiv:2101.08169v3 [cs.AI] 28 Jun 2022

niques have been tested for building autonomous agents able in financial machine learning models, as discussed in sec-
to trade financial assets. These initiatives include traditional
neural networks, fuzzy logic, reinforcement learning but also tion II-B1. The framework provides access to several stock
more recent approaches like deep neural networks and deep exchanges (NYSE,Nasdaq, London SE, Tokyo SE, Brazil’s B3
reinforcement learning. Many developers claim to be successful and others) through MetaTrader 5 platform [3]. Besides stock
in creating robots with great performance when simulating exchanges, it also allows trading with Foreign Exchanges pairs
execution with historical price series, so called backtesting. including cryptocurrencies like Bitcoin and Ethereum, through
However, when these robots are used in real markets frequently
they present poor performance in terms of risks and return. the so called Contract For Differences instrument [4]. Further-
In this paper, we propose an open source framework (mt5se) more, it is possible to use the same autonomous trader with
that helps the development, backtesting, live testing and real minor changes in real operations. It is possible to create Trad-
operation of autonomous traders. We built and tested several ing Robots using Neural networks, Random Forests, Support
traders using mt5se. The results indicate that it may help the Vector Machines, Genetic Algorithms, Bayesian Networks,
development of better traders. Furthermore, we discuss the
simple architecture that is used in many studies and propose an Reinforcement Learning, Deep Learning and other techniques
alternative multiagent architecture. Such architecture separates using the wide range of available libraries in python. The
two main concerns for portfolio manager (PM) : price prediction mt5se framework is an evolution of a previous framework
and capital allocation. More than achieve a high accuracy, a PM called mt5b3 [5], which is now deprecated.
should increase profits when it is right and reduce loss when it
is wrong. Furthermore, price prediction is highly dependent of
asset’s nature and history, while capital allocation is dependent
only on analyst’s prediction performance and assets’ correlation.
Finally, we discuss some promising technologies in the area.
A. Organization of the Text
I. I NTRODUCTION
An autonomous trader must be able of selecting, buying and We assume the reader is already familiar with Finance The-
selling financial assets in order to fulfill it’s investor require- ory, including Modern Portfolio Theory [6], Efficient Market
ments about risk and return within the investor’s horizon of Hypothesis [7], Capital Asset Pricing Model (CAPM) and
investment. This period may range from very some seconds Market micro-structure. If that is not the case, a very short
or fractions of seconds (high frequency trading [1], to longer introduction to these concepts are available in section 2 of [8].
periods as several days or even years. It is an autonomous The remainder of this paper is organized as follows: section II
agent that perceives and acts in a very particular environment: discuss the main aspects of modelling autonomous analysts,
financial markets, which brings several complex features that traders and portfolio managers, including some unique charac-
may make it very challenging for autonomous agents. It is teristics of financial markets that make them hard challenges
important to note that there are also ethical and legal implica- for artificial intelligence based agents. We present and analyze
tions that should be taken in consideration in the process of briefly some available frameworks for building autonomous
building automated trading robots. We recommend Wellman traders and compare them with our proposed framework,
and Rajan’s paper [2] about ethical issues in Autonomous called mt5se, which we present minutely in section IV. We
Trading Agents. also provide several examples of autonomous traders and
In this paper, we propose an open source framework portfolio managers and their source code in section V. We
framework for building and testing trading agents, called discuss some open problems in the development of high
mt5se. We present some examples of trading agents devel- performance autonomous traders and we also present some AI
oped using such architecture and the respective results. This related technologies that may contribute to the finance field in
framework allows the development of autonomous traders on section VI. Finally, we conclude the paper with some possible
python language, and the execution of real-time simulation. extensions and suggestion for future work in section VII.
II. AUTONOMOUS A NALYSTS , T RADERS AND P ORTFOLIO given time frame (1 minute, 1 hour or 1 day ). Your Market
M ANAGERS data provider may also allow access to the order books or
The spectrum of used AI techniques in finance field is even to semi-structured information, like FIX messages. In
wide and it includes since reinforcement Learning [9], [10], this case, you may try to recognize patterns that identify some
multiagent systems [11], [12] complex networks [13], genetic traders. For instance, human GUI traders often trade in round
algorithms [14], random forests [15] to more recent approaches lots, and you can use this fact to estimate what percentage of
like deep reinforcement learning [16]. Regardless of the picked the volume is coming from them at a given point in time.
AI technology, there are some aspects that are always present Analytics data is usually provided by human experts. It is
and we observe that quite often AI researchers disregards a kind of derivative data, based on an original source, which
significant aspects of finance theory, as for instance: the could be fundamental, market, alternative, or even a collection
correlation among assets, different investor profiles and risk of other analytics. It is not available from an original source,
control. Dealing with such issues, it is fundamental to build and it was processed in a particular way (hopefully not biased).
better autonomous agents for finance, however it is also really Independent analysts, investment banks and research firms
challenging. sell valuable information that results from deep analyses of
companies’ business models, activities, competition and so on.
A. Types of Financial Data Alternative data may refer to satellite image or video
feeds include monitoring of tankers, tunnel traffic activity,
The data that should be considered by an autonomous
or parking lot occupancy. Before an Oil company reports
analyst is far more diverse and complex than historical prices
increased earnings, before its market price shot up, before
or volumes. It can be splitted in four different types: technical,
analysts wrote their commentary of their latest filings, before
fundamentalist, analytical and alternative data [17]. Figure 1
all of that, there were movements of tankers and drillers and
presents some examples for each type.
pipeline traffic. They happened months before those activities
were reflected in the other data types. What truly characterizes
alternative data is that it is primary information, by that we
mean the information has not made it to the other sources.
There are some problematic aspects of alternative data. It
may bring high cost and privacy concerns, in fact it could
even bring legal concerns. Capture those data may be very
expensive, and the observed company may object, not to
mention bystanders. Furthermore, it is usually hard to process.
Fig. 1. Types of Financial Data However, Alternative data offers the opportunity to work
with unique datasets. Data that is hard to store, manipulate,
Fundamental data encompasses information that can be and operate may be very rewarding, because perhaps your
found in regulatory filings. It is mostly accounting data, competitors did not try to use it for logistic reasons, gave up
reported quarterly or yearly. A particular aspect of this data midway, or processed it incorrectly. Therefore, you may have
is that it is reported with some temporal lapse. You must a truly unexploited field to mining.
confirm exactly when each data point was released, so that
your analysis uses that information only after it was publicly B. Autonomous Analysts
available. A common error is to assume that this data was Investment analysis can be seen as the process of assessment
published at the end of the reporting period, which is almost and selection of investments options in terms of risk and return
never the case. Another problem is the use of back-filled in order to provide advice to the investor or manager. Analysts
or reinstated data. Back-filling means that missing data is are responsible for transforming raw data into information that
assigned some value, even if those values were unknown at can guide investment decisions. The small but relevant differ-
that time. A “reinstated value” is a corrected value that amends ence from traders is that analyst are independent from investor
an incorrect initial release. The problem is, the corrected preferences, investment policies or the amount of capital to be
values were not known on that first release date. The use of managed. Analysts are dependent of the data related to their
such data gives a unrealistic advantage to models, that will not target assets. It is important to observe that some data may be
be present in real operation. In another words, we could say very relevant for some asset, while it is irrelevant or almost
it also leads to overfitting. Nevertheless, it is common to find to another asset. For instance, oil prices can be very relevant
studies that use the final released value and assign it to the to airline companies, but perhaps they are not that relevant
time of the first release, or even to the last day in the reporting to bank stocks. Trading analysis for cryptocurrency prices are
period. most based on historical prices and volumes or some times in
Technical or Market data includes all trading activity that signal obtained from news using natural language processing
takes place in an exchange, including open, high, low and methods (sentiment analysis). In comprehensive survey about
closes prices and their historical series. It is common to use cryptocurrency with 126 research papers, no one used oil
the term bar to refer to open, high, low and close prices in a price to predict cryptocurrency prices [18]. We do not argue
that using oil prices would be a good idea, our point is that
autonomous analysts for different assets may require different
kind of information besides historical prices and volume.
Meanwhile, autonomous trading is the process of submitting
buy or sell orders by an autonomous software agent software
in order to achieve financial goals. It is quite common that
autonomous trading agent deals with just one asset [11].
However, it is well know in finance field that an investor must
consider the relationship among the investments to build a
portfolio that will best meet the investor’s objectives [19]. It
seems that developers of autonomous traders often disregard
two important aspects: investor preferences and the relation-
ship among assets. We address these issues in section II-D
and we use the term autonomous portfolio managers (APM)
to differentiate autonomous agents that deal with such issues
from those who do not, what we call simply autonomous Fig. 2. Single Agent Architecture for Autonomous Traders
traders. Some authors do not make such distinction. We discuss
increasing order of complexity: autonomous analysts ( II-B),
autonomous traders (section II-C) and autonomous portfolio from each other. Likely it is the main reason that leads such
managers (section II-D). studies to focus on creating AT that deals with just one asset
1) Overfitting: The problem of creating models with great at the time. However, it misses important issues regarding
performance in some known data set, but with bad per- dependence among assets.
formance in new data is well known in Machine Learning As stated before, it is important to reason about depen-
field and it is usually called overfitting. Nevertheless, this dencies among assets and investor preferences about risk
problem seems to be unknown or at least disregarded by many and return. For instance, an investor may be more averse
practitioners in autonomous analysis and trading. However, the to risk than others, but lower risk portfolio can be obtained
dangers of failing to avoid overfitting are even more severe by incorporation negatively correlated assets [19]. We address
in the financial field for two reasons. Overfitting is more autonomous traders that care about these important issues in
likely to happen in finance than in traditional machine learning next section.
problems, like face recognition, because of the low ratio
signal-noise and the fact that markets are not IID (independent D. Autonomous Portfolio Managers
and identically distributed). Furthermore, overfitting in finance The portfolio theory proposed by Markowitz [6] become the
leads to over optimistic expectations about performance [20]. de facto standard [19]. Under the portfolio theory assumptions,
For a deeper discussion about the problem, we suggest [8], a single asset or portfolio of assets is efficient if no other
[17] asset or portfolio of assets offers higher expected return with
the same (or lower) risk or lower risk with the same (or
C. Autonomous Traders higher) expected return. It is possible to achieve lower risk
Autonomous traders need to deal with two main concerns: portfolios by including negatively correlated assets in it. In
price prediction and capital allocation, which is also called fact, it is possible to derive portfolios that have lower risk
bet sizing [17]. More than just achieve a high accuracy, an than the assets that compose it. This ability to reduce and
autonomous trader should increase profits when it is right manage risk is the essence of diversification. Many optimiza-
and reduce loss when it is wrong. Furthermore, price (or tion techniques may be used, such as Linear programming,
trend) prediction is highly dependent of asset’s nature and Convex optimization, Quadratic programming, Meta-heuristic
history, while capital allocation is dependent only on investor methods, Genetic algorithm and others. Despite all the work
preferences about risk and return and assets’ correlation. done, portfolio optimization is still an open problem and the
Many studies in autonomous traders (AT) adopt implicitly or available methods have some drawbacks. For instance, it is
explicitly a mono agent architecture that encapsulates most well known that if the mean return vector and the return
of the complexity in a single component that defines orders covariance matrix for the target assets are known, then the
given some information, see Figure 2. It is also common to Markowitz problem has a closed-form solution. However in
define one auxiliary module that collects data from external practice that is never the case, and therefore they are estimated
sources that may be relevant to the trading strategy, that we from historical data. According to some authors [21], it may
refer as Data Collector. Another module to dispatch the defined turn the Markowitz theory impracticable in real portfolio
orders to the market is also common. Despite the simplicity management applications. We suggest [17], chapter 16, for
advantage, such architecture has an important drawback. The a deeper discussion about that.
trading strategy may become very complex as the number of However, it is interesting to observe that autonomous trader
assets increases, specially if the assets’ nature differ very much developers quite often disregard that fact. According to the
portfolio theory, any investor would prefer the efficient port-
folio, for a given level of risk or return. However, different
investor may have different levels of maximum acceptable
risk or minimum acceptable return, therefore they would
prefer different portfolios. Autonomous traders should explore
diversification and be aware of investor preferences, in order to
do a better in trading in the investor’s behalf. As stated before,
We use the term autonomous portfolio manager to refer to an
autonomous trader that uses these facts in its decision process,
despite the fact that other authors use the term, autonomous
trader, for both cases.
Autonomous portfolio managers should also reason about
investor preferences. Naturally, it include preferences about
risk and return, but we have no doubt that ethical, envi-
ronmental and social concerns will have to be incorporated
in autonomous traders in the near future, just like investors Fig. 3. Multi-agent Architecture for Autonomous Portfolio Manager
are demanding these concerns from human investment man-
agers [22].
data high dimensionality and investor preferences awareness,
Investor preferences about return and risk may be expressed
an APM should be able to explain its decisions, just like a
as a maximum acceptable risk or minimum acceptable re-
human portfolio managers are used to. In fact, the ability to
turn [11], but it could also be expressed in terms of a list
provide explanations for its decision, explainability, is consid-
of possible assets and percentage limits for some classes
ered to be essential to create artificial intelligence applications
of assets. For instance, Graham’s book ’Intelligent Investor’
that are trusted by users [24]. The interest in creating AI
suggests an investor should never have less than 25% or
models that can explain their decisions is not new, but it has
more than 75% of her portfolio in common stocks and, at
revived in the last years with machine learning models that are
the same time, never more than 75% or less than 25% in
hard to interpret [25]. Some authors argue the use of inherently
bonds [23]. In terms of autonomous portfolio managers, an
interpretable models rather than searching for methods that
investment policy statement (IPS) may be defined as a class
can interpret the so called black box models [26]. Considering
that implements a method to check if the current portfolio is
these challenges, we proposed a multiagent architecture for
adherent to IPS and other to check a set of intended orders
autonomous portfolio managers that divides these challenges
would make the new portfolio violate the IPS.
among some autonomous agents that working together make
When an APM deals with many assets, it is not hard to
the tasks expected from an APM. Our proposed architecture
realize that each asset will demand different data streams in
aims to be inherently interpretable by providing analyses for
order to be analyzed and it may require a significant aunt
each asset, configurable investment policies and optimization
of data volume for analysing all assets. One should also
algorithm that try to achieve the best portfolio for a given
note that investor information is also part of data needs to
objective function. It facilitates reusing well known approaches
be considered to define the trading strategy. That may bring
to create analysts [27] and optimize portfolios, as discussed
the so called curse of dimensionality. As the number of
in section II-D. At same time, it reduces the problem of
dimensions because the amount of data needed to support
high dimensionality, because each analyst deals only with
result (in a significant way) will likely grow exponentially
data related to its asset independently from other analysts or
with dimensionality. In another hand, it is well known that
investor preferences.
diversification (a large number of target assets) may provide a
better risk-reward relation in portfolios, by reducing the risks. The APM multiagent architecture is presented in figure 3.
So, APMs are required to deal with a large number of assets. It is composed by analyst agents and one allocator agent. An
The challenges discussed here: Curse of dimensionality, di- analyst provides a return distribution (analysis) for a given
versification and investor preferences awareness make the task asset using several concurrent models. Using such analyses,
of creating effective autonomous portfolio managers very hard. the allocator may calculate target prices, expected returns and
It may be even harder if one decides to use the single agent optimize the portfolio to maximize an objective function, the
architecture. For these reasons, we propose an multiagent Sharpe ratio, for instance. The APM architecture also counts
architecture, which is described in section II-E. with some additional software components that are based in
traditional (non AI based) algorithms. These are shown as
E. Multi-agent Architecture for Autonomous Portfolio Man- rectangles in figure 3, while autonomous agents are shown
agers as rounded rectangles. These components are described next.
There are many challenges for real autonomous portfolio • Data Collector: it collects relevant data about all target
managers (APM). Besides the challenges discussed in sec- assets that may be used by the analysts and their models.
tion II-D: risk mitigation by diversification, dealing with the It parses and integrates information from different data
 sources in a format that can be used by the analysts. useful, but it does not provide a comprehensive evaluation
• Analyst: Autonomous agents that provide an analysis of the manager’s performance. It is absolutely necessary to
about the asset, ideally a return distribution. Naturally, control for the portfolio risk [19]. However, it is quite common
it is dependent of the asset, but it is independent of to observe in evaluation of autonomous trader without a proper
investor’s preferences and capital. Ensemble several mod- control for the risk of the portfolio. We argue that there is no
els to create an analyst is a good idea, since it can good reason to evaluate autonomous traders (or autonomous
provide better predictions but also help to understand portfolio managers) using different methods than those used
the reason behind its predictions. The analysts estimate for human professionals. Despite the fact that there is not a
return distributions for each asset using their best effort single universally accepted method for evaluating portfolio
and available information. performance adjusted to risk, there are several techniques
• Allocator: It uses the analyses provided by the analysts that are used in practice, among them Sharpe’s ratio [29].
and according with investor profile it searches for the best It is widely used despite some criticism about it. For in-
allocation (or re-allocation) of resources among the target stance, high outlier returns could increase the value of the
assets. It does not need to know more about the assets Sharpe ratio’s denominator (standard deviation) more than the
itself, since it works with the analyses, therefore given value of the numerator (return in excess), that way the ratio
the analyses, it is asset independent. would be lower thereby lowering the value of the ratio and
• Investment Policy Statement (IPS) Checker: The set for positively skewed return [30]. Some alternative methods
of orders are checked to ensure that they respect the are also available, as for instance: Sortino ratio [31] and
investment policy statement. Orders can be ignored or Modigliani index [32]. Furthermore, it is important to measure
changed by the IPS checker to warrant its goal. performance for a proper track record length [33]. There is a
• Order Dispatcher: Once the orders coherent with IPS solid literature about evaluation of portfolio performance [19],
are defined, these orders need to be dispatched to some but it goes beyond the scope of this paper to cover it here.
Market (real or simulated) to be completely or partially
executed or even not executed according to the market
conditions. That is the role of the Order Dispatcher. III. C OMPARATIVE A NALYSIS OF F RAMEWORKS FOR
AUTONOMOUS T RADERS
The multiagent architecture may be seen as a very sim-
plified structure of a hedge fund, which could be described
in three parts: analysts team, portfolio manager and traders. In table I, we present a comparative analysis of some
The analysts make analysis of the target assets and pass it to selected systems with similar propose of mt5se. Such analysis
the manager. The manager checks the portfolio composition is based on some features that facilitate the development and
and makes new positions based on their risk control strategy test of autonomous trading strategies. We do not intend to
and the analysts’ analysis. Ultimately the positions updates judge the overall quality of the cited systems, but just identify
are executed by traders. Naturally, it is an overly simplified differences (positive and negative) with the mt5se framework
explanation about a hedge fund work process. proposed here. It is relevant to note that mt5se relies on
MT5 [3] to connect to stock markets. The MT4 [34] is not
F. Performance Measure for Autonomous Analysts and APMs just an earlier version of MetaTrader 5, but an alternative tool
Autonomous Analysts as described earlier are concerned focused on Forex markets. While, MetaTrader 5 allows trading
with asset price prediction. Therefore it is easy to note that in Forex or stock markets, however it comes with the cost of
their performance may be measured as traditional machine higher complexity and two accounting systems: netting and
learning models for classification (in case price is discretized) hedging systems. Therefore, MT4 is still wildly used for Forex
or regression (continuous price). Concepts like accuracy, operations. The other system, called AgEx [35], is an open
mean-square error, cross-validation and other common asso- source financial market simulation tool that uses FIPA ACL
ciated to machine learning evaluation are all relevant, but language and a market ontology created specifically to be used
one should be special careful to avoid overfitting since it for trader agents.
is more dangerous and likely in finance [17]. Furthermore,
one should consider taking into account the cost of errors. If Feature MT4 AgEx MT5 mt5se
Real Operation support Yes No Yes Yes
an Autonomous Analyst (AA) predicts a high return and the Stock operation support No Yes Yes Yes
return is even higher than expected, it may be considered an Forex operation support Yes No Yes Yes
error, but certainly if the real return were negative it would Cryptocurrency operation support Yes No Yes Yes
be a much worse error. For a deeper discussion about cost of Open source No Yes No Yes
Python support No No No Yes
errors in autonomous trading see [28]. On the other hand, in Object Oriented No Yes Yes Yes
order to address evaluation of autonomous traders or APM, Backtest support Yes Yes Yes Yes
we need to review evaluation of human portfolio managers. TABLE I
There are many possible procedures to evaluate portfolio C OMPARISON AMONG S ELECTED S YSTEMS
manager’s performance beyond the simple historical return
comparison with other managers. Such comparison may be
 IV. B UILDING AUTONOMOUS T RADERS USING MT 5 SE B. Testing Autonomous Traders
In this section, we present how to build and test autonomous Once you have built an autonomous trader, you need to
traders using the mt5se framework. It allows access to price verify if it is suitable for operation. The basic form of testing
data (open, close, high, low) and book data (bid, ask) and an autonomous trader is often called backtest. It is a kind of
order placement in simulated or real financial markets. It was evaluation for trading robots. It is basically a trading robot
designed to operate in any financial market accessible through executing with historical price series , and its performance
Metatrader platform, that uses the so called netting accounting is computed. In backtesting, time is discretized according
system. It was tested in simulated and real operation connected with bars and mt5se controls the information access to the
just to the Brazilian stock exchange (B3) and Nasdaq Ex- Trader according with the simulated time. As simulation time
change, but in principle, it could used to trade in any stock advances, the function ’trade’ is called and receives the new
exchange with Brokers that provide access to MetaTrader bar info and decides which orders to send. In order to backtest
platform. There are several Brokerage company that provide one strategy, one just need to create a Trader, establish the test
such access for Nasdaq for instance, TradeView [36] and in parameters and execute it. These parameters define trading
B3, XP [37], Clear [38], among others. guidelines (target assets, available capital and horizon of
The framework mt5se may be used to create simple trading investment) and some system parameters that define operation
robots based on algorithms, but it also supports the creation and log register. In listing 3, we present an example of backtest
of traders based on Artificial Intelligence techniques, such as definition and execution.
decision trees [39], neural networks [40], Bayesian dynamic 1 # trading data options
networks [41], reinforcement learning [42] and so on. 2 capital=100000
results_file=’data_equity_file.csv’
In listing 1, we present a very simple autonomous trader 34 assets=[’PETR4’,’VALE3’,’ITUB4’]
built using mt5se. It is able to trade with any number of target 5
assets, but it does that randomly. It select aleatory a number of 6 #backtest options
prestart=se.date(2019,12,10)
shares between and 1 and 1000, and then also aleatory picks 78 start=se.date(2019,1,10)
buying or selling that number of shares at market price for 9 end=se.date(2019,2,27)
each asset, at each moment. 10 # Use True if you want debug information for your
Trader
1 import numpy.random as rand 11 verbose=False
2 class RandomTrader(se.Trader): 12 #sets the backtest setup
3 def trade(self,dbars): 13 period=se.DAILY
4 orders=[] 14 # it may be se.INTRADAY (one minute interval)
5 assets=list(dbars.keys()) 15
6 for asset in assets: 16 bts=se.backtest.set(assets,prestart,start,end,period
7 if rand.randint(2)==1: ,capital,results_file,verbose)
8 order=se.buyOrder(asset,100) 17 #create trader instance
9 else: 18 trader=RandomTrader()
10 order=se.sellOrder(asset,100) 19 # Running the backtest
11 orders.append(order) 20 df= se.backtest.run(trader,bts)
12 return orders
Listing 3. Backtesting an Autonomous Trader
Listing 1. Random Autonomous Trader Example
If we execute the listing 3, we backtest the trader presented
in listing 1. As we said before, the random trader does make
A. Installation and Further information about mt5se intelligent decisions, just picks randomly numbers of shares
The framework mt5se may be download from github or and sells or buys them. Naturally, its performance may vary
PyPI platforms. In its repository [43], there are also available widely. In order to evaluate the trader’s performance, you may
tutorials presenting mt5se API and how to create simple use the function se.backtest.evaluate that generates a report,
autonomous Trader, and also Traders based on AI techniques. see listing 4. In figure 4, we present two very different results
Some jupyter notebooks with several examples are also avail- for the Random Trader using the same assets and trading
able. In order to install mt5se in a Python, you may just use period. In execution (a), the performance is very good, where
one of the ways presented in listing 2 trader achieves profit (13.18% return) in less than two months
1 # this package is required by mt5se
(January 10 to February 27, 2019), however another execution
2 pip install MetaTrader5 (b) using the same random trader and period achieved a
3 # installing mt5se package negative return of 21.49%.
4 pip install mt5se
5 1
6 # within a jupyter notebook, you may use: 2 #Run the trader according setup and get the results
7 #import sys 3 df=se.operations.run(trader,ops)
8 #!{sys.executable} -m pip install Metatrader5 4 #evaluate the results
9 #!{sys.executable} -m pip install mt5se 5 se.backtest.evaluate(df)
6 #Alternatively, you can evaluate using the generated
Listing 2. Installing mt5se file
7 #se.backtest.evaluateFile(fileName)
 8 #fileName is the name of file generated by the 17 else:
backtest 18 order=None
19 if order!=None:
Listing 4. Evaluating performance an Autonomous Trader 20 orders.append(order)
21 return orders
We need to note that it is hard to perform meaningful eval-
uations using backtest. There are many pitfalls to avoid and Listing 5. Autonomous Trader suitable for real operation
it may be easier to get trading robots with great performance In listing 5, we present the complete code of an autonomous
in backtest, but that perform really badly in real operation. trader called MultiAssetTrader, ready for real operations. It is
For a deeper discussion about trading strategies evaluation, based on a simple interpretation of the Relative Strength Index
we suggest [8], [17] and [33]. (RSI), and splits equally the available capital among assets.
1 #trading data
2 # target assets (three of the main assets in B3)
3 assets=[’PETR4’,’VALE3’,’ITUB4’]
4 # available capital
5 capital=100000
6
7
8 # Options for ending time of operation
9 # endTime=se.now(minOffset=1) # it will run for
one minute!
10 # endTime=se.now(hourOffset=1,minOffset=30) # the
trader will run for 1:30h after started!
11 endTime=se.operations.sessionEnd() # it will run by
the end of session!
12
Fig. 4. Two different results for the same Trader and setup
13 #if market not open, keep waiting
14 waitForOpen=True
After achieving good performance in backtesting properly 15 #get information and decise every minute
conceived and executed, the next phase would be to operate 16 timeFrame=se.INTRADAY
in real market with limited portfolio. As mentioned before 17 18 #Connect to B3 using default account in MT5
you can use mt5se traders in real environments, but you may 19 se.connect()
have to perform some minor changes in a trader so it can 20
operate in real mode and you need to define the operational 21 22 # System information
parameters, which are a little different from backtest parame- 23 # gives information during executing
ters. In listing 6, we show how to establish the parameters for 24 verbose=True
# operations register file
trader’s operation, create a trader instance from a class called 25
26 data_file=’data_equity_file.csv’
MultiAssetTrader and run it according with its setup. Some 27 # seconds to wait between trade calls
brokers provide dedicated servers and accounts for simulated 28 delay=1
# number of bars to take in each decision
operations, often called demo accounts. For instance, XP Inc is 29
30 mem=10
one of those brokers in B3 stock exchange that provide demo 31 # setup operation (ops)
accounts. Despite being simulated account, under the point of 32 ops=se.operations.set(assets,capital,\
endTime, mem,timeFrame,data_file,\
view of an autonomous trader they are just like real accounts 33 34 verbose,delay,waitForOpen)
and can use exactly the same code, as presented in listings 6 35
and 5. 36 # Create an instance of the trader
37 trader=MultiAssetTrader()
1 class MultiAssetTrader(se.Trader): 38
2 def trade(self,dbars): 39
3 assets=dbars.keys() 40 #Run the trader according setup
4 orders=[] 41 se.operations.run(trader,ops)
5 for asset in assets:
6 bars=dbars[asset] Listing 6. Setup and Running an Autonomous Trader for real operation
7 curr_shares=se.get_shares(asset)
8 money=se.get_balance()/len(assets) #
divide o saldo em dinheiro igualmente entre os C. Autonomous traders based on AI
ativos In this section, we are going to present some AI powered
9 # number of shares that you can buy of
asset trading robots, based on Machine learning and Artificial Intel-
10 price=se.get_last(bars) ligence algorithms. In listing 7, we present an example based
11 free_shares=se.get_affor_shares(asset, on Random Forests that encompass the whole decision pro-
price,money)
12 rsi=se.tech.rsi(bars) cess, taking into account data about target assets and investor
13 if rsi>=70 and free_shares>0: information. It is based on Random Forest classifier. Another
14 order=se.buyOrder(asset,free_shares) option would be split those concerns in different models or
15 elif rsi<70 and curr_shares>0:
16 order=se.sellOrder(asset,curr_shares agents, as discussed in II-E. We present an autonomous trader
) based on the first alternative, using Random Forests [44] and
 deals with investor preferences in a very simplistic way, by curr_shares)
dividing the capital equally among the assets and using the 60 else:
61 order=None
same model in listing 7 62 if order!=None:
1 ## Defines the Simple AI Trader 63 orders.append(order)
2 from sklearn.ensemble import RandomForestClassifier 64 return orders
65
3 from sklearn.preprocessing import KBinsDiscretizer
4
66 # creates instance of the Simple AI Trading
5 class RandomForestTrader(se.Trader): 67 trader=RandomForestTrader()
6
7 def setup(self,dbars): Listing 7. Example of Autonomous Trader based on AI (Random Forest)
8 assets=list(dbars.keys())
9 if len(assets)!=1: D. Building AI-based Autonomous Traders using historical
10 print(’Error, this trader is supposed to
deal with just one asset’)
data
11 return None It is quite common to use historical data to build au-
12 bars=dbars[assets[0]]
13 # Data preparation
tonomous analysts or traders using historical data to build
14 timeFrame=10 Machine Learning models. The training process may take
15 horizon=1 # it projects the closing price significant amount of computer processing time. The data
for next bar
16 attr_list=[’open’,’close’,’MA’]
preparation, which includes gathering, cleaning and selecting
17 features, and also creating new features based on available
18 #getting bars info information, is fundamental and often it requires a lot of effort
19 bars=se.get_bars(’PETR4’,timeFrame*5)
20 # creating a new feature
from practitioners and researches. In this section, we provide
21 bars[’MA’]=se.tech.ma(bars[’close’]) some source code examples of Data preparation, ML models
22 creation using mt5se and Sci-kit-learn [45]. In listing 8, we
23 # you may use get_XY providing all info
24 target=’close’
present a code snippet that get information about one asset,
25 X,y=se.ai_utils.get_XY(bars,attr_list,target and creates a supervised dataset with independent features (X)
,timeFrame,horizon) and a target feature (Y), which is the future price of the given
26
27 discretizer = KBinsDiscretizer(n_bins=3,
asset in a given time horizon. Then, we create an ML model
encode=’ordinal’, strategy=’uniform’) to predict Y given the independent features in a specific time
28 # creates the discrete target frame in listing 9. The data preparation transform a time series
29 dy=discretizer.fit_transform(y)
30
into a suitable dataset to use with sci-kit learn framework, but
31 #clf = tree.DecisionTreeClassifier() that could be used in several other frameworks. The figure 5
32 clf = RandomForestClassifier(n_estimators illustrates such transformation.
=10)
33 clf = clf.fit(X, dy)
34 self.clf=clf
35
36 def trade(self,dbars):
37 assets=dbars.keys()
38 orders=[]
39 timeFrame=10
40 horizon=1
41 attr_list=[’open’,’close’,’MA’]
42 money=se.get_balance()/len(assets) #
shares the balance equally among the assets
43 for asset in assets:
44 bars=dbars[asset]
45 curr_shares=se.get_shares(asset)
46 price=se.get_last(bars)
47 free_shares=se.get_affor_shares(
asset,price,money)
48 # get new information (bars),
transform it in X Fig. 5. Data Preparation Example: From bars to X and Y features with time
49 bars=dbars[asset] frame (tF)=2, horizon(h)=1 and [x1, x4] selected as relevant features. X and
50 bars[’MA’]=se.tech.ma(bars[’close’]) Y are numpy arrays
51 X=se.ai_utils.get_X(bars,attr_list,
timeFrame,horizon)
52 # predict the result, using the 1 # Data preparation
latest info 2 timeFrame=10
53 p=self.clf.predict([X[-1]]) 3 horizon=1 # it projects the closing price for next
54 if p==2: bar
55 #buy it 4 attr_list=[’open’,’close’,’MA’]
56 order=se.buyOrder(asset, 5
free_shares) 6 #getting bars info
57 elif p==0: 7 bars=se.get_bars(’PETR4’,timeFrame)
58 #sell it 8 # creating a new feature
59 order=se.sellOrder(asset, 9 bars[’MA’]=se.tech.ma(bars[’close’])
10 21 joblib.dump(clf, ’model.joblib’)
11 # getting X (independent features)
12 X=se.ai_utils.get_X(bars,attr_list,timeFrame,horizon Listing 10. Creating a ML model for deciding trader’s actions and saving it.
)
13 1 ## Loading a pre-trained model to create an AI based
14 #getting Y (dependent feature) Trader
15 y=se.ai_utils.get_Y(bars,target,timeFrame,horizon) 2 from sklearn.ensemble import RandomForestClassifier
16 3 from sklearn.preprocessing import KBinsDiscretizer
17 # Alternatively, you may use get_XY providing all 4 import joblib
info 5
18 X,y=se.ai_utils.get_XY(bars,[’open’,’close’,’MA’], 6 class RandomForestTrader(se.Trader):
target,timeFrame,horizon) 7
19 8 def setup(self,dbars):
20 # Example of Discretization (it may be required by 9 fileName=’model.joblib’
the chosen ML technique 10 self.clf=joblib.load(fileName)
21 # Discretization 11
22 12 def trade(self,dbars):
23 from sklearn.preprocessing import KBinsDiscretizer 13 assets=dbars.keys()
24 14 orders=[]
25 discretizer = KBinsDiscretizer(n_bins=3, encode=’ 15 timeFrame=10 # it takes into account the
ordinal’, strategy=’uniform’) last 10 bars
26 16 horizon=1 # it projects the closing
27 dy=discretizer.fit_transform(y) # you make each price for the next bar
value assume a discrete value [0,1,2,....] 17 attr_list=[’open’,’close’,’MA’]
according to discretization strategy 18 money=se.get_balance()/len(assets) #
shares the balance equally among the assets
Listing 8. Example of Data preparation for Training ML models 19 for asset in assets:
20 bars=dbars[asset]
1 # Creating an AI model 21 curr_shares=se.get_shares(asset)
2 from sklearn.ensemble import RandomForestClassifier 22 price=se.get_last(bars)
3 clf = RandomForestClassifier() 23 free_shares=se.get_affor_shares(
4 asset,price,money)
5 # Training an AI based Trader 24 # get new information (bars),
6 clf = clf.fit(X, dy) transform it in X
7 25 bars=dbars[asset]
8 # Classifing instances 26 bars[’MA’]=se.tech.ma(bars[’close’])
9 p=clf.predict([X[-3]]) 27 X=se.ai_utils.get_X(bars,attr_list,
10 p=int(p) timeFrame,horizon)
11 labels=[’sell it’,’buy it’,’do nothing’] 28 # predict the result, using the
12 latest info
13 for i in range(-8,-15,-1): 29 p=self.clf.predict([X[-1]])
14 p=clf.predict([X[i]]) 30 if p==2:
15 l=int(p) 31 #buy it
16 print(’X[’,i,’]:’,’decision=’,labels[l], ’ 32 order=se.buyOrder(asset,
prediction=’,p) free_shares)
17 # Now let’s save the trained model 33 elif p==0:
18 import joblib 34 #sell it
19 35 order=se.sellOrder(asset,
20 print(type(clf)) curr_shares)
21 joblib.dump(clf, ’model.joblib’) 36 else:
37 order=None
Listing 9. Creating a ML model for deciding trader’s actions and saving it. 38 if order!=None:
39 orders.append(order)
1 # Creating an AI model 40 return orders
2 from sklearn.ensemble import RandomForestClassifier 41

3 clf = RandomForestClassifier() 42 # creates instance of the Simple AI Trading

4 43 trader=RandomForestTrader()
5 # Training an AI based Trader
Listing 11. Creating a mt5se Trader using a pre-trained Random Forest model
6 clf = clf.fit(X, dy)
7
8 # Classifing instances Giving a ML model already trained, the trader example
9 p=clf.predict([X[-3]]) based on Random Forest presented in listingc 7 can be sim-
10 p=int(p) plified. It does not train the ML model in the setup function,
11 labels=[’sell it’,’buy it’,’do nothing’]
12
instead it simply loads the pre-trained model as presented in
13 for i in range(-8,-15,-1): listing 11, the remaining code stays the same presented in
14 p=clf.predict([X[i]]) listingc 7.
15 l=int(p)
16 print(’X[’,i,’]:’,’decision=’,labels[l], ’
prediction=’,p)
V. E XAMPLES OF AUTONOMOUS T RADERS AND
17 # Now let’s save the trained model P ORTFOLIO M ANAGERS USING MT 5 SE
18 import joblib
19
In this section, we present the results achieved by im-
20 print(type(clf)) plementing three autonomous traders and one Autonomous
 Value - Strategy MA RSI RFOR
Portfolio Manager. We have executed some backtest and An. Return (%) 28.49 28.63 28.63
present the achieved results. In order to build such examples An. Sharpe Ratio (%) 37.51 37.48 37.48
and backtest them. We have implemented a open source python Volatility (%) 0.49 0.49 0.49
framework, called mt5se. It provides some basic algorithms TABLE VI
and allows backtest using historical prices or real time opera- R ESULTS FOR THE THREE STRATEGIES FOR ASSET MSFT
tion. Such operation may be in simulated or real accounts. The
framework mt5se uses MetaTrader 5 platform to access real
markets. We provide more information about mt5se framework Value - Strategy MA RSI RFOR
in appendix IV. An. Return (%) -1.38 -0.99 -0.99
An. Sharpe Ratio (%) 3.45 3.84 3.84
A. Backtest setup Volatility (%) 0.63 0.62 0.62
TABLE VII
We executed twelve scenarios composed by three strategies R ESULTS FOR THE THREE STRATEGIES FOR ASSET UAL
(table II) and four different assets (table III), over the period
of five quarters of trading data (Oct-1-2018 to Dec-31-2019).
# ID Strategy name Value - Strategy MA RSI RFOR
1 RSI Relative Strength Index An. Return (%) 12.80 12.82 12.85
2 MA Moving Average An. Sharpe Ratio (%) 15.28 15.27 15.27
3 RFOR Random Forest
TABLE VIII
TABLE II AVERAGE R ESULTS FOR THE THREE STRATEGIES AND FOUR ASSETS
S ELECTED T RADING STRATEGIES

C. Autonomous Portfolio Manager Example

# Symbol Description
1 AAL American Airlines Group Here, we present an example of an Autonomous Portfolio
2 AMD Advanced Micro Devices Manager Example built with mt5se framework. This APM uses
3 MSFT Microsoft
4 UAL United Airlines Holdings
three analyst, which source code is presented in listing 12. Two
of them are based on technical indicators (Relative Strength
TABLE III
F OUR S ELECTED ASSETS FROM THE MOST TRADED ASSETS IN NASDAQ Index - RSI and Moving Average- MA) and the third analyst is
based on a Random Forest classifier. These analysts are used
as inputs to our APM in listing 13 to improve the accuracy of
estimated returns. The APM uses an optimizer to define the
B. Results
capital allocation. This optimizer is based on Sharpe index
In the next tables, we present the achieved results using maximization and uses an implementation provided by the
the three trading algorithms in each of the four assets in the PyPortfolioOpt library [46].
evaluated period (Oct-1-2018 to Dec-31-2019). For each sim-
1 import mt5se as se
ulation defined by scenario, stock and trading algorithm, we 2 import pandas as pd
observed the following values: Annualized Return, Annualized 3 import numpy as np
Sharpe Ratio and Volatility. We present the achieved results 4 from pypfopt import expected_returns
5 from sklearn.ensemble import RandomForestClassifier
in tables IV, V, VI and VII. 6 from sklearn.preprocessing import KBinsDiscretizer
7
Value - Strategy MA RSI RFOR 8 class RsiAnalyst(se.Analyst):
An. Return (%) -26.30 -26.39 -26.30 9 def setup(self,dbars):
An. Sharpe Ratio (%) -18.25 -18.52 -18.24 10 assets=list(dbars.keys())
Volatility (%) 0.822 0.817 0.822 11 df=se.get_close_prices_from_dbars(assets,
TABLE IV dbars)
R ESULTS FOR THE THREE STRATEGIES FOR ASSET AAL 12 # train model
13 self.mu = expected_returns.
mean_historical_return(df)
14 self.alpha=0.5
15 self.dbars=dbars
Value - Strategy MA RSI RFOR
16
An. Return (%) 36.22 36.22 36.22
17 def analyze(self,dbars):
An. Sharpe Ratio (%) 26.57 26.57 26.56 18 assets=dbars.keys()
Volatility (%) 1.22 1.22 1.21 19 returns=dict()
TABLE V 20 mul=self.mu
R ESULTS FOR THE THREE STRATEGIES FOR ASSET AMD 21 alpha=self.alpha
22 for asset in assets:
23 bars=dbars[asset]
24 # number of shares that you can buy of
asset
25 rsi=se.tech.rsi(bars)
26 er=self.mu[asset]
27 if rsi>=70: #buy 85 self.steps=dict()
28 exp_ret=er+alpha*abs(er) 86 self.alpha=0.5
29 elif rsi<70: #sell 87 self.dbars=dbars
30 exp_ret=er-alpha*abs(er) 88 self.mu=mu
31 returns[asset]=exp_ret 89 def analyze(self,dbars):
32 return returns 90 assets=dbars.keys()
33 91 returns=dict()
34 class MAAnalyst(se.Analyst): 92 timeFrame=10 # it takes into account the
35 def setup(self,dbars): last 10 bars
36 assets=list(dbars.keys()) 93 horizon=1 # it project the closing price
37 df=se.get_close_prices_from_dbars(assets, for next bar
dbars) 94 target=’close’ # name of the target
38 # train model column
39 mu = expected_returns.mean_historical_return 95 for asset in assets:
(df) 96 # get new information (bars),
40 self.alpha=0.5 transform it in X
41 self.mu=mu 97 bars=dbars[asset]
42 self.period=10 98 #remove irrelevant info
43 99 if ’time’ in bars:
44 def analyze(self,dbars): 100 del bars[’time’]
45 assets=dbars.keys() 101 # convert from bars to dataset
46 returns=dict() 102 ds=se.ai_utils.bars2Dataset(bars,
47 for asset in assets: target,timeFrame,horizon)
48 bars=dbars[asset] 103 # Get X fields
49 # number of shares that you can buy of 104 X=se.ai_utils.fromDs2NpArrayAllBut(
asset ds,[’target’])
50 er=self.mu[asset] 105 # predict the result, using the
51 m=np.mean(bars[’close’][-self.period:]) latest info
52 if se.tech.trend(bars[’close’])>0 and 106 p=self.clf[asset].predict([X[-1]])
bars[’close’].iloc[-1]<m: 107 er=self.mu[asset]
53 exp_ret=er+self.alpha*abs(er) 108 if p==2:
54 elif se.tech.trend(bars[’close’])<0 and 109 exp_ret=er+self.alpha*abs(er)
m<bars[’close’].iloc[-1]: #buy it
55 exp_ret=er-self.alpha*abs(er) 110 elif p==0:
56 else: 111 #sell it
57 exp_ret=None 112 exp_ret=er-self.alpha*abs(er)
58 returns[asset]=exp_ret 113 else:
59 return returns 114 exp_ret=None
60 115 returns[asset]=exp_ret
61 class RandomForestAnalyst(se.Analyst): 116 return returns
62 def setup(self,dbars):
63 assets=list(dbars.keys()) Listing 12. Examples of Autonomous Analysts
64 df=se.get_close_prices_from_dbars(assets,
dbars) 1 import mt5se as se
65 mu = expected_returns.mean_historical_return 2 import pandas as pd
(df) 3 import numpy as np
66 self.clf=dict() 4 from pypfopt.efficient_frontier import
67 for asset in assets: EfficientFrontier
68 bars=dbars[assets[0]] 5 from pypfopt import risk_models
69 # remove irrelevant info 6 from pypfopt import expected_returns
70 if ’time’ in bars: 7
71 del bars[’time’] 8 class PortOptTrader(se.Trader):
72 timeFrame=10 # it takes into account the 9 def setup(self,dbars):
last 10 bars 10 self.dbars=dbars
73 horizon=1 # it project the closing price 11 assets=list(dbars.keys())
for next bar 12 df=se.get_close_prices_from_dbars(assets,
74 target=’close’ # name of the target self.dbars)
column 13 # train model
75 ds=se.ai_utils.bars2Dataset(bars,target, 14 mu = expected_returns.mean_historical_return
timeFrame,horizon) (df)
76 X=se.ai_utils.fromDs2NpArrayAllBut(ds,[’ 15 S = risk_models.sample_cov(df)
target’]) 16 self.mu=mu
77 discretizer = KBinsDiscretizer(n_bins=3, 17 self.S=S
encode=’ordinal’, strategy=’uniform’) 18 #Create analysts
78 # creates the discrete target 19 self.analysts=list()
79 ds[’target’]=se.ai_utils.discTarget( 20 anl=se.analysts.RsiAnalyst()
discretizer,ds[’target’]) 21 anl.setup(dbars)
80 Y=se.ai_utils.fromDs2NpArray(ds,[’target 22 self.analysts.append(anl)
’]) 23 anl=se.analysts.MAAnalyst()
81 # train model for each asset 24 anl.setup(dbars)
82 clf = RandomForestClassifier( 25 self.analysts.append(anl)
n_estimators=10) 26 anl=se.analysts.RandomForestAnalyst()
83 clf = clf.fit(X, Y) 27 anl.setup(dbars)
84 self.clf[asset]=clf 28 self.analysts.append(anl)
29 necessary to adopt the feasible portfolio and it is shown in
30 def estimate_exp_returns(self,dbars,mu): listing 14. The basic idea is buying the closet possible share
31 assets=dbars.keys()
32 #execute analysts and update expected units to desired weights and redistribute the remain capital, if
returns present, among under allocated assets.
33 analysts_mus=list()
34 for analyst in self.analysts: 1 def volumes_from_weights(assets,weights,last_prices,
35 analysts_mus.append(analyst.analyze( capital):
dbars)) 2 # sort weights from highest to lowest
36 return se.analysts.ensembleAnalyses( 3 weights=dict(sorted(weights.items(), key=lambda
analysts_mus,mu) item:item[1],reverse=True))
37 4 #round 1 - buy while never exceeds the desired
38 def trade(self,bts,dbars): weight
39 order_list=[] 5 sum=0
40 capital=se.backtest.getBalance(bts) 6 curr=dict() # current weights
41 assets=list(dbars.keys()) 7 volumes=dict() # order’s volumes
42 #for asset in assets: 8 steps=get_volume_steps(assets)
43 # self.dbars[asset]=self.dbars[asset]. 9 for asset in assets:
append(dbars[asset]) 10 aval_capital=weights[asset]*capital
44 # self.dbars[asset]=self.dbars[asset]. 11 shares=getAfforShares(asset,aval_capital,
reset_index(drop=True) last_prices[asset],steps[asset])
45 mu=self.mu 12 if shares<=0:
46 S=self.S 13 curr[asset]=0
47 mu=self.estimate_exp_returns(dbars,mu) # 14 volumes[asset]=0
get the expected returns from an ensemble of 15 else:
analysts 16 curr[asset]=(shares*last_prices[asset])/
48 ef = EfficientFrontier(mu, S) capital
49 weights = ef.max_sharpe() 17 volumes[asset]=shares
50 cleaned_weights=ef.clean_weights() 18 sum=sum+curr[asset]
51 #create orders using cleaned_weights: 19 #round 2 - if there is remainig capital, buy
52 self.cleaned_weights=cleaned_weights more lot according weight order
53 # Calculate expected returns and sample 20 remain_capital=capital-sum*capital
covariance 21 if remain_capital<=0:
54 last_prices=se.get_last_prices(assets,dbars) 22 return volumes
55 orders=se.volumes_from_weights(assets, 23 for asset in weights.keys():
cleaned_weights,last_prices,capital,True) 24 s=steps[asset]
56 curr_shares=se.get_curr_shares(assets) 25 p=last_prices[asset]
57 new_orders=se.get_orders_from_curr_shares( 26 missing=(weights[asset]-curr[asset])*capital
orders,curr_shares) 27 while s*p<remain_capital and s*p<missing:
58 for asset in assets: 28 s=s+steps[asset]
59 if new_orders[asset]>0: #buy it 29 if s*p>remain_capital :
60 order=se.buyOrder(asset,abs( 30 s=s-steps[asset]
new_orders[asset]),last_prices[asset]) 31 curr[asset]=curr[asset]+(s*p)/capital
61 elif new_orders[asset]<0: #sell it 32 volumes[asset]=volumes[asset]+s
62 order=se.sellOrder(asset,abs( 33 remain_capital=remain_capital-s*p
new_orders[asset]),last_prices[asset]) 34 if remain_capital<=0:
63 else: 35 break
64 order=None 36 return volumes
65 if order!=None: Listing 14. From portfolio weights to orders’ volume considering capital
66 order_list.append(order) restriction
67 return order_list
Listing 13. Example of Autonomous Portflio Manager using Three D. Discussion
Autonomous Analysts (APM)
As presented in table VIII, the achieved results by the
In portfolio optimization, it is a common assumption that three strategies are very similar in terms Annualized return
there is a big amount of capital available and therefore it or Annualized Sharpe Ratio. Even when we analyze for
is possible to diversify your portfolio among many different individual assets, the performance among strategies is very
assets; transactions costs are not relevant and it is possible similar, that happens because when dealing with just one asset
to invest in a given asset even with a small fraction of it is harder to improve your performance against a competitor
your portfolio. These assumptions make sense when you are that is dealing with the exact same asset. In order to investigate
building a recommendation system for a hedge fund, that that we implemented two strategies that perform portfolio
manages several billions of dollars. However, that is almost optimization. The first, called HR, uses the historical returns
never the case when you are building an autonomous trader. as prediction for future returns and adjust portfolio weights
Furthermore, portfolio optimization algorithms often set very to maximize its Sharpe ratio using convex optimization. The
small weights for some assets. For these reasons, we propose second, called APM1, uses the previous strategies (MA,RSI
an algorithm that adjust an optimal theoretical portfolio to and RFOR) as autonomous analysts and ensembles their
a feasible portfolio considering the capital available and the signals with historical returns to define the expected returns.
number of units of asset traded in each transaction on an It is an example of utilization of the alternative multiagent
exchange (step). The algorithm returns the orders’ volume architecture, described in section II-E. When an autonomous
analyst signals it is a good time to buy a stock it is interpreted in the path to build autonomous traders that can beat the best
as higher expected return for that stock and a lower expected human experts in a consistent way.
return when it signals it is a sell moment. Therefore, each One aspect that we believe may be a fundamental ad-
autonomous analyst produces an expected return for each vantage for autonomous traders is accountability. Especially
asset. The ensemble is performed by simple average.The regarding eliciting possible conflict of interests. It is well
source code for the Trader with portfolio optimization is known that there are possible conflicts of interest among
available in appendix V-C. The HR and APM1 traders are analysts, managers and investors. One common conflict of
just version of the same trader, where HR does not include interest may happen among managers and stockholders [32]
the three autonomous traders (lines 20-28 in listing 13). (pg.12). The conflicts of interest among analysts and investors,
The results for HR and APM1 strategies are presented in ta- may take place when analysts have investments on target
ble IX. They used the same period and assets used previously, assets themselves or are contracted by securities emitters. In
but it is easy to realize that HR and APM1 performed better fact, SEC (U.S. Securities and Exchange Commission) has
than mono-asset strategies. HR and APM1 achieve higher a long history of examining potential conflicts of interests
annualized return and Sharpe ratio than any individual strat- among such roles, for more information see [47] and [48].
egy. This fact indicates that pursuing multi-asset autonomous Due to the fact that machine can have controlled or at least
strategies may lead to better results. It only confirms that what formally verifiable interests through software verification and
Markowitz showed years ago about portfolio optimization validation, possible conflict of interests can be avoided or at
and diversification is also valid for autonomous traders or very least controlled in a more efficient way. On the other
autonomous portfolio managers. Nevertheless, it seems that hand, the use of autonomous traders require that trust can
there are significantly more studies in autonomous traders be established in its development, deployment, and operation.
focused on mono-asset traders than multi-asset traders [11]. It is a challenge faced by AI in many scenarios and it is
We can also note that APM1 has performed better than HR. not different in autonomous traders. The concept of trust-
APM1 presented higher Sharpe ratio, higher return and smaller worthy artificial intelligence has five foundational principles
volatility. It indicates that the use of the alternative multiagent according to [49]: (1) beneficence, (2) non-maleficence, (3)
architecture (section 3) that uses specialized analysts to try to autonomy, (4) justice, and (5) explicability. One may argue
find better estimates for future returns, could be an improve- that accountability is strongly related to the first two principles.
ment in the efficience of information usage. The framework Autonomy refers in large extent on the promotion of human
mt5se helps the development of such autonomous analysts and oversight (e.g., Guidelines), others also consider the restriction
multi-asset traders. of AI-based systems’ autonomy, where humans retaining the
right to decide when to decide at any given time. The justice
Value - Strategy HR APM1
principle is not to be understood judicially, as in adhering
An. Return (%) 14.84 17.06
An. Sharpe Ratio (%) 18.79 21.67 to laws and regulations, but instead in an ethical way. For
Volatility (%) 0.69 0.63 instance, the utilization of AI should to amend past inequities
TABLE IX like discrimination of any kind. The last principle, explicability
R ESULTS FOR AUTONOMOUS P ORTFOLIO M ANAGEMENT USING is without any doubt, critical and challenging for autonomous
H ISTORICAL R ETURNS (HR) AND WITH AUTONOMOUS A NALYSTS traders, because given the environment complexity mistakes
(APM1)
will happen and the ability to explain and justify past decisions
is crucial, in order to build and keep trust in the system.
When reasoning about autonomous investments one may
We should also note that backtest period (Oct-1-2018 to
ask What would happen if autonomous investment analysts
Dec-31-2019) does not include severe crisis, as the one ob-
or managers become ubiquitous. We believe the scenario de-
served in 2020 or 2008. If that was the case, the performance
scribed by Fama in his Efficient Market Hypothesis (EMH) [7]
would be much worse. In fact, we believe there are many
would take place. The EMH states that financial markets
problems to be address in autonomous trading in order to build
are efficient in pricing assets. Asset prices would reflect all
traders that are able to perform as the best human experts,
information publicly available and the collective beliefs of all
for a deeper discussion, we suggest [17]. We address some
investors over the foreseeable future. Thus, it would not be
promising technologies for tackling some of these problems
possible to overcome the performance of the market, using
in the next section.
information that is known by the market, except by simple
chance.
VI. F UTURE OF AI IN F INANCE
We have discussed the financial environment complexity in
The financial environment is very challenging for au- section II. In fact, it is a challenging environment not just for
tonomous software, however, there are also some promis- autonomous agents, but also for human experts. Some argue
ing technologies and some subtle advantages in autonomous that it would be beyond the limits of algorithms. As stated
traders. In fact, many financial institutions are using au- by Marks, ”Valid approaches work some of the time but not
tonomous traders as part of the decision process and also all. And investing can’t be reduced to an algorithm and turned
trading real portfolios. Nevertheless, there is a long road ahead over to a computer. Even the best investors don’t get it right
every time.” [50]. In other words, Marks point to the fact task. The proposed framework mt5se may also contribute to
that the environment is not stationary and circumstances rarely development of new autonomous traders.
repeat exactly. Furthermore, Marks recognizes that psychology
plays a major role in financial markets. We believe that true R EFERENCES
autonomous trader should some how try to model and reason [1] Olympia Hadjiliadis Michael Carlisle and Ioannis Stamos. Trends and
about market psychological aspects. There are some initiatives trades. In Handbook of High-Frequency Trading and Modeling in
aiming to walk towards software and hardware that can mimic Finance, First Edition., New York, 2016. John Wiley Sons, Inc.
[2] Michael P. Wellman and Uday Rajan. Ethical issues for autonomous
processes that exist within the human brain, such as intuition trading agents. Minds and Machines, 27(4):609–624, Dec 2017.
and emotional memory concepts [51] and [52]. However, there [3] Andrew R. Young. Expert Advisor programming for Metatrader 5.
are more questions then answer about how to model and Edgehill Pushishing, Nashville, TN, USA, 2018.
[4] CORY MITCHELL. An introduction to contract for differences (cfds),
emotions in autonomous agents and certainly it is a topic that 2021.
requires further research. [5] Paulo André Lima de Castro. mt5b3: A framework for build-
Recent work point to some promising technologies, spe- ing autonomous trader for the brazilian stock exchange - b3.
https://github.com/paulo-al-castro/mt5b3, 2020. A Python Framework
cially regarding the use of convolutional neural networks for Autonomous Traders.
and Gated Recurrent Units (GRU) [53]; reinforcement learn- [6] Harry M. Markowitz. Portfolio selection. Journal of Finance, 7(1):77–
ing [42], [54], specially when used deep learning architec- 91, 1952.
tures [55], [56] and ensemble methods [39], [57], [58]. For [7] Eugene Fama. Efficient capital markets:a review of theory and empirical
work. Journal of Finance, 25:383–417, 1970.
a comprehensive review of recent work in autonomous ana- [8] Paulo André Lima de Castro. Is it a great autonomous fx trading strategy
lysts, we suggest [59]. There are significant research work in or you are just fooling yourself? arxiv.org/3558639, 2020.
autonomous agents focused on trading with cryptocurrencies, [9] Renato Oliveira and Adriano Pereira. Agente de negociação de ações
utilizando aprendizado por reforço. In Proceeding of the Workshop of
this paper [60] provides a good review about the theme. Artificial Intelligence Applied to Finance (WAIAF 2019), Sao Jose dos
Campos, Brazil, 2019. WAIAF.
VII. C ONCLUSIONS AND F UTURE W ORK [10] Alexander Sherstov and Peter Stone. Three automated stock-trading
agents: A comparative study. In Proceedings of the Agent Mediated
In this paper, we discussed some fundamental aspects of Electronic Commerce (AMEC) Workshop - AAMAS 2004, New York,
modelling autonomous traders, their complex environment un- 2004.
der the point of view of an autonomous agent. We proposed a [11] Paulo Andre Lima Castro and Jaime Simao Sichman. Automated asset
management based on partially cooperative agents for a world of risks.
multiagent architecture for autonomous traders that care about Applied Intelligence, 38:210–225, 2013.
diversification and investment policies (autonomous portfolio [12] J. Paulin, A. Calinescu, and M. Wooldridge. Agent-based modeling for
managers). We also discussed the main steps in the develop- complex financial systems. IEEE Intelligent Systems, 33(2):74–82, Mar
2018.
ments of systems under such architecture. Furthermore, we [13] Leandro Anghinoni and Liang Zhao. Time series trend detection and
presented a framework (mt5se) that helps the development forecasting using complex network topology analysis. In Proceeding
and testing of autonomous traders. This framework mt5se of the Workshop of Artificial Intelligence Applied to Finance (WAIAF
2018), Sao Jose dos Campos, Brazil, 2018. WAIAF.
is freely available [43] and it may also be used in real or [14] Harish Subramanian, Subramanian Ramamoorthy, Peter Stone, and
simulated operation in financial market accessible through Benjamin J. Kuipers. Designing safe, profitable automated stock trading
platform MetaTrader 5 [3]. We implemented some mono- agents using evolutionary algorithms. In GECCO ’06: Proceedings of
the 8th annual conference on Genetic and evolutionary computation,
asset traders and multi-asset traders using Technical indicators pages 1777–1784, New York, NY, USA, 2006. ACM Press.
and an AI based technique. We performed some backtest [15] Flavio Abdenur Elias Cavalcante-Filho and Rodrigo De-Losso. Machine
and the results indicate that multi-asset traders performed learning applied to accounting variables yields the risk-return metrics of
private company portfolios. In Proceeding of the Workshop of Artificial
better than mono-asset strategies, which fact indicates that Intelligence Applied to Finance (WAIAF 2019), Sao Jose dos Campos,
pursuing multi-asset autonomous strategies may lead to better Brazil, 2019. WAIAF.
results. It only confirms that what Markowitz showed years [16] Rasha AbdelKawy, Walid M. Abdelmoez, and Amin Shoukry. A
synchronous deep reinforcement learning model for automated multi-
ago about portfolio optimization and diversification, is also stock trading. Progress in Artificial Intelligence, 10(1):83–97, Mar 2021.
valid for autonomous traders. Nevertheless, it seems that there [17] Marcos Lopez de Prado. Advances in Financial Machine Learning.
are significantly more studies in autonomous traders focused Wiley, New York, 2018.
[18] Fan Fang, Carmine Ventre, Michail Basios, Hoiliong Kong, Leslie
on mono-asset traders than multi-asset traders [11]. We also Kanthan, Lingbo Li, David Martı́nez-Rego, and Fan Wu. Cryptocurrency
implemented and tested an autonomous trader based on the trading: A comprehensive survey. CoRR, abs/2003.11352, 2020.
proposed multiagent architecture that uses the mono-asset [19] Frank K. Reilly and Keith C. Brown. Investment Analysis & Portfolio
traders as autonomous analysts. This trader, called APM1, Management. South-Western, Cengage Learning, New York, 2012.
[20] David H. Bailey, Jonathan Borwein, Marcos Lopez de Prado, and
has performed better than all others. APM1 presented higher Qiji Jim Zhu. Pseudo-mathematics and financial charlatanism: The
Sharpe ratio, higher return and smaller volatility while trading effects of backtest overfitting on out-of-sample performance. Notices
the same assets. Finally, we discussed some open problems of the American Mathematical Society, 61:458–471, May 2014.
[21] I. Pollak. Covariance estimation and related problems in portfolio
in the area such as accountability and trustworthiness in optimization. In 2012 IEEE 7th Sensor Array and Multichannel Signal
autonomous systems and recognized there is still a long road Processing Workshop (SAM), pages 369–372, 2012.
ahead in the path to build autonomous traders that can beat [22] Michael O’Leary and Warren Valdmanis. An esg reckoning is coming.
Harvard Business Review, March 2021.
the best human experts consistently. We also pointed out some [23] Benjamin Graham. The Intelligent Investor. Harper Business, Revised
interesting technologies that may contribute to advance in such Edition, New York, 2006.
[24] David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone [51] James A. Crowder and Shelli Friess. Artificial psychology: The psy-
Stumpf, and Guang-Zhong Yang. Xai—explainable artificial intelli- chology of ai. SYSTEMICS, CYBERNETICS AND INFORMATICS,
gence. Science Robotics, 4(37), 2019. 11:64–68, May 2013.
[25] Alun Preece. Asking ‘why’ in ai: Explainability of intelligent sys- [52] Luis-Felipe Rodrı́guez and Félix Ramos. Development of computational
tems – perspectives and challenges. Intelligent Systems in Accounting, models of emotions for autonomous agents: A review. Cognitive
Finance and Management, 25(2):63–72, 2018. Computation, 6:351–375, 09 2014.
[26] Cynthia Rudin. Stop explaining black box machine learning models [53] ; Carmine Ventre Zheng Gon and John O’Hara. Classifying high-
for high stakes decisions and use interpretable models instead. Nature frequency fx rate movements with technical indicators and inception
Machine Intelligence, 1(5):206–215, May 2019. model. In Proceedings of the ACM International Conference on AI in
[27] Shihao Gu, Bryan Kelly, and Dacheng Xiu. Empirical asset pricing Finance 2020, New York, 2020.
via machine learning. Technical report, National bureau of economic [54] Renato Oliveira and Adriano Pereira. A tabular sarsa-based stock market
research, 2018. agent. In Proceeding of the ACM International Conference on AI in
[28] Paulo Andre Lima de Castro and Marcel Ribeiro. Online learning Finance - 2020, New York, 2020. ICAIF 2020.
applied to autonomous valuation of financial assets. In Proceeding of [55] Hongda Shen and Eren Kursun. Deep q-network based adaptive alert
the Web Intelligence Conference 2018, Santiago, Chile, December 2018. threshold selection policy for payment fraud systems in retail banking.
WI 2018. In Proceeding of the ACM International Conference on AI in Finance -
2020, New York, 2020. ICAIF 2020.
[29] William F. Sharpe. The sharpe ratio. Journal of Portfolio Management,
[56] L. Conegundes and A. C. M. Pereira. Beating the stock market with a
13(3):227–286, 1994.
deep reinforcement learning day trading system. In 2020 International
[30] Tom Rollinger and Scott Hoffman. Sortino ratio: A better measure of
Joint Conference on Neural Networks (IJCNN), pages 1–8, 2020.
risk. Futures magazine, 1(1):40–42, 2 2013.
[57] Bashar Alhnaity and Maysam Abbod. A new hybrid financial time series
[31] F. A.Sortino and H. J.Forsey. On the use and misuse of downside risk. prediction model. Engineering Applications of Artificial Intelligence,
Journal of Portfolio Management, 22(1):35–42, 1996. 95:103873, 2020.
[32] Aswath Damodaran. Applied Corporate Finance. Wiley, New York, NY, [58] Hongyang Yang; Xiao-Yang Liu; Shan Zhong and Anwar Walid. Deep
2010. ensemble reinforcement learning for automated stock trading. In
[33] David H. Bailey and Lopez de Prado. The sharpe ratio efficient frontier. Proceeding of the ACM International Conference on AI in Finance -
Journal of Risk, 15:3–44, Feb 2012. 2020, New York, 2020. ICAIF 2020.
[34] MetaQuotes Ltd. Metatrader 4 - forex platform, 2020. [59] Watthanasak Jeamwatthanachai Mehtabhorn Obthong, Nongnuch Tan-
[35] Paulo Andre Castro and Jaime S. Sichman. Agex: A financial market tisantiwong and Gary Wills. A survey on machine learning for stock
simulation tool for software agents. In Will Aalst, John Mylopoulos, price prediction: algorithms and techniques. In Proceeding of the 2nd
Norman M. Sadeh, Michael J. Shaw, Clemens Szyperski, Joaquim Filipe, International Conference on Finance, Economics, Management and IT
and Jos? Cordeiro, editors, LNBIP, volume 24, pages 704–715. Springer, Business, pages 63–71, Prague, Czech Republic, 2020. Vienna House
Berlin, 2009. Diplomat.
[36] Traview brokerage company, 2021. [60] Ahmed M. Khedr, Ifra Arif, Pravija Raj P V, Magdi El-Bannany,
[37] Xp investimentos, 2021. Saadat M. Alhashmi, and Meenu Sreedharan. Cryptocurrency price
[38] Clear corretora, 2021. prediction using traditional statistical and machine-learning techniques:
[39] Rafael Silva Wagner and André Alves Portela Dos Santos. Forecasting A survey. Intelligent Systems in Accounting, Finance and Management,
the direction of high-frequency returns: An ensemble-trees application. 28(1):3–34, 2021.
In Proceeding of the Workshop of Artificial Intelligence Applied to
Finance (WAIAF 2018), Sao Jose dos Campos, Brazil, 2018. WAIAF.
[40] A. Tan, C. Quek, and K. Yow. Maximizing winning trades using a novel
rspop fuzzy neural network intelligent stock trading system. Applied
Intelligence, 29:116–128, 2008. 10.1007/s10489-007-0055-1.
[41] Gordon Ritter and Petter Kolm. Multiperiod portfolio selection and
bayesian dynamic models. SSRN Electronic Journal, 09 2014.
[42] Yi Feng Yuriy Nevmyvaka and Michael Kearns. Reinforcement learning
for optimized trade execution. In Proceedings of the 23rd International
Conference on Machine Learning- ICML 2006, Pittsburgh, Pennsylva-
nia, 6 2006.
[43] Paulo André Lima de Castro. mt5se: A framework for building
autonomous trader for the stock exchanges. https://github.com/paulo-
al-castro/mt5se, 2021. A Python Framework for Autonomous Traders.
[44] Mark A. Hall Eibe Frank and Ian H. Witten. Data Mining: Practical
Machine Learning Tools and Techniques. Morgan Kaufmann, New York,
2016.
[45] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent
Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Pret-
tenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre
Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and
Edouard Duchesnay. Scikit-learn: Machine learning in python. CoRR,
abs/1201.0490, 2012.
[46] Pyportfolioopt library. https://pyportfolioopt.readthedocs.io/en/latest/,
2020. A Python Library that implements portfolio optimization methods.
[47] U.S. Securities and Exchange Commission. Analysts conflicts of
interest: Taking steps to remove bias, 2016.
[48] Ingo Walter. Conflicts of interest and market discipline among financial
services firms. In Proc. of the Federal Reserve of Chicago conference on
Market Discipline: Evidence Across Countries and Industries, Chicago,
USA, 11 2003. Fed-Chicago.
[49] Sebastian Lins Scott Thiebes and Ali Sunyaev. Trustworthy artificial
intelligence. Electronic Markets, 10 2020.
[50] Howard Marks. The most important thing: uncommon sense for the
thoughtful investor. Columbia Press, New York, 2011.