Algorithmic trading and Machine Learning by Michael Kearns, Professor of Computer and Information Science, UPenn
21 likes7,647 views
The document discusses challenges and methodologies in algorithmic trading using machine learning, specifically focusing on optimized execution, smart order routing in dark pools, and trading with inventory constraints. It highlights the importance of learning to act rather than just predict outcomes, incorporating risk management, and adapting to new market mechanisms. Additionally, it presents experimental frameworks and theoretical insights aimed at improving trading efficiency and decision-making in complex, automated environments.
![Learning to Act: ML for Optimized Execution
[Y.
Nevmyvaka.
Y.
Feng,
MK;
ICML
2006]
[MK,
Y.
Nevmyvaka;
In
“High
Frequency
Trading”,
O’Hara
et
al.
eds,
Risk
Books
2013]](https://image.slidesharecdn.com/algorithmictradingandmachinelearningbymichaelkearns-150317132106-conversion-gate01/85/Algorithmic-trading-and-Machine-Learning-by-Michael-Kearns-Professor-of-Computer-and-Information-Science-UPenn-3-320.jpg)
![Smart Order Routing in Dark Pools
[K.
Ganchev,
MK,
Y.
Nevmyvaka.
J.
Wortman
Vaughan;
UAI
2009,
CACM
2010]
[K.
Amin,
MK,
P.
Key,
A.
Schwaighofer;
UAI
2012]](https://image.slidesharecdn.com/algorithmictradingandmachinelearningbymichaelkearns-150317132106-conversion-gate01/85/Algorithmic-trading-and-Machine-Learning-by-Michael-Kearns-Professor-of-Computer-and-Information-Science-UPenn-7-320.jpg)
![A Distributional Model of Liquidity
• Assume each dark pool has a stationary distribution P over available shares
• If we submit v shares, min(v,s) will be executed where s ~ P
• Our observations are censored by our own actions
• MLE for P is Kaplan-Meier --- but we must address exploration across pools
• Want to learn just enough about each pool to do optimal SOR
s
P[s]
Pool
A
s
P[s]
Pool
B
s
P[s]
Pool
C
s
P[s]
Pool
D](https://image.slidesharecdn.com/algorithmictradingandmachinelearningbymichaelkearns-150317132106-conversion-gate01/85/Algorithmic-trading-and-Machine-Learning-by-Michael-Kearns-Professor-of-Computer-and-Information-Science-UPenn-10-320.jpg)
![Incorporating Risk:
Algorithmic Trading with Inventory Constraints
[E.
Even-‐Dar,
MK,
J.
Wortman
Vaughan;
ALT
2006]
[L.
Dworkin,
MK,
Y.
Nevmyvaka;
ICML
2014]](https://image.slidesharecdn.com/algorithmictradingandmachinelearningbymichaelkearns-150317132106-conversion-gate01/85/Algorithmic-trading-and-Machine-Learning-by-Michael-Kearns-Professor-of-Computer-and-Information-Science-UPenn-13-320.jpg)
Algorithmic trading and Machine Learning by Michael Kearns, Professor of Computer and Information Science, UPenn
- 1. Algorithmic Trading and Machine Learning Michael Kearns University of Pennsylvania QuantCon 2015, NYC Special thanks: Yuriy Nevmyvaka (Lehman, BofA, SAC, Engineers Gate) Long Version: h-p://techtalks.tv/talks/algorithmic-‐trading-‐and-‐machine-‐learning/60847/ or Google “techtalks ICML 2014 kearns”
- 2. ML for Trading: Challenges • Learning to Act (vs. Predict): Optimized Execution • Dealing with Censored Data: Order Routing in Dark Pools • Incorporating Risk: Trading Under Inventory Constraints
- 3. Learning to Act: ML for Optimized Execution [Y. Nevmyvaka. Y. Feng, MK; ICML 2006] [MK, Y. Nevmyvaka; In “High Frequency Trading”, O’Hara et al. eds, Risk Books 2013]
- 4. A Canonical Trading Problem • Goal: Sell V shares in T time steps; maximize revenue • Benchmarks: – Volume Weighted Average Price (VWAP) – Time Weighted Average Price (TWAP) – Implementation Shortfall (midpoint of bid-ask spread at beginning) • View as a problem of state-based control (Reinforcement Learning) – Action space: limit orders – State variables: inventory and time remaining – Additional features capturing order book activity • Experimental framework – Full historical order book reconstruction and simulation – Learn optimal policy on 1 year training; test on following 6 months – Pitfalls: directional drift, “counterfactual” market impact
- 5. Bid Volume -0.06% Ask Volume -0.28% Bid-Ask Volume Misbalance 0.13% Bid-Ask Spread 7.97% Price Level 0.26% Immediate Market Order Cost 4.26% Signed Transaction Volume 2.81% Price Volatility -0.55% Spread Volatility 1.89% Signed Incoming Volume 0.59% Spread + Immediate Cost 8.69% Spread+ImmCost+Signed Vol 12.85% AddiZonal Improvement From Order Book Features T=4 I=1 27.16% T=8 I=1 31.15% T=4 I=4 30.99% T=8 I=4 34.90% T=4 I=8 31.59% T=8 I=8 35.50% Improvement Over OpZmized Submit-‐and-‐Leave !
- 6. Desperately Seeking Alpha • A natural modification: – Change action space to buy or sell and hold for t seconds, then liquidate (+null action) – Add state features capturing directional movements • Now trying to predict movement and profit (vs. fixed optimization problem) • Definite (aggregate) predictability, but hard to overcome trading costs • Still learn broadly consistent policies across stocks: – Null action vast majority of time; trade only in extremal states/opportunities – Short holding (milliseconds): Momentum – Longer holding (seconds): Reversion
- 7. Smart Order Routing in Dark Pools [K. Ganchev, MK, Y. Nevmyvaka. J. Wortman Vaughan; UAI 2009, CACM 2010] [K. Amin, MK, P. Key, A. Schwaighofer; UAI 2012]
- 8. Dark Pools • Recently introduced trading mechanism • Intended to allow large counterparties to trade with minimal market impact • Only specify desired volume and direction (buy/sell); no price specified • Buyers and sellers matched in order of arrival • Prices will be midpoint of National Best Bid and Offer (NBBO) in lit market • Now dozens of dark pool, competing for liquidity instead of price • Break trade up over exchanges instead of over time
- 9. Buy V shares total v2 shares How should we disperse V? Dark Pool A Dark Pool B Dark Pool C Dark Pool D ? ? ? ? Smart Order Routing (SOR)
- 10. A Distributional Model of Liquidity • Assume each dark pool has a stationary distribution P over available shares • If we submit v shares, min(v,s) will be executed where s ~ P • Our observations are censored by our own actions • MLE for P is Kaplan-Meier --- but we must address exploration across pools • Want to learn just enough about each pool to do optimal SOR s P[s] Pool A s P[s] Pool B s P[s] Pool C s P[s] Pool D
- 11. A Simple and Efficient Algorithm • Provably converges quickly to optimal allocations under known distributions • Involves optimistic modification to MLE, new convergence bound • Analysis reminiscent of E3/RMAX in RL greedy allocaZon under current distribuZonal esZmates re-‐esZmate using censored observaZons
- 12. Empirical Evaluation • Data: submission/execution data from multiple pools at large brokerage • Used to build distribution models (heavy-tailed) and simulator • Comparison to uniform allocation (strawman), bandit approach, optimal
- 13. Incorporating Risk: Algorithmic Trading with Inventory Constraints [E. Even-‐Dar, MK, J. Wortman Vaughan; ALT 2006] [L. Dworkin, MK, Y. Nevmyvaka; ICML 2014]
- 14. No-Regret Learning in Finance • Originates with Cover’s Universal Portfolios; simple reweighting algorithm • Strong theoretical guarantees without stochastic assumptions – Compete with best single stock in hindsight • Can be applied directly to stocks or higher-level trading strategies • Unfortunately methods work poorly in practice: S&P500, 2005-‐2011
- 15. Trading with Inventory Constraints • Can’t manage to Sharpe Ratio, but can limit allowed positions/portfolios • Restrict to portfolios with daily standard deviation PNL at most $X historically • Leads to elliptical constraint in portfolio space depending on correlations • Only compete with strategies: – Obeying inventory constraints – Making only local moves (limit market impact) • Combine no-regret with pursuit-evasion to recover theoretical guarantees correlaZon = 0 correlaZon = 0.5 correlaZon = 1
- 16. Hedged Pursuit-‐Evasion DirecZonal
- 17. Conclusions • In the middle (beginning?) of a period of rapid change in markets: – Automation of traditional processes and trading – Introduction of new market mechanisms (open order books, dark pools) – Development of new types of trading and strategies (HFT) • Automation + Data ! Machine Learning • Challenges: – Feature design – Censored observations – Risk considerations – Strategic/adversarial behavior • More, and different, to come… Contact: [email protected] Long Version: h-p://techtalks.tv/talks/algorithmic-‐trading-‐and-‐machine-‐learning/60847/ or Google “techtalks ICML 2014 kearns”