Autonomous Trading System using Reinforcement Learning by Melissa Tan

The  idea  of  predicting  financial  instruments  has  been  the  goal  of  many  due  in  part  to  the expectation  that  predicting  these  instruments  can  prove  lucrative.  Whilst  the  accurate rediction of price seemed reasonable, they do not necessarily guarantee positive returns due to commissions, large profit draw-downs and excessive switching behaviours. Reinforcement Learning (RL) is an  autonomous approach to decision making process through repetitive self-learning and evaluation. The idea is to train an agent to learn to execute an order by acting on a suitable strategy that maximizes profit. In this capstone project, we first conduct a systematic review of 50 literature that applies RL in trading, in particular, to uncover the common theme to  maximizing  the  chance  of  a  successful  model. We then prototyped  a  trading  system that applies  Proximal  Policy  Optimization  (PPO)  which  is  the  brainchild  of  Schulman  et  al. (Schulman, Wolski, Dhariwal, Radford, & Klimov, 2017). Thismodel achieved an annualised return of 34.06%and outperformed thestudies by Xiong et al. (Xiong, Liu, Zhong, Yang, & Walid, 2018)whose DDPG model produced an annualised return of 25.87%. We also found that adding technical indicators altered the agent’s trading activities significantly.With  the added information, the model achieved a lower annualised return of 27.47% but the result was more  consistent  to  the  training  performance.  In  summary,  we conclude that  RL  can  be successfully applied to trading, however the models are highly dependent on the characteristics of  the  underlyingdata,  training  regimeand  the  RL model  itself,  thus a  rigorous  hyperparameters tuning is required to achieve good result.

• #education
• #reinforcement_learning
• #autonomous_trading_system
• #forex
• #stock