The Predictive Edge: Outsmart the Market using Generative AI and ChatGPT in Financial Forecasting

ALEJANDRO LOPEZ-LIRA

THE PREDICTIVE EDGE

OUTSMART THE MARKET

USING GENERATIVE AI AND CHATGPT IN

FINANCIAL FORECASTING

Copyright © 2024 by John Wiley & Sons. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750‐8400, fax (978) 750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748‐6011, fax (201) 748‐6008, or online at http://www.wiley.com/go/permission.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762‐2974, outside the United States at (317) 572‐3993 or fax (317) 572‐4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our website at www.wiley.com.

Library of Congress Cataloging‐in‐Publication Data is Available:

ISBN 9781394242719 (Cloth)

ISBN 9781394242726 (ePub)

ISBN 9781394242733 (ePDF)

Cover Design: Wiley

Cover Image: © iamchamp/Adobe Stock

Author Photo: Courtesy of the Author

To my wonderful wife Emma and my family.

Preface

What if artificial intelligence could accurately predict which stocks are about to increase in value? As an investor, you could know which companies to invest in before prices take off. This book explains how to use the most advanced artificial intelligence—ChatGPT—to invest better. It provides a step‐by‐step tutorial to forecast stock price movements and design and implement investment strategies using ChatGPT.

ChatGPT is the most sophisticated technology I have encountered. I was surprised by the immense capacity and sometimes cleverness that ChatGPT displays and how it vastly increased my productivity. For example, it made it trivial to proofread text, write code, brainstorm, and prototype sophisticated mathematical models. It has near (and sometimes better than) human‐level skills in multiple domains, including writing, coding, and image generation. Could it also be used for investment prediction?

Stock market prediction methods have always intrigued me. I have researched (and occasionally deployed) them during my Ph.D. at Wharton and now at the University of Florida. ChatGPT's potential captivated me, and I was eager to solve related research questions. The first question was, can ChatGPT forecast stock price movements?

This book is based on an academic paper I wrote with Yuehua Tang in April 2023 to answer this question. We wanted to know if ChatGPT could accurately predict stock price movements using news headlines. The results were startling: ChatGPT was able to forecast positive returns with unparalleled precision. We simulated the returns of a strategy that followed ChatGPT's predictions on news data and found it would have produced more than 400 percent in just 18 months—compared to average annual stock market returns of around 10 percent.

The paper immediately grabbed media attention. I was interviewed live by CNN and other media outlets. Hundreds of online articles were written about it, and the research was downloaded more than 60,000 times (by far my most downloaded).

Research articles are dense and challenging to read because we want to be rigorous and consider all possibilities. We use obscure terminology that is familiar to academics but is hard to understand for everyone else not involved in the research. Yet, I receive constant emails about the research methodology and specifics of the article. There is broad interest in how ChatGPT can transform finance, but there are better ways to present the information than academic writing.

I drafted this book to make the research accessible and to reach a wider audience. The Predictive Edge is designed to contain practical but comprehensive information on using these new powerful technologies best. It is meant for people with little background in artificial intelligence or finance. You are not expected to be an expert, and I did my best to make my book as self‐contained as possible, although you may want to search online or ask ChatGPT to clarify and exemplify some concepts.

Introduction

What if artificial intelligence could predict the stock market? Artificial intelligence (AI) is transforming numerous industries, and finance is no exception. Recent advances have led to sophisticated chatbots like ChatGPT with remarkable skills. The potential is enormous—if AI could predict price movements, investors would have an incredible advantage. However, while AI shows promise for finance, reliably forecasting complex financial markets is an immense challenge. The Predictive Edge examines the intriguing academic research documenting how ChatGPT can forecast stock price movements. It explains how to translate this theoretical knowledge into practical investment strategies, exploring how ChatGPT can be leveraged to predict stock prices accurately.

This book is grounded in academic research investigating a method using ChatGPT to predict whether news headlines indicate positive or negative returns for a stock. It presents the method and results in detail and provides a step‐by‐step guide to implementing the strategy, discussing practical considerations and refinements. We will follow the research and explore the promises and limitations of using natural language processing for finance.

The study's results are surprising. The strategy delivered more than 400% simulated returns in less than two years by going long on stocks with positive headlines and shorting negative headlines. In contrast, the stock market average is around 10% annually. While the research documents how ChatGPT captures valuable signals amid the endless news stream, The Predictive Edge aims to make these insights accessible to a broad audience with little background in AI.

Understanding these methods is increasingly relevant as the capability to quickly analyze vast volumes of text data represents a significant shift in stock trading. Traditional financial estimations often rely on charts, numbers, and fundamental analyses. These tools have served us well, but they are now being supplemented—and sometimes replaced—by algorithms and artificial intelligence.

Yet, the potential of AI in finance goes beyond simple speed. AI models can scan massive datasets for patterns that humans cannot see. They can also analyze many information sources simultaneously, from news articles and social media sentiment to economic indicators and earnings reports. This book thoroughly explains how to combine these technologies to produce valuable investment insights. It also explains the emerging landscape of finance and AI, providing valuable insights and strategies for investors, finance professionals, technologists, and anyone else interested in the convergence of these two fields.

What You Will Learn

The Predictive Edge will teach you how to exploit large language models' remarkable ability to process and analyze text‐based data, such as news headlines, to predict potential effects on stock prices. We will learn about AI's influence on stock market trading and financial forecasting, emphasizing how AI and machine learning can be applied to make better investment decisions.

We will cover not only the theoretical concepts but also practical advice on how to incorporate advanced AI models into quantitative trading strategies. Whether you are an experienced investor, finance professional, business leader, or simply eager to apply AI in investment decisions, the material is relevant across multiple backgrounds. You will understand how to use these emerging technologies in investing or advancing your career.

Moreover, we will explore the advantages of AI‐powered stock market forecasting and investment decision‐making and address critical challenges in the modern investment landscape. We will examine how traditional stock market analysis methods can often overlook subtle market sentiments, leading to inaccuracies and potential financial losses.

We will study methods to overcome current limitations by building strong fundamentals of AI in financial forecasting and following practical guidance on integrating AI tools into investment decisions. By adequately leveraging AI's predictive strengths, you can work on achieving higher returns while constructing sturdier, more adaptive portfolios.

These technologies are evolving rapidly and understanding their current state is insufficient. Therefore, in addition to learning extensively about the existing tools, you will acquire a general framework to understand future advances in AI. The chapters will assist you in gaining a critical, forward‐thinking perspective, enabling you to recognize the latest trends and developments.

Working through step‐by‐step frameworks and learning about the fundamental concepts will be valuable, especially if you are just acquiring a technical background but recognize the significance of AI in shaping the future of investing. The material will enable you to deploy AI for your financial and investment objectives.

After reading this book, you will:

Understand core concepts of stock markets, AI, and natural language processing, and how they intersect

Comprehend ChatGPT capabilities and limitations in depth, including ethical considerations

Grasp the methodology and startling results of the ChatGPT forecasting study

Have clear guidance for deploying similar trading strategies in the real world

Appreciate versatile applications of language AI across financial workflows

Recognize future opportunities and persistent challenges as AI transforms finance

Feel equipped to apply these emerging technologies in your business or investments

Be inspired by AI's potential while critically assessing its limitations

Know the best practices to implement AI analytics responsibly and avoid pitfalls

Remain updated on the state of the art in this rapidly evolving field

Applying AI in finance can be challenging, and a structured approach is critical to understanding AI's full potential. The Predictive Edge is your guide, providing a well‐organized overview of AI in finance, from its fundamentals to advanced applications. Each chapter explains AI's workings, applications, and transformative potential in financial forecasting. If you follow the material, you will be prepared to understand the AI revolution in finance and gain the practical skills to succeed in this quickly developing field.

How This Book Is Structured

The Predictive Edge has three parts. Part I covers the fundamentals of finance and AI. Part II explores ChatGPT and its incorporation into investment strategies. Part III provides further applications and speculates about the future of AI and finance. Expert readers may want to skip Part I and return for reference, as necessary. We now briefly explore each chapter's contents.

Part I: Fundamentals

These first four chapters provide the essential baseline knowledge on stock markets, artificial intelligence concepts, natural language models, and ChatGPT required to comprehend the financial forecasting application covered later. By grounding core ideas from trading approaches to AI innovations, Part I aims to make the specifics of leveraging ChatGPT for stock predictions accessible even if you do not have a technical background in these areas.

CHAPTER 1: UNDERSTANDING THE STOCK MARKET

This chapter provides the fundamental background on stock markets, setting the foundation to understand how AI and ChatGPT could be applied for financial forecasting.

The chapter begins by covering market basics—economic functions, diverse types of markets such as derivatives, and significant investment instruments. We then explore key concepts around trading stocks—what they are, major participants, and standard analysis methods like fundamental, technical, and news/sentiment‐driven approaches. We will also learn about portfolio management fundamentals and major investing strategies and how portfolio construction and risk management are instrumental in achieving investing success.

After covering the building blocks, the chapter explores current applications of AI in finance and stock prediction, providing context on the potential benefits and limitations of algorithmic forecasting. It concludes with a preview of core concepts covered in the next chapter and includes a check‐your‐understanding section and a recap to emphasize critical takeaways. These sections will also be present in all the following chapters.

Chapter 1 provides the necessary foundation for employing AI in forecasting by introducing key concepts—stocks, trading approaches, portfolios, and current applications of AI. The chapter contains the knowledge needed to help you understand the rest of the book, even if you are just getting familiar with the basics of investment and AI. The goal is to prepare you so the methods and results around forecasting are intuitive and meaningful when presented later.

CHAPTER 2: UNDERSTANDING ARTIFICIAL INTELLIGENCE

This chapter provides a high‐level explanation of AI to comprehend core concepts behind technologies like ChatGPT. It starts by defining intelligence and discussing diverse aspects of human cognition that inspired the development of thinking machines.

We will then explore the history of efforts to create intelligent computers. We will study central innovations like machine learning and deep learning that enabled recent AI breakthroughs in accessible language, focusing on high‐level intuitions rather than mathematical details.

Specifically, we will contrast the symbolic logic‐based approach that initially dominated AI research against the data‐driven machine learning paradigm that has recently gained prominence. We will also learn about the massive datasets and computational power needed to train deep neural networks behind state‐of‐the‐art AI systems.

The chapter concludes by previewing how large language models represent the latest advancement in natural language processing, setting the stage for understanding ChatGPT's abilities for financial forecasting. The goal is to provide you with enough background without getting overly technical so ChatGPT's financial forecasting approach is intuitive when covered later.

CHAPTER 3: LARGE LANGUAGE MODELS: A GAME CHANGER

This chapter dives deeper into the AI concepts directly relevant to comprehending ChatGPT's abilities. We begin by exploring natural language processing (NLP)—the subdomain of AI focused on understanding, generating, and interacting with human languages. It also covers Core NLP tasks, algorithms, and applications.

We then explore generative AI, the technology behind content‐creating systems like DALL‐E for images and ChatGPT for text. We cover key capabilities like capturing patterns from vast datasets and using that learning to produce novel, high‐quality outputs. Next, we will learn about the game‐changing innovation of large language models (LLMs). LLMs like GPT‐3 and Claude are foundationally transforming NLP by leveraging immense datasets and model sizes. We will cover their multivariate benefits over previous NLP systems and current limitations.

The goal is to provide an intuitive explanation of how advanced natural language systems work in order to understand the financial forecasting approach presented afterward.

CHAPTER 4: ADVANCED TOPICS IN LLMs

While the previous chapter overviews large language models, this chapter explores the fundamental architectures and mechanisms powering them. It starts by explaining technical elements like the self‐attention layers and Transformer models that enabled exponential progress in language AI. We cover the intuitions behind these innovations without requiring a mathematical background. We then learn about the processes to improve LLMs, including Transfer learning to train on vast datasets and fine‐tune them for specialized tasks. These techniques enabled models like ChatGPT to reach new performance heights. Finally, the chapter summarizes the landscape of significant LLMs, highlighting how capabilities proliferate.

Part II: ChatGPT and Stock Prediction

With foundational knowledge established in Part I, we now transition to Part II's comprehensive coverage of the chatbot stock prediction methodology, results, and practical implementation guidance. Chapters 5 through 8 contain the core content, spanning ChatGPT capabilities to groundbreaking findings and a step‐by‐step guide for deploying the strategies.

CHAPTER 5: WHAT IS ChatGPT?

This chapter comprehensively covers ChatGPT's capabilities, limitations, and best practices to apply the model effectively. It begins by discussing ChatGPT's diverse skills, like conversing naturally, following instructions, running code, browsing websites, understanding language, adapting to context, displaying world knowledge, generating creative content, adjusting in real time, and more. We also learn about safety and ethical considerations.

The chapter then outlines the current weaknesses of ChatGPT and similar LLMs such as lacking deeper understanding, short‐term memory, sensitivity to input phrasing, potential hallucinations, and more. It also addresses ethical pitfalls and provides guidance on prompt engineering, covering best practices for structuring effective prompts to yield optimal chatbot performance. Templates and examples are included to help you craft high‐quality prompts. Finally, the chapter covers business‐use cases and ethical considerations around responsible deployment to encourage users to apply ChatGPT safely, accountably, and for social good.

The goal is to provide a comprehensive yet accessible guide so that users can maximize value from ChatGPT while proactively addressing risks and limitations.

CHAPTER 6: CAN ChatGPT FORECAST STOCK PRICE MOVEMENTS?

This chapter presents the core research on leveraging ChatGPT for stock prediction. It explains the empirical study methodology and results in detail. This chapter presents the book's academic foundation: the groundbreaking empirical evidence demonstrating ChatGPT's ability to forecast prices. The results' interpretation, evaluation, and discussion provide a research‐backed perspective on the transformational potential of leveraging language AI in financial analysis while acknowledging current limitations and ethical considerations.

The methodology section covers the data collection, including integrating daily stock returns, news headlines, and relevance scores to filter noise, and carefully addressing look‐ahead bias. It then presents the natural language prompt posed to ChatGPT to forecast returns. Next, the startling results are examined—more than 400% simulated profits in less than a year and a half. The chapter analyzes the findings and discusses implications for financial analysts, active trading strategies, the labor market, retail investors, regulations, and the long‐term outlook for AI in finance.

CHAPTER 7: IMPLEMENTING A ChatGPT TRADING STRATEGY: A STEP‐BY‐STEP GUIDE

This chapter provides a thorough, step‐by‐step guide to implementing the ChatGPT‐based stock forecasting approach for investment returns. It begins by presenting the overall market‐neutral strategy framework. The chapter then addresses preliminary steps, such as setting goals, resource requirements, stock selection criteria, and risk management considerations.

The key processes of developing, backtesting, implementing, and monitoring the strategy are covered next in detail. The chapter discusses methods for position sizing, benchmarking, leveraging tools for automation, evaluating performance, and dynamically adjusting the model over time. It also explores compliance considerations.

The intention is to provide you with all the information you need to implement the ChatGPT trading methods. While results may vary in practice, this blueprint is designed to help investors succeed by providing the best practices refined through multiple rounds of research. The chapter serves as a bridge between abstract academic findings and practical application.

CHAPTER 8: ChatGPT IN ACTION: PRACTICAL APPLICATIONS

While the previous chapter focused on ChatGPT's stock forecasting strategy, this chapter explores integrating ChatGPT into broader financial contexts. It provides examples of combining natural language predictions with quantitative models for enhanced performance. Risk management use cases are then presented, leveraging ChatGPT to detect market regime changes and mitigate losses.

The chapter also discusses integration with automated trading systems for seamless implementation. Overall, it aims to spark ideas on versatile applications of ChatGPT in finance beyond stock prediction, including enhancing processes from data analysis to trade execution. The goal is to demonstrate the flexibility of AI to drive value across financial workflows—from research to risk management, reporting, and more.

Part III: Envisioning a Financial Future with AI

In the concluding section, the focus shifts from the details of forecasting methods to a broader perspective. We explore the future development of AI in finance, including ongoing progress but also persistent difficulties related to trust and transparency. Part III concludes with final recommendations for innovating as algorithms transform investing.

CHAPTER 9: THE FUTURE OF AI IN FINANCIAL FORECASTING

This chapter explores the outlook and opens questions about the potentially transformational applications of AI in finance. It discusses continued progress in algorithms and models, with innovations in generative AI, reinforcement learning, Transfer learning, multimodal architectures, and more that are applicable to financial analysis. It also examines potential synergies with emerging technologies like blockchain and the internet of things.

The chapter then covers crucial areas