SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.

1051/shsconf/202418102015
ICDEBA 2023

Bitcoin price prediction based on fear & greed index

Qichuan Huang*
Computer Science, CSE Department, The Chinese University of Hong Kong, 999077, Hong kong, China

Abstract. This study investigates the Fear & Greed Index, an indicator designed to reflect market sentiment
regarding Bitcoin price, intending to utilize it as a predictive parameter for future price fluctuations. Due to
the substantial volatility in Bitcoin prices and its significant influence on prediction outcomes, the dataset
was preprocessed through monthly filtering and normalization. To forecast Bitcoin prices, an array of
machine learning algorithms, including linear regression, random forest, and XGBoost, as well as their
enhanced counterparts, were employed. The optimal model was identified by comparing the Grid Search
XGBoost analysis results. This research holds implications for accurately predicting Bitcoin prices and
underscores the impact of market sentiment on its valuation.

1 Introduction refers to the collective attitude of investors and traders

towards a specific asset or market, which various factors,
The cryptocurrency market has witnessed substantial including economic news, social media discourse, and
growth and volatility in recent years, with Bitcoin individual investor behavior, can shape.
emerging as the most prominent and widely adopted Numerous studies have investigated the relationship
digital currency. The erratic nature of Bitcoin's price has between market sentiment and Bitcoin's price. For
sparked heightened interest in comprehending and instance, a study by Nadarajah and Chu discovered a
forecasting its price movements. robust positive correlation between sentiment and price
The cryptocurrency market has witnessed substantial clustering within the Bitcoin market [12]. Another study
growth and volatility in recent years, with Bitcoin by Dimpfl and Kleiman employed transaction-level data
emerging as the most prominent and widely adopted from Coinbase to derive a measure of investor sentiment,
digital currency. The erratic nature of Bitcoin's price has uncovering a significant and robust relationship between
sparked heightened interest in comprehending and escalating sentiment and price increases and vice versa
forecasting its price movements. [13].
Numerous studies have approached Bitcoin price Chiah et al. investigated the influence of investor
prediction from various perspectives. For instance, some sentiment on Bitcoin returns and conditional volatilities
research has employed blockchain data and machine during the COVID-19 era, revealing that investor
learning techniques to develop models considering sentiment positively impacted Bitcoin returns and their
factors such as Bitcoin properties, network, transactions, volatility, particularly after the COVID-19 outbreak
market, attention, and gold spot price [1]. Other studies [14]. Moreover, BeinCrypto and the International
have analyzed the sources and influences of Bitcoin Monetary Fund offer overviews of the primary factors
price volatility, encompassing market liquidity, trading that affect Bitcoin's price, encompassing supply and
volume, market sentiment, policy events, and demand, news and public opinion, emotions, innovation
cyberattacks [2-5]. Furthermore, investigations have and competition, regulation and security, institutional
explored the relationship between Bitcoin prices and adoption, and market manipulation.
financial crises and the potential of Bitcoin as an early This study’s objective is to expand upon the existing
warning indicator or hedge against financial crises [6,7]. literature by examining the relationship between market
Additional research has examined the bidirectional sentiment and Bitcoin's price, utilizing the Fear & Greed
causality between Bitcoin price and network effects, Index as a proxy for market sentiment. The study will
alongside the impact of these network effects on Bitcoin assess the predictive capability of this index in
price volatility [8,9]. Lastly, some studies have forecasting Bitcoin's price fluctuations and investigate
scrutinized the long-run equilibrium relationship potential enhancements to the predictive accuracy of the
between Bitcoin price and energy consumption, as well study’s models by incorporating additional relevant
as the influence of energy consumption on Bitcoin price factors and scrutinizing the interactions between these
stability [10, 11]. These diverse perspectives and factors and the Fear & Greed Index.
approaches to Bitcoin price forecasting underscore the
multi-faceted nature of factors influencing its valuation.
Aside from these dimensions, another potential
driver of Bitcoin's price that has garnered attention in
recent years is market sentiment. Market sentiment

*Corresponding author: [email protected]

© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution
License 4.0 (https://creativecommons.org/licenses/by/4.0/).
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

2 Data and Method presents the descriptive statistics for the dataset’s two
variables of interest.
2.1 Data Table 1. Statistics of dataset

Statistics Value BTC_Closing

2.1.1 Sample and Variables Mean: 42.39 20598.99

This study utilizes a dataset from Bitcoin & Fear and Median 39 11,464.51
Greed on Kaggle, originating from the alternative. Me. Mode 24 6,741.75
The dataset comprises five variables: Date, Value, Variance 487.85 279,449,800
Value_Classification, BTC_Closing, and BTC_Volume. Max 95 67566.82813
Date records the date information, spanning from
Min 5 3236.761719
February 1, 2018, to the present day. Value is a
numerical representation of the Fear & Greed Index, These statistics offer a preliminary understanding of the
ranging from 0 to 100. Value_Classification categorizes distribution and central tendencies of the Fear & Greed
the index into five levels: Extreme Fear, Fear, Neutral, Index and Bitcoin closing prices. In the subsequent
Greed, and Extreme Greed, reflecting the intensity of sections, the study will delve deeper into the relationship
market sentiment concerning Bitcoin's price. between these variables and employ machine-learning
BTC_Closing denotes the closing price of Bitcoin on the techniques to forecast Bitcoin prices based on the Fear
specified date, while BTC_Volume represents the & Greed Index.
market volume of Bitcoin on that date.

2.1.4 Correlation Analysis

2.1.2 First Preprocesse
To explore the relationship between the fear and greed
The dataset is initially preprocessed to guarantee data index (Value) and the closing price of Bitcoin
quality. The study begins by checking for missing (BTC_Closing), this study conducted a correlation
values. Upon inspection, it is found that data for only analysis and visualized it in Figure 1 and Figure 2.
three days are missing. Given that the study’s dataset Figures 1 and 2 show that the correlation between
consists of 1,885 rows in total, removing these three BTC_Closing and Value is not strong, with a correlation
rows does not significantly impact the results. Therefore, coefficient of 0.24. However, the visualization shows
to preserve the accuracy of the study’s analysis, these that the data points cluster around these three lines. This
three rows are removed from the dataset. lower correlation may be due to the significant ups and
downs of the Bitcoin price, as shown in Figure 3, which
2.1.3 Descriptive Statistics results in significant value changes over a long time
horizon. Also, although the Bitcoin price and the index
To gain a deeper understanding of the study’s dataset, are correlated, the different bases produce a stratified
the study provides some descriptive statistics for the relationship.
Value and BTC_Closing variables in a table. Table 1

Fig. 1. The correlation coefficient of each index

2
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

Fig. 2. Correlation between visualization processing indices

The final results revealed that the correlation was

more significant than 0.6 for a total of 40 weeks out of
62 months or 64.52%. Additionally, a total of 55 out of
62 months exhibited a correlation greater than 0.3,
amounting to a rate of 88.71%.
These findings demonstrate that in the majority of
months (64.52%), there is a strong relationship between
the Fear & Greed Index and Bitcoin prices, as indicated
by a correlation greater than 0.6. In most months
(88.71%), a moderate relationship exists, as suggested
by a correlation greater than 0.3. By excluding months
with significant fluctuations in Bitcoin prices, the study
can conclude that there is a certain degree of correlation
between Bitcoin prices and the Fear & Greed Index. The
Fig. 3. The correlation coefficient of each index study retains data from months with a correlation greater
than 0.6 for further analysis. Consequently, the study
can establish that there is a correlation between Bitcoin
2.1.5 Additional Data Preprocessing prices and the Fear & Greed Index.
Considering these findings, the study further
preprocesses the data to account for the large 2.1.5.1 Bitcoin price normalization
fluctuations in Bitcoin prices on specific days. Three
additional data preprocessing steps are performed: Although the Bitcoin prices are separated by month, in
Divide data by month and select months with higher practice, there are still considerable variations in the
correlations. price of Bitcoin between each month. To harmonize the
To mitigate the impact of large fluctuations in range of values and reduce errors, the study employs the
Bitcoin prices, this study decided to split the dataset by first day of each month as the base and normalizes the
month and investigate the correlation between Bitcoin Bitcoin price for the remaining days.
prices and the Fear & Greed Index for each month. By Normalization is a technique used to transform the
excluding individual months where prices fluctuate scale of variables so that they can be compared on a
significantly due to other factors, we aim to establish a common ground. One standard normalization method is
correlation between Bitcoin prices and the index as long the Min-Max scaling. In this study, the study can use the
as we can achieve a sufficient correlation for most first day of each month as the base and normalize the
months.

3
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

Bitcoin price for the rest of the days using the following Linear regression is a predictive model for estimating a
formula: continuous target variable based on input features. The
model posits a linear relationship between input features
(𝑃𝑃𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 − 𝑃𝑃𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 )
𝑃𝑃𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 = (1) and the target variable. The primary aim of linear
𝑃𝑃𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝
regression is to determine a linear function that
Applying this formula transforms the Bitcoin price minimizes the sum of squared prediction errors.
for each day within a month into a normalized value
relative to the base price (i.e., the price on the first day 2.2.1.2 Formula
of the month). This normalization process helps to
reduce the impact of significant price variations between The linear regression formula can be denoted as:
months. It allows for more accurate comparisons and
analysis of the relationship between the Fear & Greed 𝑦𝑦 = 𝛽𝛽0 + 𝛽𝛽1 × 𝑥𝑥1 + 𝛽𝛽2 × 𝑥𝑥2 + ⋯ + 𝛽𝛽𝑛𝑛 × 𝑥𝑥𝑛𝑛 + 𝜀𝜀 (2)
Index and Bitcoin prices.
In this formula, y is the target variable, xi represents
input features, βi denotes model parameters, β0 is the
2.1.5.2 Split Data into Training and Test Sets
intercept term, and ε is the error term.
After normalizing the dataset and dividing it by month,
the next step is to split the dataset into a training set and 2.2.1.3 Build model
a test set in an 8:2 ratio. This division allows us to train
machine learning models on most data (80%) and The study creates a linear regression model, fits it with
evaluate their performance on the remaining unseen data training data, produces predictions using test data,
(20%). calculates performance metrics, and visualizes predicted
To recap, the following preprocessing steps have results compared to actual values.
been completed:
1. Checking for and removing missing values
2.2.2 Polynomial Regression
2. Calculating descriptive statistics
3. Smoothing the data using a moving average or Polynomial Regression is an extension of linear
other smoothing techniques regression, allowing for modelling non-linear
4. Removing outliers relationships between input features and continuous
5. Dividing the data by month and selecting output targets. By introducing higher-degree polynomial
months with higher correlations terms into the linear regression equation, polynomial
6. Normalizing Bitcoin prices using the first day of regression can effectively capture non-linear patterns in
each month as the base the data.
7. Splitting the dataset into a training set and a test After observing satisfactory performance from linear
set in an 8:2 ratio regression, the study applies polynomial regression for
With the preprocessed dataset, the study can apply further improvement. In this analysis, the study chooses
different machine-learning models to build and evaluate a 5-degree polynomial for fitting.
their effectiveness in predicting Bitcoin prices based on Combining higher accuracy and simplicity of
the Fear & Greed Index and other relevant factors. calculation, this study creates a polynomial regression
Standard models include linear regression, decision model with a degree of 5, fits it with training data,
trees, random forests, and neural networks. After produces predictions using test data, calculates
training and evaluating each model, the study can performance metrics, and visualizes predicted results
compare their performance and select the most suitable compared to actual values.
model for its specific use case.

2.2.3 Random Forest

2.2 Methodology
This section shows the improvement achieved using
polynomial regression compared to linear regression.
2.2.1 Linear Regression
However, the results are still unsatisfactory, leading to
Linear Regression (LR) is a foundational supervised exploring Random Forest regression as a further
learning algorithm employed to ascertain a linear alternative.
association between input features (independent Random Forest is a supervised learning algorithm
variables) and continuous output targets (dependent that can be employed for both classification and
variables). The primary objective of the algorithm is to regression tasks. It is an ensemble method that combines
discover a linear function that minimizes the sum of multiple decision trees to enhance prediction
squared prediction errors and effectively models the performance.
underlying data points.
2.2.3.1 Definition
2.2.1.1 Definition
Random Forest is an ensemble learning technique based
on decision trees. It constructs multiple decision trees

4
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

and aggregates their predictions (through voting or 2.2.4.1 Definition

averaging) to make a final prediction. Random Forest
employs bootstrap sampling to generate multiple XGBoost, or eXtreme Gradient Boosting, is an
training datasets from the original data and builds a ensemble learning technique based on gradient boosting
decision tree on each training set. Additionally, it and decision trees. It constructs multiple decision trees
randomly selects a subset of features during the decision and optimizes the loss function by iteratively adding
tree construction process to increase model diversity. weak learners. XGBoost aims to enhance computational
performance, reduce the risk of overfitting, and allow
2.2.3.2 Formula customization of loss functions and evaluation metrics.
XGBoost is a supervised learning algorithm that can
be employed for classification, regression, and ranking
Random Forest has no specific formula since it is an
tasks. It is an ensemble method that combines multiple
ensemble method based on multiple decision trees.
decision trees using gradient-boosting techniques to
However, the prediction process for Random Forest can
enhance prediction performance.
be represented as follows:
1
𝑦𝑦𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 = × ∑ 𝑦𝑦𝑖𝑖 (3) 2.2.4.2 Formula
𝑁𝑁

Where 𝑦𝑦𝑖𝑖 is the prediction of the 𝑖𝑖th decision tree, XGBoost does not have a specific formula, as it is an
and N is the number of decision trees. The Random ensemble method based on gradient boosting and
Forest prediction is the average of all decision tree decision trees. However, the prediction process for
predictions. XGBoost can be represented as follows:

2.2.3.3 Improvement 𝑦𝑦𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 = ∑𝑘𝑘𝑘𝑘=1 𝑓𝑓𝑘𝑘 (𝑥𝑥𝑖𝑖 ) (4)

Where 𝑓𝑓𝑘𝑘 (𝑥𝑥𝑖𝑖 ) is the prediction of the 𝑘𝑘th decision

Although the Random Forest regression results show tree, and K is the number of boosting rounds. The
improvement over polynomial regression, the XGBoost prediction is the sum of all decision tree
performance is still unsatisfactory. Therefore, Grid predictions, with each tree weighted by its learning rate.
Search is employed to find the optimal parameters and
perform Random Forest regression fitting once again.
Grid Search is a hyperparameter optimization 2.2.4.3 Improvement
technique that aims to identify the best combination of
hyperparameters for a given model by systematically Although the results from Random Forest regression
searching through the parameter space. It performs an show improvement over polynomial regression,
exhaustive search over the specified parameter grid with XGBoost may further enhance the performance. To
cross-validated performance metrics, selecting the optimize the XGBoost regression model,
optimal parameters that maximize the model's hyperparameter tuning techniques, such as Grid Search
performance. or Randomized Search, can be employed.
A Random Forest Regressor object is created in the In the subsequent implementation, an XGBoost
subsequent implementation, and a parameter grid is Regressor object is created, and a parameter grid is
defined to search for the best hyperparameters. A Grid defined to search for the best hyperparameters. A Grid
Search CV object is created, and the model is fitted Search CV or Randomized Search CV object is created,
using the training data. After identifying the best and the model is fitted using the training data. After
parameters, predictions are made using the test data, and identifying the best parameters, predictions are made
performance metrics are computed. Finally, the using the test data, and performance metrics are
predicted and actual values are visualized. computed. Finally, the predicted and actual values are
By incorporating Grid Search into the approach, the visualized.
aim is to optimize the Random Forest regression model's By incorporating hyperparameter tuning into the
performance and achieve more accurate predictions. approach, the aim is to optimize the XGBoost regression
model's performance and achieve more accurate
predictions.
2.2.4 XGBoost
Although the results achieved through random forest 3 Results and Discussion
fitting are pretty satisfactory, with an R² value of 0.95,
which is very close to 1, the study still wants to try one
last method: XGBoost. 3.1 Results of LR
XGBoost is a supervised learning algorithm that can
be employed for classification, regression, and ranking The Mean Squared Error (MSE) of 0.01 and R-squared
tasks. It is an ensemble method that combines multiple value of 0.52 suggest a moderate fit for the model. The
decision trees using gradient-boosting techniques to visualization results (figure 4), with the test set values
enhance prediction performance. roughly around the predicted straight line, further
support this conclusion. While the fit is imperfect, it
demonstrates a good relationship between the observed

5
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

and predicted values. Further improvements to the hyperparameter tuning, feature engineering, or trying
model can be explored through additional feature alternative machine learning algorithms.
engineering, hyperparameter tuning, or trying
alternative machine learning algorithms.

Fig. 6. The results of Random Forest Regression

After applying Grid Search, the results show a

Fig. 4. The result of Linear Regression significant improvement in the R-squared value (0.95),
indicating a solid fit between the predicted and actual
3.2 Results of Polynomial Regression values. The substantial increase in R-squared suggests
that the model has benefited from the hyperparameter
The Mean Squared Error (MSE) of 0.01 and R-squared optimization, which may have led to a better
value of 0.60 for the fifth-order polynomial regression understanding of the underlying patterns in the data
indicate a slightly better fit than the previous model. The (figure 7).
visualization results (figure 5) also show that the curve However, the Mean Squared Error (MSE) has
fits the test set more closely, suggesting an improved increased to 12,293,304, which is unexpected given the
relationship between the observed and predicted values. improvement in R-squared. This discrepancy could be
Although the model has progressed, there may still be due to the increased complexity after the Grid Search or
room for further enhancements, such as exploring potential overfitting, where the model may perform
higher-order polynomials, additional feature exceptionally well on the training data but poorly on
engineering, or experimenting with other machine- unseen data.
learning algorithms.

Fig. 7. The results of Grid Search Random Forest Regression

Fig. 5. The result of Polynomial Regression

3.4 Results of XGBOOST
3.3 Results of Random Forest Regression The Mean Squared Error (MSE) of 9,888,501.7 and R-
squared value of 0.96 for the Random Forest model
The Mean Squared Error (MSE) of 0.01 and R-squared indicate a strong fit between the predicted and actual
value of 0.63 for the Random Forest model indicate an values. The visualization results (figure 8) further
even better fit than the previous models. The support this observation, as the prediction results and the
visualization results (figure 6) further support this test set values are closely aligned. This suggests that the
observation, as the prediction results and the test set model is highly effective in capturing the underlying
values are more closely aligned. This improvement patterns in the data.
suggests that the Random Forest model is more effective
in capturing the underlying patterns in the data. While
the model's performance has increased, additional
enhancements could still be explored, such as

6
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

3.5 Discussion
Table 2 are summary of the results. When comparing the
results of different models, it is essential to consider
both the Mean Squared Error (MSE) and the Coefficient
of Determination (R²). A smaller MSE indicates a minor
prediction error, while a larger R² suggests better
descriptive performance. Due to the significant
fluctuations in Bitcoin prices, the MSE results may be
relatively large, making R² values particularly important.

Fig. 8. The results of Grid Search XGBoost Regression

Table 2. Results of models

Grid Search Grid Search
Polynomial Random Forest
Linear Regression Random Forest XGBoost
regression Regression
Regression Regression
MSE 0.01 0.01 0.01 12293304.44 9888501.70
2
𝑅𝑅 0.52 0.60 0.63 0.95 0.96

The XGBoost and Random Forest models with volatile nature, the study advises exercising caution
hyperparameter tuning show considerably higher R² when investing in Bitcoin.
values than the other models, indicating better Further models and analyses could be improved by
descriptive performance. Among these two, XGBoost considering additional factors, incorporating more
has a relatively smaller MSE, making it superior in sophisticated features, or applying advanced machine-
prediction accuracy. Therefore, considering both its learning techniques. This would help to enhance the
descriptive ability (R²) and prediction accuracy (MSE), reliability and generalization capabilities of the models,
the XGBoost model performs better in predicting allowing for more accurate predictions and better
Bitcoin prices. decision-making in the context of Bitcoin investments.
In summary, the study concludes that the Grid
Search XGBoost model is the best model for predicting
Bitcoin prices in this analysis. References
1. Z. Chen, C. Li, & W.Sun, Journal of Computational
4 Conclusion and Applied Mathematics, 365, (2020)
2. A. H. Dyhrberg, Finance Research Letters, 16,
In summary, the study concludes that the Grid Search (2016)
XGBoost model is the best model for predicting Bitcoin
3. Georgoula, I., Pournarakis, D., Bilanakos, C.,
prices based on the Fear & Greed Index using a machine
Sotiropoulos, D. N., & Giaglis, G. M. SSRN
learning model in Python. The study found that the Fear
2607167, (2015)
& Greed Index strongly correlates with Bitcoin prices.
However, due to the high volatility of Bitcoin prices, it 4. A. Hayes, SSRN 2579445, (2015)
is not sufficient to study the relationship between the 5. L. Kristoufek, Scientific Reports, 3, 1(2013)
index and Bitcoin prices in isolation; the study must also 6. E. Bouri, R. Gupta , A. K. Tiwari, & D. Roubaud,
consider their relationship over time. The study Finance Research Letters, 23, (2017)
ultimately determined that Grid Search XGBoost yields
the best prediction results by utilizing various models 7. P. Ciaian, M. Rajcaniova, & D. A. Kancs, The
and model improvements for Bitcoin price prediction. economics of BitCoin price formation. Applied
This study provides strong evidence that market Economics, 48,19 (2016)
sentiment, as represented by the Fear & Greed Index, is 8. J. Bouoiyour, & R. Selmi, Annals of Economics
closely correlated with Bitcoin prices. This finding and Finance, 16, 2(2015)
offers a foundation for future research on Bitcoin prices 9. D. Koutmos, Annals of Operations Research, 10,
using alternative approaches influenced by market (2018)
sentiment and a direction for incorporating market 10. D.Malone, & K. J. O'Dwyer, Bitcoin mining and its
sentiment when constructing predictive models for energy footprint. In 25th IET Irish Signals &
Bitcoin prices. Systems Conference 2014 and 2014 China-Ireland
It is essential to acknowledge that Bitcoin prices are International Conference on Information and
influenced by various factors, which means the study’s Communications Technologies , (2014)
predictions are subject to a certain degree of error. The
high MSE values of the optimal model also suggest a 11. H. Vranken, Current Opinion in Environmental
potential risk of overfitting. Therefore, given its highly Sustainability, 28, (2017).

7
SHS Web of Conferences 181, 02015 (2024) https://doi.org/10.1051/shsconf/202418102015
ICDEBA 2023

12. S. Nadarajah, & J. Chu, Economics Letters, 174,

(2019).
13. T. Dimpfl & V. Kleiman, Journal of Risk and
Financial Management, 15, 2 (2022).
14. M. Chiah, A. Zhong & L. Yao, Applied Economics
Letters, 1, (2021).