ARIMA vs Prophet vs LSTM

Time series models are statistical and machine learning models used to analyze and forecast data that is collected over time. This type of data is called time series data and it typically consists of observations recorded at regular intervals and choosing the right model depends on your data, the use case and the required forecast horizon.

1. ARIMA(Autoregressive Integrated Moving Average)

ARIMA is a statistical time series model used to forecast future values based on past values and errors. It works on the assumption that future values have a linear relationship with past values and past forecast errors. It is defined by three parameters:

• p: number of autoregressive terms (AR)
• d: number of differencing operations to make the series stationary (I)
• q: number of moving average terms (MA)

Components of ARIMA

1. The autoregressive (AR) component: it models the relationship between the current value and its previous values and assumes that past values have a direct influence on the current value and this dependence can be captured by a weighted sum of prior time points.
2. The differencing (I) component: it is used to make the time series stationary which involves subtracting the previous observation from the current observation removing trends or seasonality that cause non stationarity.
3. The moving average (MA) component: it models the dependency between the current value and past forecast errors which means that errors made in previous predictions are used to improve future forecasts allowing the model to account for shocks or irregularities.

2. Prophet

Prophet is an additive time series forecasting model developed by Meta to provide a flexible and easy to use tool for forecasting. Prophet works by decomposing a time series into components including trend, seasonality and holidays and then combines them into a single forecast.

y(t) = g(t) + s(t) + h(t) + \varepsilon_t

Components of Prophet

• Trend g(t): Models the non periodic changes in the value over time. Prophet supports flexible trend modelling either linear or logistic growth with change points where the trend changes.
• Seasonality s(t): Models periodic effects like daily, weekly or yearly seasonality using Fourier series expansions. Users can enable or disable these seasonalities or add custom ones.
• Holiday Effects h(t): Accounts for the effects of holidays or special events that can cause irregular spikes or drops in the data. Prophet allows users to specify holidays and their impact windows.
• Error term( ​\varepsilon_t ): Captures noise and unexplained variability.

3. LSTM

LSTM is a specialized type of Recurrent Neural Network (RNN) designed to model sequential data and learn long term dependencies. Unlike traditional neural networks LSTMs have a unique architecture that enables them to remember information over long periods and avoid problems like the vanishing gradient that plague standard RNNs.

Components of LSTM

• Memory Cell: The core unit that maintains the cell state which acts like a conveyor belt carrying relevant information through time steps.
• Forget Gate: Decides what information from the previous cell state to discard.
• Input Gate: Determines which new information to add to the cell state.
• Output Gate: Controls what part of the cell state to output.

This gating mechanism allows LSTM networks to selectively remember or forget information and making them capable of capturing long range temporal dependencies.

Comparison of ARIMA, Prophet and LSTM