====== Week4 - Time Series ====== ===== 4.3 - Time Series Concepts ===== ==== Time Series Processes ==== * Stochastic (Random) Process: \(\{\dots,Y_1,Y_2,\dots,Y_t,Y_{t+1},\dots\} = \{Y_t\}_{t=-\infty}^\infty\) is a sequence of random variables indexed by time. * Observed time series of length T: \(\{Y_1=y1,Y_2=y2,\dots,Y_T=y_T\} = \{y_t\}_{t=1}^T\) ==== Stationary Processes ==== * Intuition: \({Y_t\}\) is stationary if all aspects of its behavior are unchanged by shifts in time. * A Stochastic process \(\{Y_t\}_{t=1}^\infty\) is **strictly stationary** if, for any given finite integer r and for any set of subscripts \(t_1,t_2,\dots,t_r\) the joint distribution of \((Y_t,Y_{t_1},Y_{t_2},\dots,Y_{t_r})\) depends only on \(t_1 - t, t_2-t, \dots, t_r -t\) but not on t. * For a strictly stationary process, Yt has the same mean, variance for all t. * Any function/transformation g(Y) of a strictly stationary process \(\{g(Y_t)\}\) is also strictly stationary. ==== Covariance (Weakly) Stationary Processes \(\{Y_t\}\) ==== * \(E[Y_t] = \mu\) for all t * \(Var(Y_t) = \sigma^2\) for all t * \(cov(Y_t,Y_{t-j}) = \gamma_j\) depends on j and not on t. This is called the **j-lag autocovariance** ===== 4.4 - Autocorrelation ===== * with assumption of covariance stationarity: \(Corr(Y_t,Y_{t-j}) = \rho_j = \frac {cov(Y_t,Y_{t-j})}{sqrt{var(Y_t) var(Y_{t-j})}} = \frac {\gamma_j}{\sigma^2} \) * The **autocorrelation function** (ACF) is the plot of \rho_j against j. ===== 4.5 - White Noise Processes ===== ==== Gaussian White Noise Process ==== * \(Y_t \sim iid N(0,\sigma^2)\) or \(Y_t \sim GW N(0,\sigma^2)\) * \(E[Y_t] = 0, ~ Var(Y_t)=\sigma^2\) * \(Y_t\) is independent of \(Y_s\) for \(t \neq s\) => \(cov(Y_t,Y_{t-s})=0\) for \(t \neq s\) * Note that "iid" means "independent and identically distributed" * Here, \(\{Y_t\}\) represents random draws from the same \(N(0,\sigma^2)\) distribution. ==== Independent White Noise Process ==== * \(Y_t \sim iid (0,\sigma^2)\) or \(Y_t \sim IW N(0,\sigma^2)\) * \(E[Y_t] = 0, ~ Var(Y_t)=\sigma^2\) * \(Y_t\) is independent of \(Y_s\) for \(t \neq s\) * Here, \(\{Y_t\}\) represents random draws from the same distribution. But we do not specify what the distribution is (only its mean and variance). ==== Weak White Noise Process ==== * \(Y_t \sim W N(0,\sigma^2)\) * \(E[Y_t] = 0, ~ Var(Y_t)=\sigma^2\) * \(cov(Y_t,Y_{t-s})=0\) for \(t \neq s\) * Here, \(\{Y_t\}\) represents an uncorrelated stochastic process with given mean and variance. (it does not imply independence). (we can have non-linear dependence) ===== 4.6 - Nonstationary processes ===== ==== Deterministically trending process ==== * \(Y_t = \beta_0 + \beta_1 t + \epsilon_t, ~~ \epsilon_t \sim WN(0,\sigma^2)\) * \(E[Y_t] = \beta_0 + \beta_1 t\), depends on t. * A simple detrending transformation yield a stationary process: \[X_t = Y_t - \beta_0 - \beta_1 t = \epsilon_t\] ==== Random Walk ==== * \(Y_t = Y_{t-1} + \epsilon_t, ~~ \epsilon_t \sim WN(0,\sigma_\epsilon^2), ~~ Y_0\) is fixed. * Then: \(Y_t = Y_0 + \sum\limits_{j=1}^t \epsilon_j\) => \(Var(Y_t) = \sigma_\epsilon^2 \times t\) depends on t. * Simple detrending transformation yield a stationary process: \[\Delta Y_t = Y_t - Y_{t-1} = \epsilon_t\] ===== 4.7 - Moving Average Processes ===== ==== MA(1) Model ==== * \(Y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}, ~~ -\infty \lt \theta \lt \infty, ~~ \epsilon_t \sim iid~ N(0,\sigma^2)\) * We then have \(E[Y_t] = \mu + E[\epsilon_t] + \theta E[\epsilon_{t-1}] = \mu\) * In practice we use -1 < \(\theta\) < 1. * \(\begin{align} Var(Y_t) & = \sigma^2 = E[(Y_t - \mu)^2] \\ & = E[(\epsilon_t + \theta \epsilon_{t-1})^2] \\ & = E[\epsilon_t^2] + 2\theta E[\epsilon_t \epsilon_{t-1}] + \theta^2 E[\epsilon_{t-1}^2] \\ & = \sigma_\epsilon^2 + 0 + \theta^2 \sigma_\epsilon^2 = \sigma_\epsilon^2 (1+\theta^2)\end{align}\) * Similarly we can find that: \(Cov(Y_t,Y_{t-1}) = \gamma_1 = \theta \sigma_\epsilon^2\) * Note that the sign of \(\gamma_1\) depends on the sign of \(\theta\). * So we have \(\rho_1 = \frac{\gamma_1}{\sigma^2} = \frac {\theta \sigma_\epsilon^2}{\sigma_\epsilon^2 (1+\theta^2)} = \frac {\theta}{1+\theta^2}\) * Note that the maximum value of \(\rho_1\) here is +/- 0.5 for \(\theta =\) +/- 1. * Note that \(\gamma_2 = 0\) for this model. * In general for this model \(\gamma_j = 0 \) for j > 1. So Gamma doesn't depend on t but only on j. => MA(1) is **covariance stationary**. === Example === * \(r_t \sim iid ~ N(\mu_r,\sigma_r^2)\) * We consider a time serie on 2 months: \(r_t(2) = r_t + r_{t-1}\) * This monthly time serie will overlap by 1 month: \[r_t(2) = r_t + r_{t-1}\] \[r_{t-1}(2) = r_{t-1} + r_{t-2}\] \[r_{t-2}(2) = r_{t-2} + r_{t-3}\] => Then \(\{r_t(2)\}\) follows an MA(1) process. ===== 4.8 - Autoregressive Processes Part 1 ===== ==== AR(1) Model (mean-adjusted form) ==== * \(Y_t - \mu = \phi (Y_{t-1} - \mu) + \epsilon_t, ~~ -1 \lt \phi \lt 1, ~~ \epsilon_t \sim iid ~N(0,\sigma_\epsilon^2)\) => AR(1) is covariance stationary provided \(-1 \lt \phi \lt 1\). ===== 4.9 - Autoregressive Processes Part 2 ===== * Notion of **Ergodicity** : The time dependence in the data will die progressively. Eg. \(Y_t\) and \(Y_{t-j}\) are essentially independent if j is big enough. * AR(1) is ergodic. Its properties are: \[E[Y_t] = \mu\] \[Var(Y_t) = \sigma^2 = \frac {\sigma_\epsilon^2}{1- \phi^2}\] \[Cov(Y_t,Y_{t-1}) = \gamma_1 = \sigma^2 \phi\] \[Corr(Y_t,Y_{t-1}) = \rho_1 = \frac{\gamma_1}{\sigma^2} = \phi\] \[Cov(Y_t,Y_{t-j}) = \gamma_j = \sigma^2 \phi^j\] \[Corr(Y_t,Y_{t-j}) = \rho_j = \frac{\gamma_j}{\sigma^2} = \phi^j\] * Note that, since \(|\phi| \lt 1\), we have: \(\lim\limits_{j \to \infty} \rho_j = \phi^j = 0\) * Concept of **mean reversion** : the AR(1) will tend to revert "around" the mean when being for a "moment" on one side of the mean of the other (the speed of reversal depends on \(\phi\)). * AR(1) model is a good description for: * Interest rates * Growth rate of macroeconomic variables * Real GDP, industrial production * Money, velocity * Real wages, unemployment.