===== III. Linear Algebra review =====

==== 3.1 - Matrices and Vectors ==== 

  * Dimension of a matrix : number of rows x num of columns, so the matrix \(\left[\begin{matrix} 1 & 2 & 3 \\ 4 & 5 & 6\end{matrix} \right]\) is 2x3. We can also say it is an element of \(\mathbb{R}^{2 \times 3}\).

  * For a Matrix \(A\), \(A_{ij}\) is refering to the element on the i<sup>th</sup> row and the j<sup>th</sup> column. The convention we use here is to have i and j indices starting at 1 on the first row and column.

  * A vector is a **nx1** matrix (eg it is a special case of a matrix with only one column). For instance the vector \(y = \left[\begin{matrix} 1 \\ 2 \\ 3 \\ 4\end{matrix} \right]\) is said to be a 4-dimensioned vector, or a vector from \(\mathbb{R}^{4}\).

  * For a vector y, there are two conventions: we can either use 1-based indices or 0-based indices. By default we should assume 1-indexed vectors in this course.

  * By convention, we use capital letters to refer to matrices and lowercase letters to refer to vectors.

==== 3.2 - Addition and scalar multiplication ==== 

  * For matrix addition and substraction we add/substract per element. We can only add/substract matrices of **the same dimensions**.

  * A **scalar** is a real number.

==== 3.3 - Matrix Vector multiplication ==== 

  * We can only multiply matrices with vectors or other matrices, when the "inner" indices matches: eg. \(A_{l\times n} \times B_{n\times m}\) will give a matrix of dimensions \(C_{l\times m}\).

==== 3.4 - Matrix Matrix multiplication ==== 

  * We can use matrix matrix multiplication to efficiently apply multiple linear regression hypothesis to a data set, for instance:
  \[\left[\begin{matrix} 1 & 2104 \\ 1 & 1416 \\ 1 & 1534 \\ 1 & 853\end{matrix} \right] \times \left[\begin{matrix} -40 & 200 & -10 \\ 0.2 & 0.1 & 0.4 \end{matrix} \right] = \left[\begin{matrix} 486 & 410 & 692 \\ 314 & 342 & 416 \\ 344 & 353 & 464 \\ 173 & 285 & 191\end{matrix} \right] \text{for hypothesis: } \begin{cases}\text{1. } h_\theta(x) = -40 + 0.25x \\ \text{2. } h_\theta(x) = 200 + 0.1x \\ \text{3. } h_\theta(x) = -150 + 0.4x \end{cases}\]


==== 3.5 - Matrix multiplication properties ==== 

  * Matrix matrix multiplication is NOT **commutative**. \(A\times B \ne B\times A\) (The dimensions will not even match.)
  * Matrix multiplcation is **associative**: \(A\times (B \times C) = (A \times B) \times C\) 

  * For scalars, the number 1 is the identity for multiplication.
  * For matrices we usually call the identity matrix \(I\) or sometimes \(I_{n\times n}\). And we have the property: \(A\times I = I \times A = A\) for all matrices A.

==== 3.6 - Inverse and Transpose ====   

  * The inverse \(A^{-1}\) of a matrix \(A\) is defined such as: \(AA^{-1} = A^{-1}A = I\).
  * Note that only **square** matrices (eg. that are mxm) may have an inverse.
  * Matrices that don't have an inverse are called **singular matrices**.

  * If A is a m x n matrix and \(B = A^T\), then B is a n x m matrix and for all i,j, \(B_{ij} = A_{ji}\).