The Foundation of Linear Algebra: Vectors

Let's start our journey into linear algebra with a quick revision of vectors.

What is a vector?

A vector is a one-dimensional array of objects. You can think of it as a ordered list of values. Vectors are used extensively in almost all fields of scientific study. You probably remember vectors from your Physics lessons, where you learnt to add and subtract vectors representing quantities such as velocty, acceleration, etc. in Newtonian Mechanics. The datasets you will use while performing Machine Learning tasks are generally tabular, indexed by rows and columns. Each column of information can be represented as a vector. 

For example, the marks of 10 students can be represented as a vector $\overrightarrow{m}$

$\overrightarrow{m}=\left[\begin{array}{cccccccccc}78& 87& 65& 74& 56& 90& 85& 79& 62& 70\end{array}\right]$

This is a vector with 10 values. These are usually denoted by $m_1,m_2,m_3,...$ , which are the individual entities of the vector

Now, we move on to the defining property of a vector.

Magnitude And Direction

From your Physics classes, you probably remember vectors as arrows pointing in a certain direction. Vectors are quantities that possess magnitude and direction. In layman's terms, magnitude is the "length" of the vector, and direction is the "angle".

Unit vectors

Unit vectors are vectors of magnitude 1. While dealing with 2D space, we often use the unit vectors ‘i’ and ‘j’, for vectors pointing in the X and Y directions respectively. For 3D space, the unit vectors are 'i', 'j' and 'k' (for X, Y and Z respectively).

Vectors in terms of Magnitude, direction and unit vectors 

If A is the magnitude and θ is the direction of $\overrightarrow{a}$, the vector can be represented as

$\overrightarrow{a}=||\overrightarrow{a}||cos\left(\theta \right)\hat{i}+||\overrightarrow{a}||sin\left(\theta \right)\hat{j}$

The unit vectors ‘i’ and ‘j’, perpendicular to each other, are also called the basis vectors. These vectors form a “basis” of 2D space. This means we can describe all of 2D space using these vectors - all the vectors present in the 2D space of Real numbers can be created using linear combinations of these two basis vectors. Any time we describe a vector numerically, there is an implicit choice of a basis for that vector.

We shall explore what this means in detail later in this module. Similarly, ‘i’, ‘j’ and ‘k’ together define the 3D space.

Why are unit vectors important? Because they are the building blocks that combine linearly to form other vectors. This is an important property. We will study this in detail in further pages. The next page talks about vector operations.

Additional Reading

1. A youtube video that helps you visualise in an animated form, the basic concepts of vectors
2. The modern system of vector analysis was developed by Joseph Willard Gibbs, in the late 1800s. This history of vectors is documented brilliantly by Michael J. Crowe in this short timeline, basically the transcript of a 1967 lecture