Linear Algebra For AI

Scalar

Scalars are single quantities that are fully described by a magnitude (size) alone, without direction.

Notation :

Typically denoted by lowercase letters (e.g., a, b, c).

Examples: Temperature, speed, mass, and time are scalars.

Fun Fact: Scalars are like ingredients in a recipe—they represent quantities without any specific direction attached.

Vector

Vectors are quantities that have both magnitude (size) and direction.

Notation

Typically denoted by bold lowercase letters (e.g. u, v, w) or by having an arrow above letter 

Examples: Displacement, velocity, force, and acceleration are vectors.

Fun Fact: Vectors are like arrows—they point in a specific direction and have a length that represents their magnitude.

Vector Representation

Mathematical Representation: Vectors are typically represented in coordinate form, like shown in above image

Geometric Representation: Visualize vectors as arrows in space, where the arrow's length represents the magnitude and its direction indicates the vector's direction.

Vector Addition

Adding two vectors results in a new vector by combining both the magnitude and direction of the original vectors. Here are two methods to add vectors:

Graphical Method (Tip-to-Tail):

Place the tail of the second vector at the tip of the first vector.

Draw a vector from the tail of the first vector to the tip of the second vector. This resultant vector is the sum of the two vectors.

Algebraic Method (Component-Wise):

Add the corresponding components of each vector.

If a=⟨3,4⟩ and b=⟨1,2⟩, then the resultant vector R=⟨3+1,4+2⟩=⟨4,6⟩

Why is Vector Addition Important?

Vector addition is crucial in physics and engineering. It helps in understanding how different forces, velocities, or displacements combine and interact. For example, calculating the net force on an object involves adding all the individual forces acting on it as vectors.

Scalar Multiplication

Multiplying a vector by a scalar stretches or shrinks the vector.

Example

k. v= k.(x,y)=(kx,ky)

Scalar multiplication is integral to various AI processes, including weight adjustment, normalization, feature scaling, regularization, data transformation, activation functions, and optimization. Its ability to scale and adjust data and parameters makes it a fundamental operation in AI model development and tuning.

Dot Product

The dot product of two vectors is a scalar. Compute the dot product u.v to find the projection of one vector onto another.

Formula

a.b= a1b1+a2b2+a3b3+...+anbn

Example

(1,2,3).(4,5,6) = 1.4+2.5 = 3.6 = 32

Cross Product

The cross product of two vectors in 3D space results in a new vector perpendicular to both original vectors.

How to Calculate (determinant method):

If a=⟨1,2,3⟩ and b=⟨4,5,6⟩

a×b=⟨2x6−3x5,3x4−1x6,1x5−2x4⟩

axb=⟨−3,6,−3⟩

 Magnitude of a Vector

The magnitude (or length) of a vector is a measure of its length.

Formula

• ∥v∥=x12+x22+…+xn2\|

• ∥v∥=(x1^2​+x2^2​+…+xn^2)^(1/2)​​

Example

• if vector a=⟨3,4⟩

• |a|=(3^2+4^2)^1/2=(25)^1/2=5

Unit Vector

A unit vector is a vector with a magnitude of 1. It indicates direction only 

Finding a Unit Vector

• u=v/∥v∥

Example

• For example vector a=⟨3,4⟩ from previous lesson with magnitude 5

• u=1/5⟨3,4⟩ =⟨3/5,4/5⟩ 

Statistics Basics

Data Insights: Statistics helps summarize and interpret data. Learn about mean, median, mode, variance, and standard deviation. These concepts provide insights into data patterns and variability, forming the basis for data analysis in AI.

Hypothesis Testing

Making Decisions: Hypothesis testing evaluates assumptions about data. It helps determine if results are significant or due to chance. Learn about p-values, confidence intervals, and common tests like t-tests, which are vital for validating AI models.

 Gradient Descent

Optimization Algorithm: Gradient descent is an optimization algorithm that adjusts model parameters to minimize error. It's like finding the lowest point in a valley. Understanding gradient descent is crucial for training machine learning models efficiently.