Math Symbols Explained with Python
Learn the meaning behind mathematical symbols used in Machine Learning using your knowledge of Python.
When working with Machine Learning projects, you will come across a wide variety of equations that you need to implement in code. Mathematical notations capture a concept so eloquently but unfamiliarity with them makes them obscure.
In this post, I’ll be explaining the most common math notations by connecting it with its analogous concept in Python. Once you learn them, you will be able to intuitively grasp the intention of an equation and be able to implement it in code.
$$\frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y_i})^2$$
Indexing
$$x_i$$
This symbol is taking the value at \( i^{\\{th}} \) index of a vector.
x = [10, 20, 30]
i = 0
print(x[i]) # 10
This can be extended for 2D vectors and so on.
$$x_{ij}$$
x = [ [10, 20, 30], [40, 50, 60] ]
i = 0
j = 1
print(x[i][j]) # 20
Sigma
$$\sum_{i=1}^{N} x_i$$
This symbol finds the sum of all elements in a vector for a given range. Both lower and upper limits are inclusive. In Python, it is equivalent to looping over a vector from index 0 to index N-1. Notice how we’re using the previously explained \(x_{i}\) symbol to get the value at index.
x = [1, 2, 3, 4, 5]
result = 0
N = len(x)
for i in range(N):
result = result + x[i]
print(result)
The above code can even be shortened using built-in functions in Python as
x = [1, 2, 3, 4, 5]
result = sum(x)
Average
$$\frac{1}{N}\sum_{i=1}^{N} x_i$$
Here we reuse the sigma notation and divide by the number of elements to get an average.
x = [1, 2, 3, 4, 5]
result = 0
N = len(x)
for i in range(N):
result = result + x[i]
average = result / N
print(average)
The above code can even be shortened in Python as
x = [1, 2, 3, 4, 5]
result = sum(x) / len(x)
PI
$$\prod_{i=1}^{N} x_i$$
This symbol finds the product of all elements in a vector for a given range. In Python, it is equivalent to looping over a vector from index 0 to index N-1 and multiplying them.
x = [1, 2, 3, 4, 5]
result = 1
N = len(x)
for i in range(N):
result = result * x[i]
print(result)
Pipe
The pipe symbol can mean different things based on where it’s applied.
Absolute Value
$$\lVert x \rVert$$
$$\lVert y \rVert$$
This symbol denotes the absolute value of a number i.e. without a sign.
x = 10
y = -20
abs(x) # 10
abs(y) # 20
Norm of vector
$$\lVert x \rVert$$
The norm is used to calculate the magnitude of a vector. In Python, this means squaring each element of an array, summing them and then taking the square root.
import math
x = [1, 2, 3]
math.sqrt(x[0]**2 + x[1]**2 + x[2]**2)
Belongs to
$$3\ \in\ X$$
This symbol checks if an element is part of a set. In Python, this would be equivalent to
X = {1, 2, 3}
3 in X
Function
$$f: X \rightarrow Y$$
This denotes a function which takes a domain X and maps it to range Y. In Python, it’s equivalent to taking a pool of values X, doing some operation on it to calculate pool of values Y.
def f(X):
Y = ...
return Y
You will encounter the following symbols in place of X and Y. Here are what they mean:
$$f: R \rightarrow R$$
R means input and outputs are real numbers and can take any value (integer, float, irrational, rational). In Python, this is equivalent to any value except complex numbers.
import math
x = 1
y = 2.5
z = math.pi
You will also encounter symbols such as
$$f: R^d \rightarrow R$$
$$R^d$$
means d-dimensional vector of real numbers.
Let’s assume d = 2. In Python, an example can be a function that takes 2-D array and returns it’s sum. It will be mapping a \( R^d \) to $ R $
X = [1, 2]
f = sum
Y = f(X)
Tensors
Transpose
$$X^{\mathsf{T}}$$
This is basically exchanging the rows and columns. In Python, this would be equivalent to
import numpy as np
X = [[1, 2, 3],
[4, 5, 6]]
np.transpose(X)
Output would be a list with exchanged rows and columns.
[[1, 4], [2, 5], [3, 6]]
Element wise multiplication
$$z = x \odot y$$
It means multiplying the corresponding elements in two tensors. In Python, this would be equivalent to multiplying the corresponding elements in two lists.
import numpy as np
x = [[1, 2],
[3, 4]]
y = [[2, 2],
[2, 2]]
z = np.multiply(x, y)
Output is [[2, 4]], [[6, 8]]
Dot Product
$$xy$$
$$x \cdot y$$
It gives the sum of the products of the corresponding entries of the two sequences of numbers.
x = [1, 2, 3]
y = [4, 5, 6]
dot = sum([i*j for i, j in zip(x, y)])
# 1*4 + 2*5 + 3*6
# 32
Hat
$$\hat{x}$$
The hat gives the unit vector. This means dividing each component in a vector by it’s length(norm).
import math
x = [1, 2, 3]
length = math.sqrt(sum([e**2 for e in x]))
x_hat = [e/length for e in x]
This makes the magnitude of the vector 1 and only keeps the direction.
import math
math.sqrt(sum([e**2 for e in x_hat]))
# 1.0
Exclamation
$$x!$$
This denotes the factorial of a number. It is the product of numbers starting from 1 to that number. In Python, it can be calculated as
x = 5
fact = 1
for i in range(x, 0, -1):
fact = fact * i
print(fact)
The same thing can also be calculated using built-in function.
import math
x = 5
math.factorial(x)
The output is
# 5*4*3*2*1
120
Some additional infos
From https://news.ycombinator.com/item?id=22644620 :
Average of a finite series
There's a statistics module in Python 3.4+:
X = [1, 2, 3]
from statistics import mean, fmean
mean(X)
# may or may not be preferable to
sum(X) / len(X)
https://docs.python.org/3/library/statistics.html#statistics.fmean
Product of a terminating iterable
import operator
from functools import reduce
# from itertools import accumulate
reduce(operator.mul, X)
https://docs.python.org/3/library/functools.html#functools.reduce
Vector norm
from numpy import linalg as LA
LA.norm(X)
https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.norm.html
Function domain and range
Function domains and ranges can be specified and checked at compile-time with type annotations or at runtime with type()/isinstance() or with something like pycontracts or icontracts for checking preconditions and postconditions.
isinstance() and type()
from numbers import Number
from fractions import Fraction
from decimal import Decimal
x_fraction = Fraction("1/2") or Fraction(1, 2)
x_decimal = Decimal("0.3")
assert isinstance(x_fraction, Fraction)
assert isinstance(x_fraction, Number)
assert type(x_decimal) == Decimal
assert type(1j) == complex
Type annotations
from numbers import Number
from typing import Callable, Iterable, Optional, Union
def sum(a: int, b: int) -> int:
return a + b
def sum(a: float, b: Number) -> Number:
return a + b
def sum(a: Number, b: Number) -> Union[Number, None]:
return a + b if not (a is None or b is None) else None
def sum(a: Number, b: Number) -> Optional[Number]:
return a + b if not (a is None or b is None) else None
def sum(a: Number, b: Number) -> Optional[Number]:
if not isinstance(a, Number):
raise TypeError(("a is not a", Number))
return a + b if not (a is None or b is None) else None
def map(function: Callable, sequence: Iterable) -> Iterable:
return (function(x) for x in sequence)
https://docs.python.org/3/library/typing.html :
Note The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
Preconditions and Postconditions and Design-By-Contract
def sum(a: Number, b: Number) -> Optional[Number]:
# preconditions
if not isinstance(a, Number):
raise TypeError(("a is not a", Number, "a is a", type(a)))
if a < 0:
raise ValueError(("a is < 0", a))
# function ("command" in Hoare Logic)
if b is None:
return None
a_ = a
output = a_ + b
# postconditions
assert isinstance(output, (Number, bool, int, float, complex, Fraction, Decimal))
assert a == a_ # "invariance"
# a['num1'] += 1; assert a['num1'] == a['num1']
return output
import pytest
def test_sum():
assert sum(False, True) == 1
with pytest.raises(TypeError):
sum(None, 0)
with pytest.raises(ValueError):
sum(-1, 0)
assert sum(1, None) == None
# !pip install pytest-cov
# !pytest -v --cov=this_file --cov-report=term-missing ./this_file.py
test_sum()
In addition to as conditionals that raise subclasses of Exception like TypeError and ValueError and assertions that raise AssertionError,
runtime-checked preconditions and postconditions can be specified as annotations, decorators, or in docstrings:
Dot product
Y = [4, 5, 6]
np.dot(X, Y)
https://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html
Unit vector
X / np.linalg.norm(X)
Also
Exponentiation and XOR
x**3 # x*x*x
x**1/2 # math.sqrt(x)
x^2 # x XOR 2
x^3 # operator.xor(x, 3)
Matrix multiplication operator
import numpy as np
a = np.matrix([[1, 0], [0, 1]])
b = np.matrix([[4, 1], [2, 2]])
assert a @ b == np.matmul(a, b)
p.testing.assert_equal(np.matmul(a,b), a @ b)
