Tensors - English
Tensors - English
Abstract
This document presents tensors as the essential abstraction in data science and machine learn-
ing, offering two complementary perspectives: (1) their nature as a multilevel container that
extends scalars, vectors, and matrices to higher-order structures; and (2) their interpretation as a
multilinear map that models proportional relationships between multiple factors. Through intu-
itive analogies (shelves, cubes, and buildings of data), mathematical examples, and Python code
snippets with NumPy, the numerical construction of tensors, their fundamental operations (outer
product, contraction, and index raising/lowering), and their application in real-world tasks—from
recommendation systems to computer vision and force analysis in engineering—are illustrated. Fi-
nally, practical use cases in physics, machine learning, and image processing are discussed, demon-
strating the versatility and scalability of tensors for representing and processing high-dimensional
data.
1 Introduction
In the era of data science and machine learning, tensors have become the fundamental abstraction for
representing and processing multi-dimensional data. This document offers a global vision of tensors
from two complementary perspectives: first, as a generalization of data structures (scalars, vectors,
and matrices) into multilevel containers; and second, as a multilinear map that captures proportional
relationships between several entities.
In the first part, we intuitively describe what a tensor is and why it is so useful in real applications,
relying on simple analogies (shelves, cubes, and buildings of data) and concrete examples in Python
that illustrate how NumPy encapsulates scalars (order 0), vectors (order 1), and matrices (order 2)
into a single object type. Next, we delve into the concept of multilinearity: we explain the formal
definition of a tensor as a multilinear function over vector spaces and show key examples, such as the
dot product and its use in recommendation systems.
In the second part, we present the approach of tensors as multidimensional arrays of components.
We define the necessary steps to numerically construct a tensor—choosing a basis, evaluating the
tensor machine, and organizing the results in an array—and exemplify how color images, videos, and
other complex data are naturally encoded into tensors of order 3 or higher, with Python code and
visualizations using Matplotlib.
Throughout the document, we also explore basic tensor operations—outer product, contraction,
index raising/lowering—with applied examples for calculating costs, accumulating metrics, and con-
verting units. Finally, we discuss practical applications in physics (curvature and stress tensors),
computer vision, machine learning, and engineering, illustrating how tensors serve as a unifying tool
to model multilinear relationships in a precise and scalable way.
The following sections detail what a tensor is, why a tensor is useful, the utility of a multilevel
container, and what multilinear relationships are.
What is a tensor?
A tensor is a multilevel container that generalizes scalars (order 0), vectors (order 1), and matrices
(order 2) to higher dimensions, capturing multilinear relationships between entities. These allow for
storing data of varying complexity in a uniform and scalable format.
Intuitively, you can imagine it as a series of labeled shelves, where each level adds a new dimension
of information.
1
Single box (scalar, order 0): A scalar is a single value, for example, the current temperature: t =
23 °C.
Row of boxes (vector, order 1): A vector is a list of values: (t1 , t2 , . . . , t7 ) represents the daily
temperatures for a week.
Shelfof rows
(matrix, order 2): A matrix stacks vectors in rows and columns. For example, M =
Mij , with i for cities and j for the days of the week.
Room of shelves (3rd-order tensor): A 3D tensor is a cube of data, like a color image Tijk , where:
• i runs through the height (pixels).
• j runs through the width (pixels).
• k runs through the color channels (R,G,B).
Entire building (4th-order or higher): A higher-order tensor adds more dimensions, such as time
in a video:
Vtijk (t = time instant, i = height, j = width, k = channel).
1 import numpy as np
2
3 # Order 0: Scalar
4 temperature = 23
5 tensor_0 = np . array ( temperature )
6 print ( tensor_0 , tensor_0 . shape ) # ()
7
8 # Order 1: Vector
9 w e e k _ t e m p e r a t u r e s = [23 ,24 ,22 ,25 ,26 ,24 ,23]
10 tensor_1 = np . array ( w e e k _ t e m p e r a t u r e s )
11 print ( tensor_1 , tensor_1 . shape ) # (7 ,)
12
13 # Order 2: Matrix
14 city_temperatures = [
15 [23 ,24 ,22 ,25 ,26 ,24 ,23] ,
16 [20 ,21 ,20 ,22 ,23 ,22 ,21] ,
17 [28 ,29 ,30 ,31 ,29 ,28 ,29]
18 ]
19 tensor_2 = np . array ( c i t y _ t e m p e r a t u r e s )
20 print ( tensor_2 , tensor_2 . shape ) # (3 ,7)
Listing 1: Tensors by Order
Scalability: When moving from a vector (order 1) to an image (order 3) or a video (order 4), you
don’t change tools: you continue working with tensors, only the number of dimensions varies.
Real-world context in data science: The following are some examples:
• A customer dataset can be modeled as a matrix (order-2 tensor): rows=customers, columns=attributes.
• A set of medical images can be represented as an order-4 tensor: (patients × height × width
× channels).
• A language model sometimes uses order-3 tensors: (words × positions × batches of sen-
tences).
2
What are multilinear relationships between entities?
Multilinear relationships between entities are composed of the following elements:
Entities: These are the ”ingredients” or ”factors” that affect an outcome. They can be things like
the direction of the wind, the force applied to an object, or a person’s tastes.
Relationships: This is simply how these ingredients combine to produce an effect.
Linear: This means ”proportional and fair.” If you double an ingredient, the result doubles.
Multi: This means ”several.”
Thus, a multilinear relationship is a fair and proportional relationship that depends on several
factors at once.
The bakery calculates the price using a ”multilinear relationship.” What does this mean?
• Linear with size: If you keep the same decoration but order a cake that is twice as large, the
final price will double.
• Linear with decoration: If you keep the same size but order a decoration that is twice as
complex (e.g., requires twice the working hours), the final price will also double.
A multilinear relationship is precisely this: the final result (the price of the cake) depends on the
combination of several factors, and if you change only one of those factors, the result changes in a
proportional and predictable way.
• The force felt by a material (result), which depends on the direction of the applied pressure
(entity 1) and the orientation of the material’s surface (entity 2).
• The energy generated by a solar panel (result), which depends on the intensity of the sun (entity
1) and the angle of inclination of the panel (entity 2).
In all these cases, a tensor acts as that ”calculator” that understands how all the factors combine
fairly and proportionally to produce a final result.
3
What are standard measurement vectors?
A ”standard measurement vector” is a simple way of referring to what is known in mathematics as a
basis. Let’s think of a very simple analogy: writing a cooking recipe.
Imagine you are at the center of a grid map, where moving from one square to another is
one step. Our ”standard measurement vectors” would be:
1. A single step exactly to the East (let’s call it e⃗1 ).
2. A single step exactly to the North (let’s call it e⃗2 ).
With just these two ”standard measurements,” we can describe any point on the map
precisely. If I want to tell you how to get to a treasure, I don’t say ”walk that way for a
while.” I give you exact instructions based on our standard vectors: ”Walk 5 steps East
and then 3 steps North.”
Why are they ”standard”? Because once we agree on what e⃗1 and e⃗2 are, my ”5 steps East” are
identical to yours. We have created a reliable coordinate system.
4
3 Mathematical Definition
Formally, a tensor admits two equivalent interpretations that we will see next: a tensor as a multilinear
map and a tensor as a multidimensional array of components.
T : V ∗ × · · · × V ∗ × V × · · · × V → R.
| {z } | {z }
p times q times
The tensor is called multilinear because if one of its arguments (input vectors) is scaled and the
others are held fixed, the numerical result is scaled by the same proportion. Multilinearity ensures
that scaling one of the arguments proportionally scales the result.
Explanation of the (p, q) notation: The type (p, q) notation acts as a label that tells us how many
inputs of each class a tensor requires:
• p (first number): number of covectors. A covector is a linear function that takes a vector and
returns a number.
• q (second number): number of normal vectors (those with direction and magnitude).
Example: The Dot product: One of the simplest ”machines” of this type is the dot product.
Consider the tensor defined by
T (u, v) = u · v
which takes two vectors u and v and returns a number.
1. How many ingredients (inputs) does it take in total? It takes two: the vector u and the vector v.
2. What type are these ingredients? Both u and v are normal vectors. There are no covectors in
the input list.
(0, 2)
because it is a ”machine” that operates on zero covectors and two vectors to produce a single number.
So a (0,2)-type tensor takes two vectors (u and v) and combines them.
n
X
T (u, v) = u · v = ui vi
i=1
This operation is multilinear (in this case, bilinear). If you make the vector u twice as long but keep
v the same, the result of the dot product will also double, and vice versa.
5
Numerical demonstration of linearity: Let’s take two simple vectors:
2 4
⃗u = and ⃗v =
3 1
⃗u′ · ⃗v = (4 × 4) + (6 × 1) = 16 + 6 = 22
As you can see, the result (22) is exactly double the original result (11), demonstrating the linearity
property.
1 import numpy as np
2 u = np . array ([2 ,3]) ; v = np . array ([4 ,1])
3 print ( np . dot (u , v ) ) # 11
4 print ( np . dot (2* u , v ) ) # 22
Listing 2: Demonstration of linearity
Context: We will recommend movies based on three genres: Action, Science Fiction, and Com-
edy.
Step 1: Define the vectors. We represent the tastes of a user Alex and the characteristics of two
movies as vectors.
9 ← Action 10 0
⃗uAlex = 8 ← Sci-Fi ⃗vA = 8 (Cyber-Fighter) ⃗vB = 1 (Office Laughs)
1 ← Comedy 0 9
Step 2: Calculate the similarity score. We calculate the dot product of the user with each
movie.
1 import numpy as np
2
3 # --- Step 1: Define the Vectors ---
4
5 # The genres are in the order : [ Action , Science Fiction , Comedy ]
6
7 # Vector representing the tastes of user Alex .
8 # He really likes Action (9) and Science Fiction (8) , but not Comedy (1) .
9 alex_tastes = np . array ([9 , 8 , 1])
10
11 # Vector representing the features of Movie A : " Cyber - Fighter ".
12 # It ’s pure Action (10) and Science Fiction (8) , with no Comedy (0) .
13 movie_A = np . array ([10 , 8 , 0])
6
14
15 # Vector representing the features of Movie B : " Office Laughs ".
16 # It has no Action (0) and almost no Sci - Fi (1) , but is pure Comedy (9) .
17 movie_B = np . array ([0 , 1 , 9])
18
19 print ( " --- Defined Vectors ---" )
20 print ( f " Alex ’s Tastes : { alex_tastes } " )
21 print ( f " Features of ’ Cyber - Fighter ’: { movie_A } " )
22 print ( f " Features of ’ Office Laughs ’: { movie_B }\ n " )
23
24
25 # --- Step 2: Calculate Similarity Score ( Dot Product ) ---
26
27 # We use the np . dot () function from NumPy to calculate the dot product .
28 score_A = np . dot ( alex_tastes , movie_A )
29 score_B = np . dot ( alex_tastes , movie_B )
30
31 print ( " --- Calculated Similarity Scores ---" )
32 print ( f " Score for ’ Cyber - Fighter ’: { score_A } " )
33 print ( f " Score for ’ Office Laughs ’: { score_B }\ n " )
34
35
36 # --- Step 3: Conclusion ---
37 print ( " --- Re commenda tion Conclusion ---" )
38 if score_A > score_B :
39 print ( f " Since { score_A } > { score_B } , the system would recommend ’ Cyber - Fighter ’ to
Alex . " )
40 else :
41 print ( f " Since { score_B } > { score_A } , the system would recommend ’ Office Laughs ’ to
Alex . " )
Listing 3: Simple Recommendation System Code
Step 3: Conclusion. Since 154 > 17, the system concludes that Alex will like Cyber-Fighter much
more. This mechanism is the basis for more complex techniques like matrix factorization in the
field of collaborative filtering.
Data science example: A color image. A digital color image is a perfect example of an order-3
tensor. It is not a single matrix (order 2), but a set of three stacked matrices, one for each color
channel: Red, Green, and Blue (RGB).
How are the colors and structure stored? Let’s imagine a tiny 2x2 pixel image. Its repre-
sentation as a tensor Tijk would consist of three 2x2 matrices, where the index k tells us which color
matrix we are looking at.
210 205 90 95 30 25
50 45 240 235 80 75
| {z } | {z } | {z }
Red Matrix (R) Green Matrix (G) Blue Matrix (B)
k=1 k=2 k=3
The complete tensor is the collection of all these numbers. The indices i (row) and j (column) give
us the pixel’s position.
7
How is the color of a single pixel obtained? To know the color of the bottom-left corner
pixel (row 2, column 1), we take the component from that position in each of the three matrices:
The combination of these three values (R = 50, G = 240, B = 80) defines the specific color of that
pixel (in this case, a shade of teal). Therefore, the ”cube” of numbers that forms the tensor is the
complete numerical representation of the image. In a data science library, the shape of this tensor
would be (2, 2, 3), representing (height, width, channels).
1 import numpy as np
2 import matplotlib . pyplot as plt
3
4 # We create the order -3 tensor representing a 2 x2 pixel image .
5 # The structure is ( height , width , co lor_cha nnels ) -> (2 , 2 , 3) .
6 # The channels are in the order ( Red , Green , Blue ) .
7 image_tensor = np . array ([
8 # Row 1 of pixels : [[ R ,G , B ] , [R ,G , B ]]
9 [ [210 , 90 , 30] , [205 , 95 , 25] ] ,
10 # Row 2 of pixels : [[ R ,G , B ] , [R ,G , B ]]
11 [ [50 , 240 , 80] , [45 , 235 , 75] ]
12 ] , dtype = np . uint8 ) # We use uint8 (0 -255) , the standard data type for images .
13
14 print ( " --- 2 x2 Image Tensor ---" )
15 print ( image_tensor )
16 print ( f " \ nShape of the tensor : { image_tensor . shape }\ n " )
17
18 # --- Accessing the components of a specific pixel ---
19 # For the pixel at row 2 , column 1 , we use indices [1 , 0]
20 # because in programming , indices start at 0.
21 row_idx = 1
22 col_idx = 0
23 pixel_color = image_tensor [ row_idx , col_idx ]
24
25 print ( f " --- Components of the pixel at ( row ={ row_idx +1} , column ={ col_idx +1}) ---" )
26 print ( f " - Red component ( k =1) : { pixel_color [0]} " )
27 print ( f " - Green component ( k =2) : { pixel_color [1]} " )
28 print ( f " - Blue component ( k =3) : { pixel_color [2]}\ n " )
29
30 # --- Visualizing the tensor as an image ---
31 print ( " --- Visualizing the Tensor as an Image ---" )
32 # Matplotlib directly interprets an array of this shape as an image .
33 plt . imshow ( image_tensor )
34 plt . title ( " Visualization of the 2 x2x3 Tensor " )
35 # We turn off the axes for a better image visualization .
36 plt . axis ( ’ off ’)
37 plt . show ()
Listing 4: Creating and visualizing an image tensor in NumPy
Why does the code create a 3-dimensional array? The way the code is written is the standard
syntax in Python for creating nested lists. The NumPy library is designed to interpret this nesting
and automatically construct a multidimensional structure. The final result is indeed a 3-dimensional
array.
Breaking down the code structure Let’s analyze the structure of the Python list that is passed
to np.array():
1 # The outermost list defines the complete tensor ( the image )
2 [
3 # This inner list represents the FIRST ROW of pixels
4 [
5 [210 , 90 , 30] , # Pixel at Row 1 , Column 1 ( with its 3 R ,G , B values )
8
6 [205 , 95 , 25] # Pixel at Row 1 , Column 2 ( with its 3 R ,G , B values )
7 ],
8
9 # This inner list represents the SECOND ROW of pixels
10 [
11 [50 , 240 , 80] , # Pixel at Row 2 , Column 1 ( with its 3 R ,G , B values )
12 [45 , 235 , 75] # Pixel at Row 2 , Column 2 ( with its 3 R ,G , B values )
13 ]
14 ]
Listing 5: Structure of the nested list in Python
When we pass this structure to np.array(), NumPy analyzes it and creates the dimensions based
on the levels of nesting:
1. Dimension 1 (Height): NumPy sees that the main list contains 2 elements (the two rows).
Therefore, the first part of the tensor’s shape is 2.
2. Dimension 2 (Width): Then, it looks inside each row and sees that each one contains 2
elements (the two pixels). Therefore, the second part of the shape is 2.
3. Dimension 3 (Color Channels): Finally, it looks inside each pixel and sees that each one
contains 3 numbers (the R, G, B values). Therefore, the third part of the shape is 3.
The final result is a single NumPy object with a shape of (2, 2, 3), which is, by definition, a
3-dimensional array or an order-3 tensor.
Why isn’t it represented as three separate matrices? Although conceptually we can think of
it as three stacked matrices, in data science practice it is much more efficient and convenient to have
all the image information in a single, coherent data structure.
• Having a single tensor image tensor, we can access the full color of a pixel at once with
image tensor[1, 0], which gives us the array [50, 240, 80].
• If we had three separate matrices (red matrix, green matrix, blue matrix), to get the same
color we would have to make three different accesses: red matrix[1, 0], green matrix[1, 0],
and blue matrix[1, 0], which is less practical and efficient.
Summary The syntax of the code is the standard way in Python/NumPy to initialize a multidi-
mensional array from static data, and the result is indeed a single, cohesive 3-dimensional array.
4 Summary of Properties
4.1 Tensor orders (or ranks)
The order of a tensor is the number of dimensions in its array of components.
Order Example
0 Scalar a ∈ R
1 Vector v i
2 Matrix M i j
3 Data cube T ijk or T i jk
... Tensors of order d
9
Their outer product is
1·3 1·4 1·5 3 4 5
u⊗v = = ,
2·3 2·4 2·5 6 8 10
Example: Price Matrix Imagine you want to buy chairs for your living room. There are
two types of chairs:
3
u= (3 wooden chairs, 2 wicker chairs).
2
And there are three stores, each with a different price per chair:
50 $
v = 60 $ (price per chair at Stores A, B, and C).
55 $
Their outer product u ⊗ v gives you the cost matrix if you buy all the chairs of a single type at
each store:
3 × 50 3 × 60 3 × 55 150 180 165
u⊗v = = .
2 × 50 2 × 60 2 × 55 100 120 110
The first row shows how much you would pay for the wooden chairs at each store, and the second
row shows what you would pay for the wicker chairs.
1 import numpy as np
2
3 # Vector definitions
4 u = np . array ([3 , 2]) # 3 wooden chairs , 2 wicker
5 v = np . array ([50 , 60 , 55]) # prices at stores A , B , C
6
7 # Outer product
8 costs = np . outer (u , v )
9
10 print ( " Cost matrix : " )
11 print ( costs )
12 # Output :
13 # [[150 180 165]
14 # [100 120 110]]
15
tr(A) = 5 + 2 + 7 = 14.
Example: Favorite Tea Combinations Carmen notes how many cups of tea she drinks
according to the time of day and the type of tea:
10
– Columns (j): type of tea
∗ j = 1: Black tea
∗ j = 2: Green tea
∗ j = 3: Chamomile
Each entry Ai j is the number of cups of tea of type j she drinks at time i. For example:
5 1 3
A = 8 2 9
4 6 7
– A1 1 = 5: black tea in the morning.
– A2 2 = 2: green tea in the afternoon.
– A3 3 = 7: chamomile at night.
The trace by contracting the upper and lower index is
tr(A) = 5 + 2 + 7 = 14.
This means that, in total, Carmen enjoyed 14 cups of tea in her favorite combinations of the
day.
1 import numpy as np
2
3 # Matrix definition
4 A = np . array ([[5 ,1 ,3] ,
5 [8 ,2 ,9] ,
6 [4 ,6 ,7]])
7
8 # Trace calculation
9 trace = np . trace ( A )
10
11 print ( " Matrix A : " )
12 print ( A )
13 print ( " Trace of A : " , trace )
14 # Output :
15 # Trace of A : 14
16
• Index raising/lowering: With a ”metric” gij , you can convert indices from one type to another,
as in vj = gij v i .
Example: Converting Cups to Grams Imagine that ”lowering” an index is like converting
cups of flour to grams using a conversion table. This process is very similar to how in geometry
and physics we use a metric gij to convert between two types of vectors.
1. Original measurement: You have 3 cups of flour. Let’s call this quantity v i .
2. Conversion table: We know that each cup is equivalent to 200 grams. This equivalence
is our ”metric” gij .
3. Convert (lower the index):
g
vj = gij v i =⇒ vj = 200 cup × 3 cups = 600 g.
11
In summary, lowering or raising an index with a metric gij is analogous to converting units:
– v i = quantity in cups (contravariant).
– gij = conversion factor (degrees of equivalence).
– vj = quantity in grams (covariant).
Just as we multiply cups by grams per cup to get grams, in geometry and physics we use the
metric gij to convert a contravariant vector v i to its covariant form vj . This way we avoid
confusion and keep everything fair and proportional.
1 # Quantity in cups
2 cups = 3
3
4 # Conversion metric ( grams per cup )
5 g = 200 # g / cup
6
7 # Lower the index ( convert to grams )
8 grams = g * cups
9
10 print ( f " { cups } cups = { grams } grams " )
11 # Output :
12 # 3 cups = 600 grams
13
Thus, in physics and geometry, contravariant and covariant vectors transform exactly like our
cups and grams, in a fair and proportional way.
Physics and differential geometry: Imagine a sheet under which you place a ball. When you push
the ball down, the sheet curves and stretches. The curvature tensor is like a map that tells you
at each point how much the sheet is bending. On the other hand, if you step on the ball, the
stress-energy tensor tells you how much force and energy is distributed throughout the sheet and
the ball. Thus, physicists use these ”maps” to understand how objects bend and tense in space.
Machine Learning: Think of a large collection of postcards from your album: some are photos,
others are written news snippets, and others are tiny videos. For a computer to ”understand”
all this, it groups each type into boxes labeled by day and theme. These stacked boxes form
a tensor. Tools like TensorFlow or PyTorch handle these tensors to detect patterns, such as
recognizing a face in a photo or translating a text.
Computer vision: Imagine you record a spinning carousel with your old camera and then show it
frame by frame. With tensors, a program separates each image and calculates where each part
of the carousel is moving. It’s as if an assistant took each photo for you, pointed out the horse’s
movement, and then told you exactly how much it turned each second.
Engineering: When you sit in your favorite chair, the legs and seat receive pressure. A stress and
strain tensor is like a network of tiny imaginary sensors that tell an engineer how much force
each part of the chair receives and how the metal or wood bends or stretches. Thanks to this
”force map,” they can design safer and more comfortable chairs.
12