Welcome to the next step in our Introduction to TensorFlow Basics course! In this lesson, we're going to delve deeper into Tensor operations using TensorFlow. We'll learn about basic operations such as tensor addition, multiplication, and broadcasting operations. Let's get started!
Before we start with tensor operations, let's briefly review TensorFlow constant tensors. In the previous lesson, we had covered the creation of constant tensors using the tf.constant()
function. In TensorFlow, these constant tensors allow us to store data in arrays of varying dimensions (1D, 2D, 3D, etc.), which are immutable.
For the purpose of this lesson, let's create two constant tensors using TensorFlow. Remember, we can specify the datatype of the tensors using the dtype
keyword.
Python1import tensorflow as tf 2 3# Creating two tensors 4tensor_a = tf.constant([[1, 2], [3, 4]], dtype=tf.int32) 5tensor_b = tf.constant([[5, 6], [7, 8]], dtype=tf.int32)
We've created two 2x2 tensors with integer elements: tensor_a
and tensor_b
. Now, let's do some operations on these tensors.
Tensor addition, akin to conventional matrix addition, is an element-wise operation — meaning that the addends must have the same shape. TensorFlow's tf.add()
function allows us to perform this operation easily.
Here's how we apply tf.add()
to our tensors:
Python1# Addition 2tensor_sum = tf.add(tensor_a, tensor_b) 3print(f"Tensor Addition:\n{tensor_sum}\n")
The output of the above code will be:
1Tensor Addition: 2[[ 6 8] 3 [10 12]]
This output demonstrates how tf.add()
performs an element-wise addition, giving us a new tensor where each element is the sum of the corresponding elements in tensor_a
and tensor_b
.
Element-wise multiplication operates on corresponding elements of the matrices (or tensors). This means the shapes of the two tensors must match exactly.
The tf.multiply()
function in TensorFlow helps us perform this operation, as shown in the following code snippet:
Python1# Element-wise Multiplication 2tensor_product = tf.multiply(tensor_a, tensor_b) 3print(f"Element-wise Multiplication:\n{tensor_product}\n")
The output of the above code will be:
1Element-wise Multiplication: 2[[ 5 12] 3 [21 32]]
Here, tensor_product
shows the product of corresponding elements from tensor_a
and tensor_b
. This example straightforwardly illustrates how element-wise multiplication works in TensorFlow.
Matrix multiplication differs from element-wise multiplication in its operation: instead of multiplying corresponding elements, it combines rows of the first matrix with columns of the second, allowing for operations between tensors of different shapes. For matrix multiplication, the crucial requirement is that the number of columns in the first matrix matches the number of rows in the second.
Let's introduce tensor_c
with a shape that differs from tensor_a
to demonstrate matrix multiplication more effectively.
Python1# Introducing tensor_c with a different shape 2tensor_c = tf.constant([[1], [2]], dtype=tf.int32) 3 4# Matrix Multiplication 5tensor_matmul = tf.matmul(tensor_a, tensor_c) 6print(f"Matrix Multiplication:\n{tensor_matmul}\n")
The output of the above code block will be:
1Matrix Multiplication: 2[[ 5] 3 [11]]
Here, tensor_a
has a shape of 2x2 and tensor_c
has a shape of 2x1, demonstrating matrix multiplication's requirement that the first tensor's columns align with the second tensor's rows, resulting in a new tensor of shape 2x1.
In TensorFlow, broadcasting allows operations on arrays of different shapes by automatically expanding the smaller array to match the larger one without actually copying data. This is useful not only for basic arithmetic operations but also for matrix multiplication as discussed earlier. Broadcasting follows these rules: dimensions must match or be 1. When comparing dimensions from right to left, sizes must either match or one of them must be 1.
Example 1: Matching Dimensions
Python1# Tensor shapes: (2, 3) and (1, 3) 2tensor_a = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.int32) 3tensor_b = tf.constant([[1, 1, 1]], dtype=tf.int32) 4 5# Broadcasting and adding tensors 6result = tensor_a + tensor_b 7print(f"Broadcasted Addition (Matching dimensions):\n{result}\n")
Here, tensor_b
is broadcasted to shape (2, 3)
:
Plain text1Broadcasted Addition (Matching dimensions): 2[[2 3 4] 3 [5 6 7]]
Example 2: Dimension of Size 1
Python1# Tensor shapes: (2, 1) and (2, 2) 2tensor_c = tf.constant([[1], [2]], dtype=tf.int32) 3tensor_d = tf.constant([[3, 4], [5, 6]], dtype=tf.int32) 4 5# Broadcasting and adding tensors 6result = tensor_c + tensor_d 7print(f"Broadcasted Addition (Dimension of size 1):\n{result}\n")
Here, tensor_c
is broadcasted to shape (2, 2)
:
Plain text1Broadcasted Addition (Dimension of size 1): 2[[4 5] 3 [7 8]]
Let's add a scalar to a tensor. The scalar can be considered as having dimensions of size 1 in every compatible dimension, fitting the second broadcasting rule.
Python1# Broadcasted Addition (Tensor + scalar) 2tensor_add_scalar = tensor_a + 5 3print(f"Broadcasted Addition (Adding scalar value):\n{tensor_add_scalar}\n")
This output shows the result of adding a scalar value (5) to each element of tensor_a
, demonstrating the concept of broadcasting where a scalar is "broadcasted" across an entire tensor.
Plain text1Broadcasted Addition (Adding scalar value): 2[[6 7 8] 3 [9 10 11]]
Great job! You've just learned about basic tensor operations in TensorFlow. You've seen how to add and multiply tensors, both element-wise and in the form of matrix multiplication. We also discussed broadcasting, a powerful TensorFlow feature that allows us to perform operations on tensors of different shapes.
Now it's time for some practice exercises. We've prepared some exercises based on tensor operations, which will allow you to apply the concepts we just covered. Practice is key to solidifying your understanding and improving your TensorFlow skills, so let's get started!