Welcome to the next step in our Introduction to TensorFlow Basics course! In this lesson, we're going to delve deeper into Tensor operations using TensorFlow. We'll learn about basic operations such as tensor addition, multiplication, and broadcasting operations. Let's get started!

Before we start with tensor operations, let's briefly review TensorFlow constant tensors. In the previous lesson, we had covered the creation of constant tensors using the tf.constant() function. In TensorFlow, these constant tensors allow us to store data in arrays of varying dimensions (1D, 2D, 3D, etc.), which are immutable.

For the purpose of this lesson, let's create two constant tensors using TensorFlow. Remember, we can specify the datatype of the tensors using the dtype keyword.

Python
Copy
1import tensorflow as tf
2
3# Creating two tensors
4tensor_a = tf.constant([[1, 2], [3, 4]], dtype=tf.int32)
5tensor_b = tf.constant([[5, 6], [7, 8]], dtype=tf.int32)

We've created two 2x2 tensors with integer elements: tensor_a and tensor_b. Now, let's do some operations on these tensors.

Tensor addition, akin to conventional matrix addition, is an element-wise operation — meaning that the addends must have the same shape. TensorFlow's tf.add() function allows us to perform this operation easily.

Here's how we apply tf.add() to our tensors:

Python
Copy
1# Addition
2tensor_sum = tf.add(tensor_a, tensor_b)
3print(f"Tensor Addition:\n{tensor_sum}\n")

The output of the above code will be:

Copy
1Tensor Addition:
2[[ 6  8]
3 [10 12]]

This output demonstrates how tf.add() performs an element-wise addition, giving us a new tensor where each element is the sum of the corresponding elements in tensor_a and tensor_b.

Element-wise multiplication operates on corresponding elements of the matrices (or tensors). This means the shapes of the two tensors must match exactly.

The tf.multiply() function in TensorFlow helps us perform this operation, as shown in the following code snippet:

Python
Copy
1# Element-wise Multiplication
2tensor_product = tf.multiply(tensor_a, tensor_b)
3print(f"Element-wise Multiplication:\n{tensor_product}\n")

The output of the above code will be:

Copy
1Element-wise Multiplication:
2[[ 5 12]
3 [21 32]]

Here, tensor_product shows the product of corresponding elements from tensor_a and tensor_b. This example straightforwardly illustrates how element-wise multiplication works in TensorFlow.

Matrix multiplication differs from element-wise multiplication in its operation: instead of multiplying corresponding elements, it combines rows of the first matrix with columns of the second, allowing for operations between tensors of different shapes. For matrix multiplication, the crucial requirement is that the number of columns in the first matrix matches the number of rows in the second.

Let's introduce tensor_c with a shape that differs from tensor_a to demonstrate matrix multiplication more effectively.

Python
Copy
1# Introducing tensor_c with a different shape
2tensor_c = tf.constant([[1], [2]], dtype=tf.int32)
3
4# Matrix Multiplication
5tensor_matmul = tf.matmul(tensor_a, tensor_c)
6print(f"Matrix Multiplication:\n{tensor_matmul}\n")

The output of the above code block will be:

Copy
1Matrix Multiplication:
2[[ 5]
3 [11]]

Here, tensor_a has a shape of 2x2 and tensor_c has a shape of 2x1, demonstrating matrix multiplication's requirement that the first tensor's columns align with the second tensor's rows, resulting in a new tensor of shape 2x1.

In TensorFlow, broadcasting allows operations on arrays of different shapes by automatically expanding the smaller array to match the larger one without actually copying data. This is useful not only for basic arithmetic operations but also for matrix multiplication as discussed earlier. Broadcasting follows these rules: dimensions must match or be 1. When comparing dimensions from right to left, sizes must either match or one of them must be 1.

Example 1: Matching Dimensions

Python
Copy
1# Tensor shapes: (2, 3) and (1, 3)
2tensor_a = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.int32)
3tensor_b = tf.constant([[1, 1, 1]], dtype=tf.int32)
4
5# Broadcasting and adding tensors
6result = tensor_a + tensor_b
7print(f"Broadcasted Addition (Matching dimensions):\n{result}\n")

Here, tensor_b is broadcasted to shape (2, 3):

Plain text
Copy
1Broadcasted Addition (Matching dimensions):
2[[2 3 4]
3 [5 6 7]]

Example 2: Dimension of Size 1

Python
Copy
1# Tensor shapes: (2, 1) and (2, 2)
2tensor_c = tf.constant([[1], [2]], dtype=tf.int32)
3tensor_d = tf.constant([[3, 4], [5, 6]], dtype=tf.int32)
4
5# Broadcasting and adding tensors
6result = tensor_c + tensor_d
7print(f"Broadcasted Addition (Dimension of size 1):\n{result}\n")

Here, tensor_c is broadcasted to shape (2, 2):

Plain text
Copy
1Broadcasted Addition (Dimension of size 1):
2[[4 5]
3 [7 8]]

Let's add a scalar to a tensor. The scalar can be considered as having dimensions of size 1 in every compatible dimension, fitting the second broadcasting rule.

Python
Copy
1# Broadcasted Addition (Tensor + scalar)
2tensor_add_scalar = tensor_a + 5
3print(f"Broadcasted Addition (Adding scalar value):\n{tensor_add_scalar}\n")

This output shows the result of adding a scalar value (5) to each element of tensor_a, demonstrating the concept of broadcasting where a scalar is "broadcasted" across an entire tensor.

Plain text
Copy
1Broadcasted Addition (Adding scalar value):
2[[6 7 8]
3 [9 10 11]]

Great job! You've just learned about basic tensor operations in TensorFlow. You've seen how to add and multiply tensors, both element-wise and in the form of matrix multiplication. We also discussed broadcasting, a powerful TensorFlow feature that allows us to perform operations on tensors of different shapes.

Now it's time for some practice exercises. We've prepared some exercises based on tensor operations, which will allow you to apply the concepts we just covered. Practice is key to solidifying your understanding and improving your TensorFlow skills, so let's get started!