Lesson 1

Understanding Neural Networks: An Introduction to the Perceptron Algorithm


Welcome aboard our exploration of the building block of Neural Networks: the perceptron! This significant algorithm sparks your comprehension of advanced Neural Networks utilized in Machine Learning. The objectives of this lesson include mastering and coding a perceptron using Python from scratch. We will decipher the structure of a perceptron, the prediction method, and the training process. To conclude, we will design a fully functioning model that abstracts a simple logical scenario using the AND operator data.

Understanding the Perceptron

We start by delving into the perceptron, a simple form of binary linear classifiers in the Neural Network family. A perceptron operates by accepting multiple inputs, aggregating them, and democratically deciding the output based on these inputs.

Think of perceptrons as a democratic process. Each "voter" (input) contributes with differing weights. The "candidate" (output) who secures the majority of votes (aggregate of weighted inputs) wins and is chosen.

Mathematically, the predicted output of a perceptron can be formulated as follows:

y=f(wx+b)y = f(w \cdot x+b)

For our purposes:

  • yy is our output, which we are predicting.
  • ww represents weights — consider these as the importance accorded to each contributing voter.
  • xx shows our inputs, i.e., voters.
  • bb is called bias — it's akin to an incumbent's advantage, a prior tendency towards a particular party or candidate.
  • ff is an activation function — this takes in the sum of all votes and outputs the election result.
Initializing Perceptrons

Let's kickstart our algorithm by setting the stage in our Python perceptron class with the __init__ method.

1def __init__(self, no_of_inputs, max_iterations=100, learning_rate=0.01): 2 self.max_iterations = max_iterations 3 self.learning_rate = learning_rate 4 self.weights = np.zeros(no_of_inputs + 1)


no_of_inputs refers to the number of inputs.

max_iterations is the maximum number of iterations the model will hold.

learning_rate indicates how fast we want the weights to adapt or learn based on the outcomes.

The initialized weights are set to zero, with an additional weight for the incumbent known as bias.

Perceptron Predict Method

Let's see how our system decides using the predict method.

1def predict(self, inputs): 2 summation = np.dot(inputs, self.weights[1:]) + self.weights[0] 3 if summation > 0: 4 activation = 1 5 else: 6 activation = 0 7 return activation

First, we calculate the sum of the weighted votes, akin to election results, by taking a dot product between the inputs and weights (excluding the incumbent bias). The incumbent bias is added to this sum. Finally, the activation function (step function in this case) is used to determine the election result.

Perceptron Training Function

Let's discuss how to train our model to yield better results in the train method of perceptrons.

1def train(self, training_inputs, labels): 2 for _ in range(self.max_iterations): 3 for inputs, label in zip(training_inputs, labels): 4 prediction = self.predict(inputs) 5 self.weights[1:] += self.learning_rate * (label - prediction) * inputs 6 self.weights[0] += self.learning_rate * (label - prediction)

This training process is similar to refining policies over multiple iterations based on previous prediction errors. Each iteration enables the system to learn from its mistakes and make better decisions in the future.

Applying the Perceptron Model

Let's integrate all we've learned so far and apply it to an example. We create an AND operator data. This data contains two inputs, and the output is them combined using an AND operator. It means 1 and 1 result in 1, but any other input combination results in 0.

1training_inputs = [] 2training_inputs.append(np.array([1, 1])) 3training_inputs.append(np.array([1, 0])) 4training_inputs.append(np.array([0, 1])) 5training_inputs.append(np.array([0, 0])) 6labels = np.array([1, 0, 0, 0]) 7 8perceptron = Perceptron(2) 9perceptron.train(training_inputs, labels)

After the training using AND operator data, our model is now ready to predict the results for new inputs.

1inputs = np.array([1, 1]) 2print(perceptron.predict(inputs)) # Output: 1 3 4inputs = np.array([0, 1]) 5print(perceptron.predict(inputs)) # Output: 0

Hurrah! Our model successfully passed the test, correctly predicting the results.

Lesson Summary and Practice

Congratulations! You've successfully transitioned from understanding to designing a perceptron using Python. Practicing is instrumental in solidifying this knowledge, and our upcoming exercises provide the perfect launching pad. Continue to explore machine learning spaces and enjoy your journey!

Enjoy this lesson? Now it's time to practice with Cosmo!

Practice is how you turn knowledge into actual skills.