The artificial neuron

Why start with one neuron?

Deep learning powers face unlock, speech-to-text in Indian languages, photo search and large language models. Under all of them is one small building block repeated millions of times: the artificial neuron. If you understand one neuron, the rest is about connecting many of them and training them together.

What a neuron computes

1. Take the inputs x₁, x₂, … (for example, a loan applicant's income and existing debt, already scaled).
2. Multiply each input by its weight w₁, w₂, …. A weight says how important that input is, and whether it pushes the output up (positive) or down (negative).
3. Add them up and add a bias b: z = w₁x₁ + w₂x₂ + … + b. The bias shifts the threshold, like a starting score.
4. Pass z through an activation function to get the output. The step function outputs 1 if z > 0 and 0 otherwise; the sigmoid outputs a value between 0 and 1.

def neuron(inputs, weights, bias):
    z = sum(x * w for x, w in zip(inputs, weights)) + bias
    return 1 if z > 0 else 0

print(neuron([1, 1], [0.6, 0.6], -1.0), neuron([1, 0], [0.6, 0.6], -1.0))

Inspired by the brain, but not a brain

The idea came from biological neurons, which fire when enough signal arrives from their neighbours. Artificial neurons are a very loose, simplified mathematical version. Modern deep learning has little to do with real brain biology; think of it as a flexible maths function, not a digital brain.

The perceptron learning rule

The perceptron (Frank Rosenblatt, 1958) learns from labelled examples. Start with weights of zero. For each example, predict; if the prediction is wrong, adjust: w = w + learning_rate × (target − prediction) × x, and b = b + learning_rate × (target − prediction). If the prediction was too low (target 1, predicted 0), the weights of the active inputs go up; if too high, they go down. Repeat over the data for several epochs.

The XOR problem

A single neuron can only separate classes with one straight line (a hyperplane in more dimensions). AND and OR can be separated that way. XOR ("exactly one of the two is 1") cannot: no single line puts (0,1) and (1,0) on one side and (0,0) and (1,1) on the other. This limitation, highlighted by Minsky and Papert in 1969, was solved by stacking neurons into layers with non-linear activations, which is the next topic.

Common mistakes

• Forgetting the bias. Without it, the dividing line must pass through the origin, which limits what the neuron can learn.
• Feeding unscaled inputs (₹ incomes beside ages), which makes learning slow and unstable.
• Thinking more epochs always fix a perceptron. If the data is not linearly separable, it never settles.