Neural Networks tutorial

Overview

What is Neural Networks?

Neural Networks is a supervised or unsupervised learning technique. Learns layered nonlinear transformations through connected units. Its central idea is summarized by a⁽ˡ⁾ = σ(W⁽ˡ⁾a⁽ˡ⁻¹⁾ + b⁽ˡ⁾).

This guide connects theory to practice. You will trace the input, intermediate process, and output; decode the notation; run a dependency-light implementation; then repeat the workflow with PyTorch and evaluate the result.

Preparation

Prerequisites and tools

You do not need an advanced software stack. Start with a recent Python environment and the fundamentals below, then install only the packages used by the practical example.

• Comfort with Python functions, NumPy arrays, and basic descriptive statistics.
• A clear distinction between training data, validation data, and untouched test data.
• Familiarity with features, targets, preprocessing, and task-appropriate evaluation metrics.

Process

How Neural Networks works

1. 1

   Input

   Feature vectors, images, sequences, or embeddings.

2. 2

   Prepare and configure

   Check shapes, value ranges, ordering assumptions, missing values, and the parameters that control the algorithm’s behavior.

3. 3

   Algorithm process

   Forward-propagate activations and backpropagate errors.

4. 4

   Output

   Probabilities, values, embeddings, or generated signals.

5. 5

   Validate

   Keep a validation split and plot both training and validation loss by epoch.

Core concept

Formula and intuition

The notation captures the main operation or complexity statement behind Neural Networks. Read it together with the symbol key above and the step-by-step process in this guide.

Applications

When to use Neural Networks

Vision, language, speech, and complex nonlinear tasks.

Implementation

Neural Networks from scratch in Python

This dependency-light example emphasizes the algorithm’s mechanics so each important step remains visible.

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# XOR-style pattern: the label is 1 only when both features share the same sign
rng = np.random.default_rng(0)
X = rng.normal(0, 1, (200, 2))
y = (X[:, 0] * X[:, 1] > 0).astype(float).reshape(-1, 1)

# one hidden layer of 4 units, trained with plain backpropagation
W1 = rng.normal(0, 1, (2, 4))
b1 = np.zeros(4)
W2 = rng.normal(0, 1, (4, 1))
b2 = np.zeros(1)
lr = 0.1

for _ in range(2000):
    hidden = sigmoid(X @ W1 + b1)       # forward pass: input layer to hidden layer
    output = sigmoid(hidden @ W2 + b2)  # forward pass: hidden layer to output

    d_output = (output - y) * output * (1 - output)      # error term at the output layer
    d_hidden = (d_output @ W2.T) * hidden * (1 - hidden)  # error term propagated back to the hidden layer

    W2 -= lr * hidden.T @ d_output / len(y)  # update weights and biases using their own error terms
    b2 -= lr * d_output.mean(axis=0)
    W1 -= lr * X.T @ d_hidden / len(y)
    b1 -= lr * d_hidden.mean(axis=0)

Practical tutorial

Build Neural Networks with PyTorch

PyTorch provides tensors, automatic differentiation, modules, optimizers, and accelerators in one widely used deep-learning workflow.

1. Prepare the data.Feature vectors, images, sequences, or embeddings. Validate its shape, type, range, and ordering before training or execution.
2. Configure the algorithm.Begin with explicit, conservative parameters and a fixed random seed whenever the library supports one.
3. Fit or execute.Forward-propagate activations and backpropagate errors.
4. Inspect the result.Probabilities, values, embeddings, or generated signals. Then apply the evaluation checks in the next section.

import torch
from torch import nn

torch.manual_seed(7)
X = torch.randn(800, 2)
y = ((X[:, 0] * X[:, 1]) > 0).float().unsqueeze(1)
model = nn.Sequential(
    nn.Linear(2, 16), nn.ReLU(),
    nn.Linear(16, 8), nn.ReLU(),
    nn.Linear(8, 1),
)
loss_fn = nn.BCEWithLogitsLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.02)

for _ in range(400):
    optimizer.zero_grad()
    loss = loss_fn(model(X), y)
    loss.backward()
    optimizer.step()

with torch.no_grad():
    prediction = (model(X).sigmoid() >= 0.5).float()
    print("accuracy:", (prediction == y).float().mean().item())

API details and version-specific options: official PyTorch reference →

Evaluation

How to evaluate the result

A successful run is not enough. Evaluate the output against the intended use, compare it with a simple baseline, and preserve a genuinely unseen test case whenever the task involves learned parameters.

• Keep a validation split and plot both training and validation loss by epoch.
• Inspect class-specific metrics and calibration for probabilistic outputs.

Common mistakes

Pitfalls and how to avoid them

These failure modes are common in tutorials and production systems. Treat them as review questions, not just after-the-fact debugging advice.

• Calling model outputs probabilities before applying the correct activation leads to mistakes.
• Training and evaluation modes differ for dropout and batch normalization.
• A larger network can memorize data without learning a generalizable pattern.

Project checklist

Before using Neural Networks in a project

• ✓Define the prediction or discovery objective before selecting the algorithm.
• ✓Split data before fitting preprocessing and tune only inside cross-validation.
• ✓Record data versions, random seeds, features, hyperparameters, and evaluation metrics.
• ✓Compare against a simple baseline and inspect errors by meaningful subgroups.
• ✓Monitor input drift and real-world performance after deployment.

Further reading

Official documentation and next steps

This guide is an educational introduction, not a substitute for domain validation. For consequential applications, review the source documentation, test against representative data, and involve a subject-matter expert.

Continue learning

Related machine learning guides

• Linear RegressionFits a straight line that minimizes squared prediction errors.
• Logistic RegressionMaps a weighted feature score to a class probability.
• Decision TreeSplits the feature space into rule-based regions.