What are neural networks?

Definition

What is a neural network?

A neural network is a computational model that imitates the structure of the human brain and recognises patterns in data on its own. It is one of the most important methods of artificial intelligence.

Biological model

In the human brain, around 86 billion neurons communicate via electrical and chemical signals. Each neuron receives inputs from other cells, processes them and passes the result on.

Artificial neural networks adopt this principle in simplified form. They consist of digital nodes that receive numerical values, weight them and pass them on to the next layer.

Artificial neurons

An artificial neuron receives numerical values as input, multiplies them by weights and adds a threshold value (bias). An activation function then decides whether the neuron passes on a signal.

From the simple perceptron to the modern transformer – all neural networks build on this computational principle.

Structure

How are neural networks structured?

A neural network consists of an input layer, one or more hidden layers and an output layer. The more layers a network has, the more complex the relationships it can learn.

Input layer

The input layer receives the raw data – such as pixel values of an image or words of a text. Each neuron in this layer represents a single feature of the input.

Hidden layers

In the hidden layers, the actual computation takes place. Each layer transforms the data and recognises increasingly abstract patterns.

An image recognition network recognises edges in the first layer, shapes in the second and objects in the third. The level of abstraction rises with each layer.

Output layer

The output layer delivers the result – such as a probability for a certain category. In spam detection, it indicates whether an email is spam or not.

Weights and bias

Weights control how strongly an input value influences a neuron. The bias shifts the threshold above which a neuron becomes active.

During training, the network adjusts both values step by step. That is how it learns which inputs are relevant for the prediction.

How it works

How do neural networks work?

Neural networks learn by processing data forward through their layers, measuring the error and adjusting the weights via backpropagation.

Forward propagation

In forward propagation, the input data flows through the network layer by layer. Each neuron calculates a weighted sum of its inputs and applies an activation function.

At the end there is a prediction – such as “this image shows a cat with 92% probability”.

Activation functions

Activation functions make neural networks powerful. Without them, a network could only model linear relationships – that is, straight lines.

ReLU (rectified linear unit) is the most commonly used activation function today. It passes positive values on unchanged and sets negative ones to zero.

Backpropagation

After each prediction, the network compares its result with the correct answer. The difference is the error.

. Backpropagation calculates the error back layer by layer and determines how much each weight contributed to it. An optimisation algorithm such as gradient descent then adjusts the weights so that the error becomes smaller.

Backpropagation and gradient descent together form the learning mechanism that makes neural networks a central method in machine learning.

Types

What types of neural networks are there?

Depending on the task, different architectures are used – from simple feedforward networks and CNNs to transformers.

Feedforward networks

In a feedforward network, data flows in only one direction – from input to output. There is no feedback between the layers.

The simplest feedforward network is the perceptron. It consists of a single layer and can only perform linear separations.

Convolutional neural networks (CNNs)

CNNs recognise spatial patterns in images. Filters scan the image in small sections and identify edges, textures and shapes.

They form the basis for facial recognition, medical image analysis and autonomous driving.

Recurrent neural networks (RNNs)

RNNs process data in sequences and take previous steps into account. That makes them suitable for time series, language and music.

A further development is the LSTM network (long short-term memory), which can also store information from far back.

Transformers

Transformers process all inputs in parallel rather than one after another. Their attention mechanism weights which parts of the input are relevant for the output.

GPT, Claude and Gemini are based on the transformer architecture. It forms the basis of today's large language models.

Practice

Where are neural networks used?

Neural networks are found in voice assistants, search engines, medical diagnoses and self-driving cars.

Image recognition and computer vision

CNNs analyse X-ray images, recognise faces in photos and sort products on conveyor belts. Smartphones use them for camera optimisation in real time.

Language processing and chatbots

Transformer models translate texts, answer questions and write code. ChatGPT, Claude and Google Gemini are examples of generative AI based on neural networks.

Medicine and research

Neural networks detect tumours in CT scans with an accuracy that matches that of experienced radiologists. In the pharmaceutical industry, they speed up the search for new active substances.

Autonomous driving

Self-driving cars combine CNNs for image recognition with RNNs for predicting movement patterns. Tesla, Waymo and other manufacturers use neural networks for control and navigation.

Context

What distinguishes neural networks from AI and machine learning?

Neural networks are a method within machine learning, which in turn is a subfield of artificial intelligence.

AI as the umbrella term

Artificial intelligence describes all systems that solve tasks which normally require human thinking. These include rule-based systems as well as learning algorithms.

Machine learning as a subfield

Machine learning is a subfield of AI in which algorithms learn from data instead of following fixed rules. Neural networks are one of many ML methods – alongside decision trees, support vector machines or k-nearest neighbours.

Deep learning and neural networks

Deep learning refers to neural networks with more than two hidden layers. The term emphasises the depth of the network – that is, the number of layers.

Not every neural network is deep learning. A simple perceptron with one layer does not fall into this category.

FAQ

Frequently asked questions about neural networks