Last reviewed:
What is deep learning? Definition and difference from machine learning
Deep learning is a form of machine learning that uses artificial neural networks made up of many layers, each processing data at a higher level of abstraction. It drives the progress in image recognition, speech recognition, translation and generative AI.
France's official definition (Journal officiel, December 9, 2018) describes it as machine learning that uses an artificial neural network “made up of a large number of layers”, each corresponding to an increasing level of complexity. For an image, the first layers detect edges, the next ones shapes, the last ones objects. The practical difference from classic machine learning lies in data preparation: the network extracts useful features from raw data itself, whereas a classic model expects variables chosen by a human. In return, it needs a lot of data and computing power. The ideas are old: backpropagation, the algorithm that trains multi-layer networks, was popularised in 1986 by Rumelhart, Hinton and Williams. The turning point came in 2012. AlexNet, a 60-million-parameter convolutional network designed by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet competition with a 15.3% top-5 error rate, against 26.2% for the runner-up. It was trained on two consumer graphics cards: GPUs became the field's standard tool. Two awards have since recognised the discipline. The 2018 Turing Award went to Yoshua Bengio, Geoffrey Hinton and Yann LeCun for making deep neural networks “a critical component of computing”. The 2024 Nobel Prize in Physics, announced on October 8, 2024, went to John Hopfield and Geoffrey Hinton for discoveries that enable machine learning with artificial neural networks. Large language models (LLMs) are deep learning models.
Concrete example
Illustrative case: a French injection-moulded plastic parts manufacturer with 200 employees visually inspects parts as they leave the press. Defects (bubbles, flash, scratches) vary and are hard to describe with rules. The company installs cameras and has its quality inspectors label 20,000 photos: compliant or not, with the defect type. A convolutional network, pre-trained on generic images then adapted to these photos, sorts parts in real time. Inspectors now only check the doubtful cases flagged by the system. Classic machine vision rules, tested earlier, triggered too many false alarms whenever the lighting changed.
Comparison
| Year | Event | Why it matters |
|---|---|---|
| 1986 | Rumelhart, Hinton and Williams demonstrate training by backpropagation | Makes it possible to train multi-layer networks |
| Late 1980s | Yann LeCun develops convolutional networks (CNNs) | Used from the mid-1990s to read bank cheques |
| 2012 | AlexNet wins ImageNet (15.3% error vs 26.2%) | Start of the deep learning boom, trained on GPUs |
| 2017 | The transformer architecture is published | Basis of today's large language models |
| 2018 | Turing Award to Bengio, Hinton and LeCun | Recognition by computer science |
| 2024 | Nobel Prize in Physics to Hopfield and Hinton | Recognition beyond computer science |
FAQ
What is deep learning, in simple terms?
It is an AI technique that learns from examples using an artificial neural network with many layers. Each layer transforms the data a little more: in a photo, the first layers detect edges, the last ones recognise a face or an object.
What is the difference between deep learning and machine learning?
Deep learning is part of machine learning. Classic machine learning works on variables prepared by a human (age, amount, date). Deep learning extracts what matters from raw data (pixels, sounds, words) on its own, at the cost of far more data and computing power.
What are examples of deep learning?
Facial recognition on phones, speech transcription, machine translation, automated document reading, camera-based quality control, support for medical imaging diagnosis, and all generative AI assistants such as ChatGPT or Claude.
Why did deep learning take off in 2012?
Three ingredients came together: large labelled image datasets (ImageNet), the power of graphics cards (GPUs) and training improvements. In 2012, the AlexNet network won the ImageNet competition with a 15.3% error rate, against 26.2% for the next competitor.
Who are the pioneers of deep learning?
Yoshua Bengio, Geoffrey Hinton and Yann LeCun, winners of the 2018 Turing Award. Frenchman Yann LeCun developed convolutional networks in the 1980s. Geoffrey Hinton received the 2024 Nobel Prize in Physics with John Hopfield.
Is ChatGPT deep learning?
Yes. ChatGPT, Claude and Mistral rely on large language models, which are deep neural networks with a transformer architecture and billions of parameters. Generative AI is an application of deep learning.
See also
Further reading
Sources
- ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky, Sutskever and Hinton, NIPS 2012. https://proceedings.neurips.cc/paper_files/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
- Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award (2018 Turing Award), ACM. https://awards.acm.org/about/2018-turing
- The Nobel Prize in Physics 2024, press release, Royal Swedish Academy of Sciences, October 8, 2024. https://www.nobelprize.org/prizes/physics/2024/press-release/
- Vocabulaire de l'intelligence artificielle, official French definition of “apprentissage profond”, Journal officiel No. 0285, December 9, 2018. https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000037783813