Foundations & Models
Deep Learning
“Deep” refers to the many layers of a neural network, which work out increasingly abstract features one after another — edges from pixels, shapes from edges, faces from shapes. Only this depth, combined with large amounts of data and fast chips, made the leaps of recent years possible. The price: what exactly happens between input and output is hard to trace.
An example
Until about 2012, image recognition was built by having people tell the program what to look for: edges, corners, color gradients, typical shapes. That was laborious and worked only moderately well. Then it turned out that a many-layered network solves the same task better if you simply show it enough images and let it find the features itself. The lowest layers learn edges, the middle ones shapes, the top ones whole objects — without anyone ever having specified it that way.
A closer look
Many-layered means the input passes through many stages connected in sequence. Each builds on the features of the previous one, from edges through shapes to objects. The advantage is that these intermediate stages do not have to be designed by hand but emerge during training.
Why it matters
Deep learning explains why AI progress for a good ten years has depended on compute and data volume rather than on cleverer rules. For an investment decision that means: the effort no longer lies in formulating domain knowledge but in sourcing examples and paying for compute time. And it means that the result can, in principle, no longer be read in detail. Anyone who needs a justification in the sense of a traceable rule — in medicine, in lending, in public administration — has to produce it after the fact. It does not come for free.
Commonly misunderstood
Depth is not a seal of quality. More layers only help if there is enough data and compute. Below that, a simple method is often better and much easier to check.