MediumVapnik and Chervonenkis: The Founding Fathers of Machine Learning | by Valeriy Manokhin, PhD, MBA, CQF | Jul, 2026 | MediumSitemapOpen in appSign up<br>Sign in
Medium Logo
Get app<br>Write
Search
Sign up<br>Sign in
Vapnik and Chervonenkis: The Founding Fathers of Machine Learning
Valeriy Manokhin, PhD, MBA, CQF
4 min read·<br>3 days ago
Listen
Share
How two Soviet mathematicians laid the theoretical foundation of machine learning long before it became a global phenomenon.<br>In today’s AI-driven world, we casually speak of generalization, overfitting, and VC dimensions — as if these ideas were always part of the machine learning landscape. But their origins are remarkably precise, and their creators remarkably overlooked.<br>Meet Vladimir Vapnik and Alexey Chervonenkis , two Soviet mathematicians who built the mathematical foundation of machine learning — at a time when “AI” was mostly philosophical and “big data” didn’t exist.
🎓 Who Were They?<br>Vladimir Vapnik<br>Born : 1936, Soviet Union<br>Education :<br>Undergraduate : Mathematics and Physics, Uzbek State University (Tashkent)<br>PhD : Institute of Control Sciences, USSR Academy of Sciences<br>Discipline : Mathematical statistics, information theory, pattern recognition<br>Profession :<br>Worked at the Institute of Control Sciences in Moscow (a major Soviet research hub in applied math and engineering)<br>Later joined AT&T Bell Labs in the U.S. (1990s), where he further developed Support Vector Machines<br>Despite his later association with statistical theory, Vapnik was never trained or employed as a statistician . His work was grounded in mathematics, physics, and control theory — disciplines concerned with information, systems, and optimization.
Alexey Chervonenkis<br>Born : 1938, Soviet Union<br>Education :<br>Undergraduate and graduate degrees in mathematics, Moscow Institute of Physics and Technology (department of Radiotechnics and Cybernetics)<br>Discipline : Probability theory, mathematical modeling, pattern recognition<br>Profession :Researcher at the Institute of Control Sciences , where he began his long collaboration with Vapnik. Later taught at Moscow Institute of Physics and Technology (MIPT)<br>Chervonenkis was a quiet but profound thinker. He was never part of the Soviet statistics community; his orientation was always in theoretical computer science and applied mathematics .
🧠 What They Did — and Why It Matters<br>In 1964 , the two researchers published their breakthrough paper introducing VC theory (Vapnik–Chervonenkis theory). This work asked — and answered — one of the most fundamental questions in learning:<br>How can we know if a machine will perform well not just on the training data, but on unseen examples?
They formalized:<br>Generalization bounds<br>Capacity control (via VC dimension)<br>Uniform convergence<br>Empirical risk minimization (ERM)<br>This wasn’t just a mathematical curiosity. These tools became the core principles behind supervised learning — they explained how and why learning from data could work at all.<br>In essence, Vapnik and Chervonenkis gave machine learning its first rigorous statistical soul — without coming from statistics themselves.
❌ Not Statisticians — And That’s Important<br>It’s a historical misunderstanding to attribute the birth of machine learning to traditional statisticians.<br>Vapnik and Chervonenkis did not publish in mainstream statistical journals.<br>They weren’t interested in estimation or hypothesis testing in the classical sense.<br>They worked on learning theory : a new field, with its own rules, concerns, and mathematics.<br>In fact, many in the Western statistical community ignored or misunderstood their work for decades. It was only when Vapnik moved to the U.S. and helped develop non linear version of Support Vector Machines in the 1990s that his contributions began to be widely recognized. The original liner version of Support Vector Machines we already developed by Vapnik long before that in the early 1960s in the USSR, it was known under the name of ‘The Methods of Generalised Portraits.’ Thirty years later, whilst working in Bell Labs in the USA Vapnik and Cortes combined his linear Support Vector Machine method with kernel methods (invented by Aizerman and his colleagues like Rozonoer from the Aizerman’s Lab in late 1950s-early 1960 in the same institute that Vapnik has worked in the USSR)
🇺🇸🇷🇺 Two Nations, One Legacy<br>Machine learning, as a field, has only two true birthplaces : the United States and the Soviet Union .<br>The U.S. gave us early systems like Arthur Samuel’s checkers-playing program (1959) and Frank Rosenblatt’s Perceptron (1958).<br>The USSR gave us the mathematical foundations through Vapnik and Chervonenkis, beginning in the early 1960s.<br>Every country since has contributed to scaling, refining, and applying machine learning — but only the USA and USSR can claim to have founded it.
📘 A Legacy That Shaped the Future<br>When Vapnik published The Nature of Statistical Learning Theory in 1995 , it was more than a summary of his research. It...