Alexey Ivakhnenko

From Wikipedia, the free encyclopedia
  (Redirected from Alexey Grigorevich Ivakhnenko)
Jump to navigation Jump to search
Oleksiy Ivakhnenko
Alexey Ivakhnenko.gif
Born
Oleksiy Hryhorovych Ivakhnenko

(1913-03-30)March 30, 1913
Kobelyaky, Poltava Governorate, Russian Empire
DiedOctober 16, 2007(2007-10-16) (aged 94)
Kyiv, Ukraine
ResidenceUSSR, Ukraine
NationalityUkrainian
Alma materLeningrad Electrotechnical Institute (M.Sc)
Known forGroup Method of Data Handling,
Deep Learning,
Inductive Modelling
AwardsTwo State Prizes of the Ukraine
Zaslnauki.jpg Order friendship of peoples rib.png DistinguishedLabourRibbon.png RibbonLabourDuringWar.png
Scientific career
FieldsArtificial Intelligence, Computer Science, Machine learning
InstitutionsGlushkov Institute of Cyberneticsuk,
Kyiv Polytechnic Institute (D.Sc)
ThesisTheory of Combined Automatic Control of Electric Motors (1944)
Doctoral studentsV. M. Kuntsevich, V. I. Kostiuk
V. I. Ivanenko, V. I. Vasylyev
O. A. Pavlov

Oleksiy Ivakhnenko (Ukrainian: Олексíй Григо́рович Іва́хненко, Russian: Алексей Григо́рьевич Іва́хненко); (30 March 1913 – 16 October 2007) was a Soviet and Ukrainian mathematician most famous for developing the Group Method of Data Handling (GMDH), a method of inductive statistical learning, for which he is sometimes referred to as the "Father of Deep Learning".[1]

Early life and Education[edit]

Oleksiy was born in Kobelyaky, Poltava Governorate in a family of teachers.[2] In 1932 he graduated from Electrotechnical college in Kyiv and worked for two years as an engineer on the construction of large power plant in Berezniki. Then in 1938 after graduation from the Leningrad Electrotechnical Institute, he worked in the All-Union Electrotechnical Institute in Moscow during wartime. There Ivakhnenko investigated the problems of automatic control in the laboratory led by Sergey Lebedev and then, continued research in other institutions in Ukraine after his return to Kyiv in 1944. In that year he received the Ph.D. degree and later in 1954 had received D.Sc. degree. In 1964 he was appointed as a Head of the Department of Combined Control Systems at the Institute of Cybernetics. Simultaneously working at first as a Lecturer, and from 1961, as a Professor of Automatic Control and Technical Cybernetics at the Kyiv Polytechnic Institute.

Research[edit]

Ivakhnenko is known to be the founder of Inductive modelling, a scientific approach used for pattern recognition and complex systems forecasting.[3] He had used this approach during development of the Group Method of Data Handling (GMDH). In 1968 the journal "Avtomatika" had published his article "Group Method of Data Handling - a rival of the method of stochastic approximation",[4] marking the beginning of a new stage in his scientific work. He led the development of this approach, with a professional team of mathematicians and engineers at the Institute of Cybernetics.

Group Method of Data Handling[edit]

The GMDH method presents a unique approach to the problem of modelling and even a new philosophy to scientific research, which is possible using modern computers.[3] A researcher may not adhere precisely to traditional deductive way of building models "from general theory - to a particular model": monitoring an object, studying its internal structure, understanding the physical principles of its operation, developing theory and testing the mathematical model of an object. Instead, the new approach is proposed "from specified data - to a general model": after the input of data, a researcher selects a class of models, the type of models-variants generation and sets the criterion for model selection. As most routine work is transferred to a computer, the impact of human influence on the objective result is minimised. In fact, this approach can be considered as one of the implementations of the Artificial Intelligence thesis, which states that a computer can act as powerful advisor to humans.

The development of GMDH consists of a synthesis of ideas from different areas of science: the cybernetic concept of "black box" and the principle of successive genetic selection of pairwise features, Godel's incompleteness theorems and the Gabor's principle of "freedom of decisions choice",[5] the Adhémar's incorrectness and the Beer's principle of external additions.[6]

GMDH is the original method for solving problems for structural-parametric identification of models for experimental data under uncertainty.[7] Such a problem occurs in the construction of a mathematical model that approximates the unknown pattern of investigated object or process.[8] It uses information about it that is implicitly contained in data. GMDH differs from other methods of modelling by the active application of the following principles: automatic models generation, inconclusive decisions, and consistent selection by external criteria for finding models of optimal complexity. It had an original multilayered procedure for automatic models structure generation, which imitates the process of biological selection with consideration of pairwise successive features. Such procedure is currently used in Deep learning networks.[9] To compare and choose optimal models, two or more subsets of a data sample are used. This makes it possible to avoid preliminary assumptions, because sample division implicitly acknowledges different types of uncertainty during the automatic construction of the optimal model.

In the early 1980s Ivakhnenko had established an organic analogy between the problem of constructing models for noisy data and signal passing through the channel with noise.[10] This made possible to lay the foundations of the theory of noise-immune modelling.[7] The main result of this theory is that the complexity of optimal predictive model depends on the level of uncertainty in the data: the higher this level (e.g. due to noise) - the simpler must be the optimal model (with less estimated parameters). This initiated the development of the GMDH theory as an inductive method of automatic adaptation of optimal model complexity to the level of information in fuzzy data. Therefore, GMDH is often considered to be the original information technology for knowledge extraction from experimental data.

Results[edit]

Alongside to GMDH, Ivakhnenko had developed the following set of results:

  • New principles of automatic speed control for AC and asynchronous electric motors.[11]
  • Theory of invariant systems for automatic control with compensation of measured disturbances.[12] He had developed the principle of indirect measurement of disturbances, called as "differential fork" that was used later in practice.
  • Principle of combined control (with negative feedback for the controlled variables and positive feedback for the controlled disturbances).[13] A number of such systems, for the speed control of electric motors had been implemented in practice. That proved the practical feasibility of invariant conditions in a combined control systems that unite the advantages of closed systems for control by deviation (high precision) and open systems (performance).
  • The non-searching extreme regulators on the basis of situations recognition.[14]
  • Principle of self-learning pattern recognition. It was demonstrated at first in the identification system "Alpha",[15] created under his leadership.
  • Basis for the construction of cybernetic prediction devices.[14]
  • Theory of models self-organization according to experimental data.[16]
  • Method of control with forecast optimization.[17]
  • Noise-immune principles of modelling for fuzzy data.[7]
  • Principle of construction of self-organizing polynomial neural networks.[3]
  • Principle of multilayered neural networks construction with active neurons. Each neuron in it can be an inductive algorithm.

Ivakhnenko had developed new principles and devices for the automatic control of speed in electric motors and the calculation of systems with magnetic amplifiers. Well known are his achievements in the theory of invariance and theory of combined automatic control systems, that operates on the principle of measured disturbances compensation.

He is the author of the first Ukrainian monograph on technical cybernetics,[14] which was published worldwide in seven languages.[18] In his study, a further development of the principles of combined control was connected with the implementation of methods and devices of self-tuning and self-organisation, pattern recognition and forecasting in control systems.

In recent years, his main innovation - the GMDH method was developed as a method of inductive modelling, complex processes and systems forecasting. His ideas are utilised now in the Deep Learning networks.[19] The effectiveness of the method was confirmed repeatedly during the solution of real complex problems in ecology, meteorology, economics and technology, which aided increase its popularity among the international scientific community.[20] In parallel, there were conducted developments of self-organisation algorithms in a related field - problems of pattern recognition.[21] Advances in the modelling of environmental processes reflected in the monographs,[22][10] economic processes - in the books [17] .[23] The results of exploration of multilayered recurrent GMDH algorithms are described in the books.[16]

Scientific school[edit]

From 1963 to 1989 Ivakhnenko was the editor of the specialized scientific journal "Avtomatika" (later "Problems of management and computer science"), that played a crucial role in the formation and development of the Ukrainian school of Inductive modelling. Throughout these years the magazine was translated and reprinted in the United States as "Soviet Automatic Control" (later "Journal of Automation and Information Sciences").

Alongside constant innovation in his field since 1945, Ivakhnenko maintained an active teaching career, at first as the Assistant Professor at the Department of Theoretical Mechanics, and then at the Control Systems faculty. Since 1960 as Professor of the Department of Technical Cybernetics, Kyiv Polytechnic Institute, where he contributed lectures to the University and student body, as well as oversaw the work of many graduate students. In 1958-1964 he was an organiser of the All-Union Conferences of Invariance in Kiev, where the development of the invariant control systems theory was restored after prohibition.[24]

His inexhaustible enthusiasm helped more than 220 young scientists to prepare and successfully defend their Ph.D. dissertations under his leadership in the KPI and the Institute of Cybernetics and nearly 30 of his students defended their post-doctoral dissertations. Scientific school of Ivakhnenko was and is a real cradle of highly qualified scientific professionals. Furthermore, his students V.M.Kuntsevych, V.I.Kostyuk, V.I.Ivanenko, V.I.Vasiliev, A.A.Pavlov and others had created their own respected scientific schools. Ivakhnenko was a shining example of a scientist, with a keen sense of new and remarkable scientific intuition. Until his last days, he continued to work actively and generously generated original scientific ideas and results.

Awards and Honours[edit]

Ivakhnenko is the Honorary Scientist of the USSR (1972), two-time winner of the State Prize of Ukraine (1991, 1997) for his works on the theory of invariant automatic systems and set of publications on Information technology in the field of Artificial intelligence. Author of 40 books and over 500 scientific articles. Honorary Doctor of National Technical University "KPI" (2003) and Lviv Polytechnic (2005). He was the Corresponding Member of Academy of Sciences USSR (1961) and Academician of NAS of Ukraine (2003).

Selected works[edit]

  • Ivakhnenko O.G. Heuristic Self-Organization in Problems of Engineering Cybernetics, Automatica, vol.6, 1970 — p. 207-219.
  • Ivakhnenko O.G. Polynomial Theory of Complex Systems, IEEE Transactions on Systems Man and Cybernetics, 4, 1971 — p. 364-378.
  • Ivakhnenko, O.G.; Ivakhnenko, G.A. (1995). "The Review of Problems Solvable by Algorithms of the Group Method of Data Handling (GMDH)". Pattern Recognition And Image Analysis (5): 527–535.

References[edit]

  1. ^ [Jurgen Schmidhuber. Critique of Paper by "Deep Learning Conspiracy" (Nature 521 p 436), June 2015. http://people.idsia.ch/~juergen/deep-learning-conspiracy.html]
  2. ^ Бобрищев, К.В. (2002). Отчий Край (in Ukrainian). Полтава: Дивосвіт. pp. 284–293. ISBN 966-95846-9-8.
  3. ^ a b c Madala, H.R.; Ivakhnenko, O.G. (1994). Inductive Learning Algorithms for Complex Systems Modeling. Boca Raton: CRC Press. ISBN 0849344387.
  4. ^ Ivakhnenko, O.G. (1968). "The Group Method of Data Handling - a Rival of the Method of Stochastic Approximation". Soviet Automatic Control. 13 (3): 43–55.
  5. ^ Gabor, D. (1971). Perspectives of Planing. Organization of Economic Cooperation and Development. London: Imp.Coll.
  6. ^ Beer, S. (1959). Cybernetics and Management. London: English Univ. Press.
  7. ^ a b c Ivakhnenko, O.G.; Stepashko, V.S. (1985). Pomekhoustojchivost' Modelirovanija (Noise Immunity of Modeling) (PDF). Kyiv: Naukova Dumka.
  8. ^ Ivakhnenko, O.G.; Lapa, V.G. (1967). Cybernetics and Forecasting Techniques (Modern Analytic and Computational Methods in Science and Mathematics, v.8 ed.). American Elsevier.
  9. ^ Takao, S.; Kondo, S.; Ueno, J.; Kondo, T. (2017). "Deep feedback GMDH-type neural network and its application to medical image analysis of MRI brain images". Artificial Life and Robotics: 1–12.
  10. ^ a b Ivahnenko, O.G. (1982). Inductive Method of Models Self-organisation for Complex Systems (PDF). Kyiv: Naukova Dumka.
  11. ^ Ivakhnenko, O.G. (1953). Automatic Control of Velocity of Asynchronous Motors with Moderate Power. Kyiv: Izd. AN USSR.
  12. ^ Ivakhnenko, O.G. (1954). Theory of Combined Automatic Control of Electric Motors. Kyiv: Izd.KPI.
  13. ^ Ivakhnenko, O.G. (1954). Electroautomatika. Kiev: Gostechizdat USSR.
  14. ^ a b c Ivakhnenko, O.G. (1959). Tekhnicheskaya Kibernetika. Kiev: Gostechizdat USSR.
  15. ^ Ivakhnenko, O.G. (1969). Self-learning systems of recognition and automatic control. Kyiv: Tehnika.
  16. ^ a b Ivakhnenko, O.G.; Yurachkovsky, Yu.P. (1986). Modelirovanie Slozhnykh System po Exsperimentalnym Dannym (Modelling of Complex Systems on Experimental Data). M:: Radio i Svyaz.
  17. ^ a b Ivakhnenko, O.G.; Müller, J.A. (1985). Self-Organisation of Forecasting Models (PDF). Kyiv: Tehnika.
  18. ^ Ivachnenko, O.G. (1962). Techniche kybernetik. Berlin: Verlag Technik.
  19. ^ Schmidhuber, J. (January 2015). "Deep Learning in Neural Networks: An Overview" (PDF). Neural Networks: 85–117. arXiv:1404.7828. doi:10.1016/j.neunet.2014.09.003.
  20. ^ Farlow, S.J. (ed.) (1984). Self-organizing Methods in Modeling: GMDH Type Algorithms (Statistics: Textbooks and Monographs, vol.54 ed.). Marcel Dekker Inc. ISBN 0824771613.CS1 maint: Extra text: authors list (link)
  21. ^ Ivahnenko, O.G.; Zaychenko, Yu.P.; Dimitrov, V.D. (1976). Decision Making on the basis of Self-Organisation. M:: Sov.Radio.
  22. ^ Ivahnenko, O.G. (1975). Long-term Forecasting and Complex Systems Control (PDF). Kyiv: Tehnika.
  23. ^ Ivachnenko, O.G.; Müller, J.A. (1984). Selbstorganisation von Vorhersagemodellen. Berlin: Veb Verlag Technik.
  24. ^ "Nauka i Promyshlennost` (Science and Industry)". Pravda. Communist Party of the USSR. 16 May 1941.

External links[edit]

Retrieved from "https://en.wikipedia.org/w/index.php?title=Alexey_Ivakhnenko&oldid=872114092"