Neurons, like other types of cells, include cell membrane, cytoplasm and nucleus. However, nerve cells have special morphology and many processes, so they are divided into three parts: cell body, axon and dendrite. There are nuclei in cells, and the function of processes is to transmit information. Dendrites are protrusions that introduce input signals, while axons are protrusions that act as output terminals. There is only one dendrite.
Dendrites are extensions of cell bodies, which gradually become thinner after being emitted from the cell bodies. All parts of the whole length can be connected with axonal endings of other neurons to form so-called "synapses". At the synapse, two neurons are not connected, but only the connection point where information transmission takes place. The gap between contact interfaces is about (15 ~ 50) × 10 meter. Synapses can be divided into excitatory and inhibitory types, corresponding to the polarity of coupling between neurons. The number of synapses in each neuron is normal, with a maximum of 10. The connection strength and polarity between neurons are different and can be adjusted. Based on this characteristic, the human brain has the function of storing information. An artificial neural network composed of a large number of interconnected neurons can show some characteristics of the human brain.
Artificial neural network is an adaptive nonlinear dynamic system composed of a large number of simple basic elements-neurons. The structure and function of each neuron are simple, but the system behavior produced by a large number of neurons is very complicated.
Artificial neural network reflects some basic characteristics of human brain function, but it is not a realistic description of biological system, but an imitation, simplification and abstraction.
Compared with digital computer, the composition principle and functional characteristics of artificial neural network are closer to the human brain. It does not perform operations step by step according to the given program, but can adapt itself to the environment, summarize the laws and complete some operations, identification or process control.
Artificial neural network must learn according to certain learning criteria before it can work. Taking the recognition of letters "A" and "B" written by artificial neural network as an example, it is stipulated that "A" outputs "1" and "B" outputs "0".
Therefore, the rule of online learning should be: if the network makes a wrong judgment, the network should reduce the possibility of making the same mistake next time through online learning. Firstly, each connection weight of the network is given a random value in the interval of (0, 1), and the image pattern corresponding to "a" is input into the network. The network adds the input modes by weight, compares them with the threshold, and then performs nonlinear operation to get the output of the network. In this case, the probability that the network output is "1" and "0" is 50% respectively, which means it is completely random. At this time, if the output is "1" (the result is correct), the connection weight is increased, so that the network can still make a correct judgment when it encounters the "A" mode input again.
If the output is "0" (that is, the result is wrong), the network connection weight is adjusted in the direction of reducing the comprehensive input weight, so as to reduce the possibility that the network will make the same mistake next time it encounters the "A" mode input. With this operation adjustment, when several handwritten letters "A" and "B" are input into the network in turn, the correct rate of network judgment will be greatly improved after learning several times through the network according to the above learning method. This shows that the network has successfully learned these two modes and memorized them in every connection weight of the network. When the network encounters any of these modes again, it can make a quick and accurate judgment and identification. Generally speaking, the more neurons a network contains, the more patterns it can remember and recognize. (1) The human brain has strong adaptability and self-organization characteristics, and the acquired learning and training can develop many unique activity functions. For example, the hearing and touch of the blind are very sensitive; Deaf people are good at using gestures; Well-trained athletes can show extraordinary sports skills and so on.
The function of an ordinary computer depends on the knowledge and ability given in the program. Obviously, it will be very difficult to plan intelligent activities through summarization.
Artificial neural network also has preliminary adaptive and self-organizing ability. Change the synaptic weight in the process of learning or training to meet the requirements of the surrounding environment. The same network can have different functions due to different learning methods and contents. Artificial neural network is a system with learning ability, which can develop knowledge beyond the original knowledge level of designers. Usually, its learning and training methods can be divided into two types. One is supervised or supervised learning, in which given sample criteria are used for classification or imitation. The other is unsupervised learning or unsupervised tutor learning. At this time, only the learning method or some rules are specified, and the specific learning content changes with the environment in which the system is located (that is, the input signal situation). The system can automatically discover environmental characteristics and laws, which is more similar to the function of the human brain.
(2) Generalization ability
Generalization ability refers to the ability to predict and control untrained samples. Especially when there are some noise samples, the network has good prediction ability.
(3) Nonlinear mapping ability
When the system is very thorough or clear to designers, mathematical tools such as numerical analysis and partial differential equations are generally used to establish accurate mathematical models. However, when the system is complex, unknown or with little information, it is difficult to establish an accurate mathematical model. The nonlinear mapping ability of neural network shows advantages, because it does not need to have a thorough understanding of the system, but at the same time it can realize the mapping relationship between input and output, which greatly simplifies the design difficulty.
(4) High parallelism
Parallelism is controversial. Reasons for admitting parallelism: Neural network is a mathematical model abstracted from human brain. Since people can do one thing at the same time, from the perspective of functional simulation, neural network should also have strong parallelism.
Over the years, people have tried to understand and answer the above questions from the perspectives of medicine, biology, physiology, philosophy, informatics, computer science, cognition and organizational synergetics. In the process of finding the answers to the above questions, a new interdisciplinary technical field called "neural network" has gradually formed over the years. The research of neural network involves many disciplines, which are combined, infiltrated and promoted each other. Starting from the interests and characteristics of their respective disciplines, scientists in different fields put forward different questions and conduct research from different angles.
Let's compare the working characteristics of artificial neural network and general computer:
From the speed point of view, the speed of information transmission between human brain neurons is much lower than that of computers, the former is in the order of milliseconds, and the latter often reaches hundreds of megahertz. However, because the human brain is a large-scale parallel and serial processing system, it can make rapid judgments, decisions and treatments on many problems, and its speed is much higher than that of ordinary computers with serial structure. The basic structure of artificial neural network imitates the human brain and has the characteristics of parallel processing, which can greatly improve the working speed.
The characteristic of information storage in human brain is to use the change of synaptic efficiency to adjust the storage content, that is, information is stored in the distribution of connection strength between neurons, and the storage area is integrated with the computer area. Although a large number of nerve cells die every day in the human brain (about 1000 per hour on average), it does not affect the normal thinking activities of the brain.
Ordinary computers have independent memories and arithmetic units, and knowledge storage and data operation are not related to each other. Only through programs written by people can we communicate with each other, and this kind of communication cannot exceed the programmer's expectations. Local damage to parts and minor errors in procedures may lead to serious disorder. Psychologists and cognitive scientists study neural networks to explore the mechanism of human brain processing, storing and searching information, clarify the mechanism of human brain function, and establish the microstructure theory of human cognitive process.
Experts in biology, medicine and brain science try to promote the development of brain science to a quantitative, accurate and theoretical system through the study of neural network, and also hope that clinical medicine will make new breakthroughs; The purpose of information processing and computer scientists to study this problem is to find new methods to solve a large number of unsolvable or extremely difficult problems and to construct a new generation of computers closer to the function of the human brain.
The early research work of artificial neural network should be traced back to the 1940s. The following is a brief introduction to the development history of artificial neural network in chronological order with famous figures or outstanding research results as clues.
From 65438 to 0943, psychologist W Mcculloch and mathematical logician W Pitts first put forward the mathematical model of neurons on the basis of analyzing and summarizing the basic characteristics of neurons. This model has been used until today, which directly affects the research progress in this field. Therefore, the two of them can be called pioneers of artificial neural network research.
From 65438 to 0945, the design team led by von Neumann successfully trial-produced the stored program electronic computer, marking the beginning of the electronic computer era. 1948, in his research work, he compared the fundamental difference between the human brain structure and the stored program computer, and proposed the regenerative automata network structure composed of simple neurons. However, due to the rapid development of instruction storage computer technology, he abandoned the new way of neural network research and continued to devote himself to the research of instruction storage computer technology, and made great contributions in this field. Although von Neumann's name is associated with ordinary computers, he is also one of the pioneers in the study of artificial neural networks.
At the end of 1950s, Rosenblat designed and manufactured a kind of "perceptron", which is a multi-layer neural network. This work pushes the research of artificial neural network from theoretical discussion to engineering practice for the first time. At that time, many laboratories around the world followed the example of making perceptrons and applied them to the study of character recognition, speech recognition, sonar signal recognition and learning and memory problems. However, the research climax of artificial neural network did not last long, and many people gave up the research work in this field one after another, because the development of digital computer was in its heyday at that time, and many people mistakenly thought that digital computer could solve all problems in artificial intelligence, pattern recognition, expert system and so on, which made the work of perceptron ignored. Secondly, the level of electronic technology at that time was relatively backward, and the main components were electron tubes or transistors. The neural network they made is huge and expensive, so it is impossible to be similar to the real neural network in scale. In addition, in a book called Perceptron from 65438 to 0968, it is pointed out that the linear perceptron has limited function and cannot solve basic problems such as heteroesthesia, and the multi-layer network cannot find an effective calculation method. These arguments have prompted a large number of researchers to lose confidence in the future of artificial neural networks. In the late 1960s, the research of artificial neural network entered a low tide.
In addition, in the early 1960s, Widrow proposed an adaptive linear element network, which is a linear weighted summation threshold network with continuous values. Later, a nonlinear multi-layer adaptive network was developed on this basis. At that time, although these works were not labeled with the name of neural network, they were actually an artificial neural network model.
With the decline of people's interest in perceptron, the research of neural network has been silent for a long time. In the early 1980s, the manufacturing technology of analog and digital mixed VLSI was raised to a new level and put into practical application. In addition, the development of digital computers has encountered difficulties in several application fields. This background shows that the time is ripe to find a way out from artificial neural network. Hopfield, an American physicist, published two papers on artificial neural networks in the Proceedings of the National Academy of Sciences in 1982 and 1984, which caused great repercussions. People have re-recognized the power of neural network and the reality of its application. Immediately, a large number of scholars and researchers carried out further work around the method proposed by Hopfield, forming a research upsurge of artificial neural networks since the mid-1980s.
1985, Ackley, Hinton and Sejnowski applied simulated annealing algorithm to neural network training and proposed Boltzmann machine. The advantage of this algorithm is to avoid falling into extreme value, but the training time needs a long time.
In 1986, Rumelhart, Hinton and Williams proposed a learning algorithm of multilayer feedforward neural network, namely BP algorithm. The correctness of the algorithm is deduced from the point of view of proof, which provides a theoretical basis for learning the algorithm. From the point of view of learning algorithm, it is a great progress.
1988, Broomhead and Lowe first proposed radial basis network: RBF network.
Generally speaking, neural network has experienced a tortuous process from climax to trough and then to climax.