آمار استنباطی چیست؟
October 22, 2022
معرفی کامل 4 الگوریتم یادگیری عمیق
January 3, 2023
In your opinion, how can an intelligent software understand certain issues and can solve similar issues with the least errors? To answer this question, it is better to raise another question; How can humans learn a subject? How do we learn from our failures?
In fact, by using neural networks, we can understand different issues, learn from our mistakes and build a better life for ourselves.
So, in order to be able to design an intelligent machine that has the power to learn, the best choice is to simulate human neurobiological networks on a computer. Therefore, the neural networks in artificial intelligence are inspired by the neural networks of the human brain.
What is a neural network?
In order to be able to implement neural networks in artificial intelligence, we must first better understand the concept of neural networks in our brain.
In general, in order to design intelligent tools with decision-making power, in addition to understanding the concept of being intelligent, we must also know intelligent beings better. What creature is smarter than humans?
Biological neural network
The human brain consists of billions of neurons (about 85 billion) and these neurons can communicate with each other.
Using this definition, we can conclude that our brain consists of a huge neural network of nerve cells. Neurons are nerve cells. There are different types of neurons; However, let us take a look at the general structure of a neuron in the brain:
- Cell body: This part, also called soma, includes the cell nucleus and cell organelles.
- Dendrite: The task of dendrites is to transmit nerve messages to the cell nucleus.
- Axon: It is responsible for transmitting the nerve message from the cell body to the axon terminal.
- Axon terminal: It is responsible for transmitting messages to a neuron or organ through a synaptic space.
In fact, neurons are the most fundamental nerve cells. Each neuron can communicate with more than 10,000 other neurons. When we’re learning a new skill, one of the most important things that happens in our brain is making new connections between different neurons, and when we’re practicing, we’re strengthening those connections.
The process of learning in the brain is too complicated to be explained in a few paragraphs and many books can be written for this process in the brain.
Artificial neural network
In computer science, it is also possible to form an artificial neural network by designing a data structure similar to NOREN and connecting its samples with each other. In fact, the neural network is an information processing system that has the same function as the biological neural network.
Artificial neural networks are actually a combination of mathematics and a structure similar to the neural network of the human brain:
· Information processing is done in simple units called neurons.
· Signals (information) are sent through connections between neurons.
· Each connection has its own weight; The weights are usually multiplied by the input signals of the connection.
· Each neuron applies an activation function (usually non-linear) to the network inputs. The activation function actually determines the output of the neuron.
You can see a simple neural network in the picture below:
In general, a neural network consists of three general layers, which are respectively called input layer, hidden layer, and output layer. Although a neural network can have layers other than these, if the hidden layer contains more than one sub-layer, it is called a deep neural network.
An artificial neural network can be characterized by the following three characteristics:
- The pattern of connections between neurons, which is also called neural network architecture.
- Their method for identifying and updating the weights of connections between neurons which is also called a learning algorithm.
- Their activation function: A neural network consists of a large number of simple information processing units called neurons or nodes. Each neuron is connected to the input of other neurons using directional communication links. Usually, any activation function in a neuron is a non-linear mathematical function. Each link also includes a weight that can help solve the problem, and they are always multiplied by the inputs.
Important factors that determine the behavior of a neuron include the activation function, the input/output connections, and the weights of each connection. The only layer whose input data cannot be the output of other neurons is the input layer.
The input layer does not perform special calculations and only sends data to other layers; So, they can be ignored when counting the layers. Also, the output layer is also ignored in counting neural network layers. In other words, the number of network layers can be determined as follows:
If a layer contains weighted input and output connections, it can be considered as a part of network layers.
History of neural networks
The history of neural networks probably began in the late 1800s with scientific efforts to study the activity of the human brain. In 1890, William James published his first work on patterns of brain activity. However, the main building block of today’s neural networks was the result of the efforts of a neurophysiologist named McCulloch and a mathematician named Walter Pitts.
In 1943, the two wrote a paper on how neurons work. Also, to describe the function of neurons in the brain, they modeled a simple neural network using electrical circuits.
In 1949, Donald Hebb wrote a work called “Organization of Behavior”; A work that refers to the fact that neural pathways are strengthened with each use. He reasoned that each firing of electrical signals between two nerve cells strengthens the connection between them.
This rule, later known as Hebbian Learning Rule in honor of Donald Hebb, is one of the simplest learning rules for artificial neural networks. In artificial neural networks, this is a learning rule called Hebbian.
In the 1950s, when computers had become more powerful, hopes rose for simulating a neural network on a computer. The first simulation was done by Nathaniel Rochester (computer scientist) in the IBM laboratory, which unfortunately failed.
In 1951, Narvin Minsky built the first artificial neural network (ANN) at Princeton University.
However, efforts have continued until today, and one of the fruits of these efforts has been voice assistants like Alexa.
The structure of the neural network
As you have already seen, the structure of a neural network usually consists of three parts, namely the input layer, the hidden layer, and the output layer. However, it can also include convolutional layers and an encoding or decoding layer.
Input layer
In a neural network, input layer neurons play a different role than other neurons. The neurons of the input layer are called “passive neurons” because they do not receive information from previous neurons (quite obvious why, since these neurons are in the first layer).
Neurons in an artificial neural network usually need to contain weighted inputs and act on them. But this theorem can be different for the neurons of the input layer because the neurons of the input layer cannot include weights and the weights are calculated in the neurons of the input layer.
One of the common ways to generate weights is to use random numbers. The neurons of the input layer are responsible for sending the weighted output to the hidden layer; Because the hidden layer includes neurons whose input must include weight.
It is also necessary to note that the output connection of an input layer can be to a neuron in the second sublayer in the hidden layer; That is, there is no obligation to connect the input layer to the first sublayer in the hidden layer.
In an artificial neural network, the input layer can be more than one layer.
Inputs to the input layer must first be normalized and then sent to the input layer. Data normalization is one of the most important steps in data mining.
Hidden layer
This layer is between the input layer and the output layer and can include the substrate. Perhaps it can be said that the most important part in an artificial neural network is the hidden layer; Because important calculations are done in this layer and it is prepared for the output layers. If the number of hidden layer sub-layers in an artificial neural network is more than one, that network is considered a deep neural network.
Note that each sublayer in the hidden layer is a hidden layer itself, and we have considered the hidden layer as a set in this section. This rule applies to other layers as well.
In this layer, activities similar to the activities of the human brain are performed. The hidden layer contains the neurons that contain the weighted input and the activation function.
The activation function is the most important part of a neuron, which is responsible for generating output for the next neuron or neurons according to the parameters that come with the original data (usually weight and bias).
The methods of configuring hidden layer neurons can be different. Some are fine-tuned and calibrated by a process called post-release. The simplest definition of them is:
It receives a possible signal (data) like a biological neuron, performs the necessary calculations and produces a weighted output for other neurons (activator function).
In machine learning, the focus is more on the hidden layer. Each sublayer in the hidden layer can contain a specific activation function. Neurons of the same layer usually behave similarly.
Output layer
The output layer is responsible for delivering the final result to the program. More than one output layer can be used in a neural network. An output function can represent a concept. For example, in a neural network that is supposed to recognize the image of a number, it is possible to have an output layer that contains 10 neurons.
Each of them represents a number from 0 to 9 and the output of each is a number between 0 and 1. If the output number in neuron number 10 is equal to 0.8 and the output of other neurons is below this number, we can conclude that the target number is 9 (because we started from 0).
Connections and weights and biases
A network has no meaning without connecting its components to each other. It can be said that the most important component of a network is the connections between its components.
In the neural network of the human brain, each connection has an intensity that determines its importance, this phenomenon should also be implemented in artificial neural networks. For this purpose, in the artificial neural network, each connection contains a weight, and each weight indicates the importance of a connection. A weight in an artificial neural network represents intensity in a biological neural network, and the bias is a constant number that helps the calculations in the neural network.
It is quite clear that weights are the most important element in the learning process of a neural network.
In fact, a neural network works in such a way that it receives a huge set of information that includes the correct question and answer. This data should always be delivered to the network in numerical form. The task of the neural network is to find a connection between this information and the correct answer through mathematical patterns.
This can accompanied with many errors and for each error, the neural network is required to update the weights and biases and get closer to the ideal state, therefore, we need to consider an error bound for the neural network. For each update, the distance from the correct answer increases and decreases, and this process continues until we reach the desired error limit.
Activator function
As we mentioned earlier, the most important part of a neuron is its activation function. Activator function is a mathematical function. Sometimes these functions are also called motivational functions.
Activator functions usually produce a value between 0 and 1. This can be different based on the function itself, and the value may be more than 1 for some functions. For example, in the ReLU function, the output values can be more than 1. Usually, the activation function is a non-linear function. Some of the activator functions are:
- Sigmoid / Logistic
- ReLU
- Leaky ReLU
- Parametric ReLU
Learning rules
A learning rule is a math-based method that helps a neural network learn from its mistakes and improve its performance. Obviously, this process is an iterative process; Because we can’t learn until we make mistakes, and when we learn, we try again. This process continues until our mistakes are minimized. Learning rules can be thought of as methods for updating weights and biases.
Some of the learning methods are:
- Hebbian
- Perceptron
- Delta
Types of artificial neural network
There are different types of artificial neural networks, each of which can be used for specific purposes. Nonetheless, an artificial neural network can be used in many fields. Some artificial neural networks are:
- Long short-term memory
- Recurrent neural network
- Convolutional neural network
- Perceptron
Neural network applications
Neural networks have many applications in different sciences. Neural networks can be used in antivirus, data mining, modeling of complex physics phenomena, image processing, stock price prediction, etc.
Some applications of neural network are:
- information classification
- Weather forecasting
- image processing
- speech recognition
- natural language processing
- Stock price prediction
- making medicine
- simulation of complex physical phenomena
Conclusion
Artificial neural networks are considered the most important part of deep learning. These networks are inspired by the human brain and its neurons. Usually, neural networks include three input, hidden and output layers. The input and output layers are not a part of the neural network.
The most important part of an artificial neural network is its hidden layer, and if it contains more than one sub-layer, that network is called a deep neural network. The most important element in the learning process of a neural network is the weight of their connections. Learning rules are methods for updating weights and biases. Biases are an auxiliary number in calculations. Activator functions are the most important part of a neuron; These functions are a mathematical formula.
Implementing neural networks and understanding them is not easy and definitely requires a lot of time and practice. At Big Pro One, we have tried so that researchers who need to analyze their data can achieve their goals with a few simple clicks without needing any knowledge in the field of artificial intelligence.
Resource: https://b2n.ir/z95042
