Neural networks are mathematical models inspired by the human brain that are used for things like image and speech recognition. Researchers create equations to represent how neurons in the brain connect and communicate.
However, these neural network models can be very complex mathematically. Often, researchers are unable to solve them fully to find exact solutions. Instead, they rely on approximate solutions.
This research studied a special type of neural network that deals with quaternions – a more complex type of number beyond regular real numbers. The researchers wanted to know – if you have a sequence of better and better approximate solutions, does an exact solution exist?
They introduced a concept called “Hyers-Ulam stability” to study this question. Basically, if small changes in the approximate solutions lead to small changes in the exact solution, the model is Hyers-Ulam stable. This would indicate an exact solution does exist.
The researchers used successive approximation, where they iteratively improved the approximate solutions. They found that under certain conditions, the approximations do converge to an exact, unique solution for the quaternion networks.
In simpler terms, this means that even if we are unable to directly solve these complex neural networks fully, the approximations we can calculate can tell us useful information about the underlying exact solution. So the approximations are valuable beyond themselves.
The researchers also demonstrated this with some numerical examples. Their findings help both theoretically and practically when working with approximations of neural networks.
Background
- QVNNs are more complex extensions of real/complex neural networks. They can better represent high dimensional data like images.
- It is often difficult or impossible to obtain exact solutions for neural network models due to complexity and limitations in solving methods.
- Approximate solutions can often be obtained and improved on successively. The question is – does an exact solution exist as the limit of better and better approximate solutions?
- This property is studied using the concept of “Hyers-Ulam stability”. If a system is Hyers-Ulam stable, its approximate solutions will converge to an exact solution.
Methods
- The researchers propose the concept of Hyers-Ulam stability for QVNNs with time delays.
- They use a “successive approximation method” where they define a sequence of approximate solutions and study its convergence.
- Sufficient conditions are derived for the Hyers-Ulam stability of delay-dependent and delay-independent QVNNs.
Results
- It is shown that under certain conditions, successive approximate solutions converge to an exact, unique solution of the QVNN.
- This proves the existence of exact solutions, and bounds the difference between approximate and exact solutions.
- Simulations verify the theoretical results on example QVNNs and special cases like real-valued NNs.
- Research link: https://www.researchgate.net/publication/356743367_On_the_Existence_of_the_Exact_Solution_of_Quaternion-Valued_Neural_Networks_Based_on_a_Sequence_of_Approximate_Solutions