Quantum-Inspired Neural Network with Quantum Weights and Real Weights

To enhance the approximation ability of neural networks, by introducing quantum rotation gates to the traditional BP networks, a novel quantum-inspired neural network model is proposed in this paper. In our model, the hidden layer consists of quantum neurons. Each quantum neuron carries a group of quantum rotation gates which are used to update the quantum weights. Both input and output layer are composed of the traditional neurons. By employing the back propagation algorithm, the training algorithms are designed. Simulation-based experiments using two application examples of pattern recognition and function approximation, respectively, illustrate the availability of the proposed model.

• Quantum Computing
• Quantum Rotation Gate
• Quantum-Inspired Neuron
• Quantum-Inspired Neural Network

1. Introduction

In 1980s, Benioff firstly proposed the concept of quantum computation [1] . Shor discussed the first quantum algorithm of very large integer factorization [2] in 1994. In 1996, Grover explored an important quantum algorithm, which can search for a marked state in an unordered list [3] . Although the quantum machines are not yet technologically feasible, the quantum algorithms that can be applied on the quantum computers are indeed interesting and significantly different from the classical computing. As we know, fuzzy logic, evolutionary computation, and neural networks are regarded as intelligent computing (soft computing), and also have some comparability with the quantum computation [4] . Therefore, combination of these computing methods is emerging. Different from the Hebbian learning, a quantum neural network can be used for the enriched learning of neural networks. Proposed by Penrose in 1989 [5] , the idea of quantum information processing in the human brain was still a controversial theory, which has not been experimentally proved. However, the exploration of quantum information devices is a promising research topic, because of the enhanced capacity and speed from the quantum mechanism. With characteristics such as smaller size of quantum devices, larger capacity of quantum networks and faster information processing speed, a quantum neural network can mimic some distinguishing properties of the brain better than the classical neural networks, even if the real human brain does not even have any quantum element. In a word, the quantum neural networks have important research significance in both theory and engineering.

Since Kak firstly proposed the concept of quantum neural computation [6] in 1995, the quantum neural networks had attracted great attention during the past decade, and a large number of novel techniques had been studied for the quantum computation and neural networks. For example, Ref. [7] proposed the model of quantum neural networks with multi-level hidden neurons based on the superposition of quantum states in the quantum theory. Ref. [8] proposed a neural network model with quantum gated nodes and a smart algorithm for it, which showed superior performance in comparison with a standard error back propagation network. Ref. [9] proposed a weightless model based on quantum circuit. It is not only quantum-inspired but is actually a quantum NN. This model is based on Grover’s search algorithm, and it can perform both quantum learning and simulate the classical models. Ref. [10] proposes the neural networks with the quantum gated nodes, and indicates that such quantum networks may contain more advantageous features from the biological systems than the regular electronic devices. Ref. [11] have proposed a quantum BP neural networks model with learning algorithm based on the single-qubit rotation gates and two-qubit controlled-NOT gates.

In this paper, we study a new hybrid quantum-inspired neural networks model with quantum weights and real weights. Our scheme is a three-layer model with a hidden layer, which employs the gradient descent principle for learning. The input/output relationship of this model is derived based on the physical meaning of the quantum gates. The convergence rate, number of iterations, and approximation error of the quantum neural networks are examined with different restriction error and restriction iterations. Three application examples demonstrate that this quantum-inspired neural network is superior to the classical BP networks.

2. Quantum-Inspired Neural Network Model

2.1. Qubits and Quantum Rotation Gate

In the quantum computers, the “qubit” has been introduced as the counterpart of the “bit” in the conventional computers to describe the states of the circuit of quantum computation. The two quantum physical states labeled as and express 1 bit information, in which corresponds to the bit 0 of classical computers, while bit 1. Notation of “” is called the Dirac notation, which is the standard notation for the states in the quantum mechanics. The difference between bits and qubits is that a qubit can be in a state other than and. It is also possible to form the linear combinations of the states, namely superpositions

where and are complex numbers, called probability amplitudes. That is, the qubit state collapses into either state with probability, or state with probability, and we have

Hence, the qubit can be described by the probability amplitudes as.

Suppose we have n qubits, and correspondingly, a n qubits system has computational basis states. Similar to the case of a single qubit, the n qubits system may form the superpositions of basis states:

where is called probability amplitude of the basis states, and “” means “the set of strings of length two with each letter being either zero or one”. The condition that these probabilities can sum to one is expressed by the normalization condition

In the quantum computation, the logic function can be realized by applying a series of unitary transform to the qubit states, which the effect of the unitary transform is equal to that of the logic gate. Therefore, the quantum services with the logic transformations in a certain interval are called the quantum gates, which are the basis of performing the quantum computation.

The definition of a single qubit rotation gate is given

Let the quantum state, and can be transformed by as follows:

It is obvious that shifts the phase of.

2.2. Quantum-Inspired Neuron Model

A neuron can be described as a four elements array: (input, weight, transform function, output), where input and output is the outer attribute of the neuron, and weight and transform function are the inner attribute of the neuron. Therefore, the different neuron models can be constructed by modifying types of weight and transform function. According this viewpoint, for the quantum-inspired neuron proposed in this paper, the weights are represented by qubits, and the transform function is represented by inner-product operator. The difference from the traditional neuron is that the quantum-inspired neuron carries a group of single-bit quantum gates that modify the phase of quantum weights. The quantum-inspired neuron model is shown in Figure 1.

In quantum-inspired neuron model, the weights are represented by qubits. Let is the input real vector, y is an output real number, is the quantum weight, the input/output relation of quantum-inspired neuron can be described as follows:

where is an inner product operator, , , the is a quantum rotation gate to modify the phase of the.

2.3. Quantum-Inspired Neural Network Model

Quantum-inspired Neural Network (QINN) is defined as the model that all the input, output, and linked weights for each layer may be qubits. The QINN structure is the same as the general ANN which includes input layer, hidden layer, and output layer. Obvious, the network including neuron in Figure 1 is QINN. The QINN only including quantum neurons is defined as the normalization QINN, and the QINN including quantum neurons and general neurons is defined as the hybrid QINN. Since QINN transform function adopts linear operator, the non-

Figure 1

Quantum-inspired neuron model

linear mapping capability falls under restriction. Therefore, this paper considers the hybrid QINN (HQINN) including a hidden layer of quantum neuron, which holds the advantage of the quantum computing and the nonlinear mapping capability of the general ANN. HQINN model is presented in Figure 2.

The model includes three layers, the input layer and output layer contains n and m general neurons, respectively, and the hidden layer contains p quantum-inspired neurons. The input/output relation can be described as follows:

where;;, is the linked weight between the jth neuron in hidden layer and the kth neuron in output layer, g is the Sigmoid function or Gauss function.

2.4. Learning Algorithm

For the HQINN in Figure 2, when transform function in output layer is continuous and differentiable, the learning algorithm may adopt BP algorithm. The output error function is defined as follows:

where represents the desired output.

According to BP algorithm, the modifying formula for output layer weight is described as follows:

where, , , represents the learning ratio, and t represents the iteration steps.

The weights between the input layer and quantum hidden layer are modified by quantum rotation gates described as follows:

where,. According BP algorithm, the can be gained from the following formula.

where is the phase of the. For, the modifying formula is described as follows.

Figure 2

Hybrid quantum-inspired neural network model

Let the vector, the modifying formula of is described as follows:

where t represents the iteration steps.

Synthesizing above depiction, the HQINN learning algorithm can be described as follows:

Step 1: Initialize network parameters. Include: The number of nodes in each layer, the learning ratio, the restriction error, the restriction steps. Set the current steps.

Step 2: Initialize network weights. Hidden layer:, , and output layer:, where is a random number in (0,1), , ,.

Step 3: Compute network outputs according to Equation (8), modifying the weights of each layer according to Equation (10) and Equation (14), respectively.

Step 4: Compute network output error according to Equation (9), if or then go to Step 5，else, go back to Step 3.

Step 5: Save the weights, and stop 3.

3. Simulations

To testify the validity of HQINN, two kinds of experiments are designed and the HQINN is compared with the classical BP network in this section. In two experiments, the HQINN adopts the same structure and parameters as the BP network.

3.1. Twenty-Five Sample Patterns Classification

For twenty-five sample points in Figure 3, determine the pattern of each point by HQINN. This is a typical two-pattern classification problem, which is regard as the generalization of XOR problem.

The network construct is set to 2-10-1 , and the learning ratio is set to 0.8. The HQINN and the BP network are learned 50 times, restively, and then compute the average of iteration steps. Training results are shown in Table 1. When the restriction error takes 0.05, the convergence curves are shown in Figure 4.

Figure 3

Twenty-five pattern classification samples

Figure 4

The convergence curve comparison

Table 1

The training results comparison between HQINN and BP network for 25 sample patterns classification

Restriction error	Restriction steps	HQINN		BP
		Iteration steps	Convergence ratio	Iteration steps	Convergence ratio
0.10	1000	583	90%	1000	Non-convergence
0.05	2000	1131	84%	2000	Non-convergence
0.01	10000	8893	82%	10000	Non-convergence

3.2. Double Spiral Curves Classification

For two spiral curve in Figure 5, fetch 30 points from each curve, respectively, compose the sample set containing 50 sample points, and classify these points in sample set by HQINN.

The network construct is 2-20-1 , and the learning ratio is 0.7. The HQINN and the BP network are training 30 times, restively, and then compute the average of iteration steps. Training results are shown in Table 2. When the restriction error takes 0.10, the convergence curves are shown in Figure 6.

3.3. Function Approximation

In this simulation, such function is selected as follow:

where, , and are integers in the set {1, 2, 3, 4, 5}.

To approximate the nonlinear function, we sample 40 groups of discrete data, half of which is used to train networks and the other half to test its performance. Set the maximum of iteration steps 5000. By changing the number of hidden neurons and the learning coefficient, we present an experimental evaluation of HQINN’s and compare their performance with that of the other algorithms. This example is simulated 30 times for each group of parameters by the HQINN, the BP, and the algorithm in Ref. [12] , respectively. When the number of hidden neurons is 6 and 8, the training result are presented in Table 3 and Table 4.

Firstly, we investigate how to change for the average convergence rate when the learning coefficient changes. The networks structure is 3-5-1 , the restriction error is 0.1, and the restriction iteration steps are 5000. The learning coefficient set is {0.1, 0.2, …, 1.0}. This example is simulated 30 times for each learning coefficient by the HQINN and the BP respectively. When the learning coefficient changes the average convergence ratio of HQINN always is 100% and is insensitive to the learning coefficient. However, the average convergence ratio of BP changes in a large range when the learning coefficient changes. The maximum of average convergence ratio of BP reaches to 80%, and the minimum is only 10%. The comparison result is shown in Figure 7.

The comparison result shows that the HQINN is evidently superior to the BP in both the average convergence ratio and the robustness.

Figure 5

Double spiral curves

Figure 6

The convergence curve comparison

Figure 7

The relationship between the average convergence rate and the learning coefficient

Table 2

The training results comparison between HQINN and BP network for double spiral curves classification

Restriction error	Restriction steps	HQINN		BP
		Iteration steps	Convergence ratio	Iteration steps	Convergence ratio	Necessary steps for convergence
0.10	2000	871	82%	2000	Non-convergence	3961
0.05	10000	6893	92%	8631	76%	―

Secondly, we investigate how to change for the average iteration steps when the learning coefficient changes. The networks structure is 3-5-1 , and the restriction error is 0.1. The learning coefficient set is still {0.1, 0.2, …, 1.0}. This example is simulated 30 times for each learning coefficient by the HQINN and the BP, respectively. When the learning coefficient changes, for the average iteration steps of HQINN, the maximum is 958 steps, the minimum is 496 steps, and the average is 693 steps. However, for the average iteration steps of BP, the maximum is 4619 steps, the minimum is 1532 steps, and the average is 2489 steps. Hence, the performance of HQINN is evidently superior to BP in both the average iteration steps and its fluctuation range when the learning coefficient changes. The comparison result is shown in Figure 8.

The comparison results show that the HQINN is evidently superior to the BP in both the average iteration steps and the robustness.

4. Conclusion

The HQINN is the amalgamation of quantum computing and nerve computing, which holds the advantage such as parallelism and high efficiency of quantum computation besides continuity, approximation capability, and generalization capability that the classical ANN holds. In HQINN, the weights are represented by qubits, and the phase of each qubit is modified by the quantum rotation gate. Since both probability amplitudes participate in optimizing computation, the computation capability is evidently superior to the classical BP network. Experimental results show that the HQINN model proposed in this paper is effective.

Figure 8

The relationship between the average iteration steps and the learning coefficient

Table3. The simulation result for the nonlinear function approximation (the number of hidden neurons is 6).

Table4. The simulation result for the nonlinear function approximation (the number of hidden neurons is 8).

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61170132), the Scientific research and technology development project of CNPC (2013E-3809), and the Major National Science and Technology Program (2011ZX05020-007).

References

1. Benioff, P. (1982) Quantum Mechanical Hamiltonian Models of Turing Machines. Journal of Statistical Physics, 3, 515-546. http://dx.doi.org/10.1007/BF01342185

2. Shor, P.W. (1994) Algorithms for Quantum Computation: Discrete Logarithms and Factoring. Proceedings of the 35th Annual Symposium on Foundations of Computer Science, Santa Fe, 20-22 November 1994, 124-134. http://dx.doi.org/10.1109/SFCS.1994.365700

3. Grover, L.K. (1996) A Fast Quantum Mechanical Algorithm for Database Search, Proceedings of the 28th Annual ACM Symposium on the Theory of Computing, Pennsylvania, May 1996, 212-219.<br /> http://dx.doi.org/10.1145/237814.237866

4. Penrose, R. (1994) Shadows of the Mind: A Search for the Missing Science of Consciousness. Oxford University Press, London, 196-201.

5. Penrose, R. (1989) The Emperor’s New Mind: Concerning Computers, Minds, and the Laws of Physics. Oxford University Press, London, 89-93.

6. Kak, S. (1995) On Quantum Neural Computing. Information Sciences, 83, 143-160. <br /> http://dx.doi.org/10.1016/0020-0255(94)00095-S

7. Gopathy, P. and Nicolaos B.K. (1997) Quantum Neural Networks (QNN’s): Inherently Fuzzy Feed Forward Neural Networks. IEEE Transactions on Neural Networks, 8, 679-693. <br />http://dx.doi.org/10.1109/72.572106

8. Li, P.C., Song, K.P. and Yang, E.L. (2010) Model and Algorithm of Neural Network with Quantum Gated Nodes. Neural Network World, 11, 189-206.

9. Adenilton, J., Wilson, R. and Teresa, B. (2012) Classical and Superposed Learning for Quantum Weightless Neural Network, Neurocomputing, 75, 52-60. http://dx.doi.org/10.1016/j.neucom.2011.03.055

10. Shafee, F. (2007) Neural Networks with Quantum Gated Nodes. Engineering Applications of Artificial Intelligence, 20, 429-437. http://dx.doi.org/10.1016/j.engappai.2006.09.004

11. Li, P.C. and Li, S.Y. (2008) Learning Algorithm and Application of Quantum BP Neural Networks Based on Universal Quantum gates. Journal of Systems Engineering and Electronics, 19, 167-174.<br /> http://dx.doi.org/10.1016/S1004-4132(08)60063-8

12. Zweiri, Y.H., Whidborne, J.F. and Seneviratne, L.D. (2003) A Three-Term Backpropagation Algorithm. Neurocomputing, 50, 305-318. http://dx.doi.org/10.1016/S0925-2312(02)00569-6