Keywords: genetic algorithm, neural network architecture, optimal design, pruning.

In recent years, artificial neural networks (ANNs) have been intensively studied and testified successfully in a wide range of applications. Despite these advances, certain questions still remain unsolved and keep challenge to researchers. One of these challenges is to determine the most appropriate network size for solving a special task. The network designer's dilemma stems from the fact that either large or small network exhibits its own advantages. If a network has too many free parameters, i.e., weights and/or neurons, not only the learning is fast, but also a local minimum is easily avoided. In particular, a theoretical study has shown that when the number of hidden neurons equals the number of training examples (minus one), the back-propagation (BP) error surface is guaranteed to have no local minima [1]. On the other hand, a network with fewer free parameters exhibits a better generalization performance. Such character can be explained by recalling the analogy between neural network learning and curve fitting. Moreover, from an implementation point, small networks only require limited resources in physical computational spaces [2]. Therefore, more attention has focused on how to construct a suitable network structure for a given task.

To solve the problem of choosing the proper network size, a trial and error method is commonly used in most cases but with the penalty of heavy computational burden and low efficiency. To date, there are mainly two adaptive methods: constructive algorithm and destructive algorithm. The former begins with a small network; the neurons and links are then gradually added to it until the network meets the given requirements. The main drawback is that the neurons in the network may be more than that needed. The latter starts with a large network; the neurons and links may be deleted to obtain a suitable network based on actual requirement. The main problems are that it is difficult to set the initial network, and whether a connection weight is deleted or not depends on its relative importance, which is related to the special mapping problem that network targets to achieve. It is also very difficult for the destructive algorithms to find a general cost function, which can generate a small network for any mapping problems.

Currently, the research and application of evolutionary artificial neural networks (EANNs) is in the ascendant. EANNs refer to a special class of artificial neural networks (ANNs) in which evolution acts as another fundamental form of adaptation in addition to learning [3,4]. Evolutionary algorithms can be used to perform various tasks, such as connection weight training, architecture design, learning rule adaptation, input feature selection, connection weight initialization, rule extraction from trained ANNs, etc. EANNs can also adapt to an environment where variations frequently occur. The two adaptation forms in EANNs, i.e., evolution and learning, lead to the adaptation to a dynamic environment more efficient.

In this paper, an effective designing method of neural network architectures is presented. A synthesis method is developed to optimize the neural network architectures in the study. The synthesis method is regarded as a combination of both constructive algorithm and destructive algorithm, and possesses high efficiency in dealing with complex dynamic environment. The steps include: a dynamic constructive method is firstly adopted to train neural network; the genetic algorithm is then used to prune the trained network; the global optimal solution can be obtained rapidly due to the good initial solutions comparing with the general destructive algorithms. Moreover, the algorithm presented here is simple and computable. Simulation results will be presented to illustrate the efficiency of the proposed method. The results show that the performance of the developed model in generalization and expandability is reliable, efficient, and superior to the general destructive algorithms.

1

Angeline, PJ, Saunders, GM, Pollack, JB, "An evolutionary algorithm that constructs recurrent neural networks", IEEE Trans. Neural Networks, 5, 54-65, 1994. doi:10.1109/72.265960

2

Castellano, G, Fanelli, AM, "An iterative pruning algorithm for feed-forward neural networks." IEEE Trans. Neural Networks, 8(3), 519-53, 1997. doi:10.1109/72.572092

3

Yao, X, "A review of evolutionary artificial neural networks", Int. J. Intell. Sys. 539-567, 1993. doi10.1002/int.4550080406

4

Yao, X, "Evolutionary artificial neural networks", Encyclopedia of Computer Science and Technology (A. Kent and J. G. Williams, eds), New York, NY10016: Marcel Dekker Inc., 33,137-170, 1993.

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £82 +P&P)