Keywords: artificial neural networks, number-theoretic methods (NTMs), NT-net, discrepancy, good lattice points (GLP-net), hammersley-net.

Flexibility in generalization is always what is to be pursued when an artificial neural network (ANN) model is set up. For this purpose, this paper makes efforts to improve the quality of the "teacher", i.e., to ensure that the uniformity of training samples distribution by use of the series of Number-Theoretic Methods (NTMs). NTMs are a series of deterministic number-theoretic algorithms used to generate points that uniformly scatter in s-dimensional unit cube . As the ANN prediction shows that the nature of nonlinear interpolation, uniformity of samples is helpful to produce small errors on new samples unseen during training.

Under NTMs theory frame, discrepancy is defined as a quantitative measurement for the uniformity of a set of points. The smaller the discrepancy is, the more uniformly samples distribute. Actually, discrepancy describes how well a set of points represents the uniform distribution on , .

In this paper, GLP-net, Halton-net, and Hammersley-net, are introduced as typical NT-nets. Training samples are prepared, respectively, by GLP-net, Hammersley-net, and compared with equal-spaced samples in uniformity in terms of discrepancy value.

Trained, respectively, by these three types of samples, ANN models show quite different performance in computational precision and stability. ANNs trained by NTM-based samples outperform in terms of generalization flexibility. This is demonstrated through an engineering case study in this paper.

Conclusively, good uniformity of training samples, instead of unselectively piling more and more data, is really helpful to enhance the ANNs' generalization performance. It is mathematically proven to obtain uniformly scattering samples through NTMs other than equal spaced sampling.

1

Garret, J H., "Where and why artificial neural networks are applicable in civil engineering", Journal of Computing in Civil Engineering, 8(2), 129 130, 1994. doi:10.1061/(ASCE)0887-3801(1994)8:2(129)

2

Yuan, Z.R., "Artificial neural network and its application", Beijing: Tsinghua University Press, 1999. (in Chinese)

3

Prechelt, L., "Automatic early stopping using cross validation: quantifying the criteria. Neural Networks", 11, 761 767, 1998. doi:10.1016/S0893-6080(98)00010-0

4

Tong, F., Liu, X.L., "Samples Selection for Artificial Neural Network Training in Preliminary Structural Design", Tsinghua Science and Technology, to be published.

5

Lunani, M., Sudjianto, A., Johnson, P.L., "Generating efficient training samples for neural networks using latin hypercube sampling", Intelligent Engineering Systems Through Artificial Neural Networks, 5, 209 214, 1995.

6

Fang, K.T., Wang, Y., "Number-theoretic methods in statistics". Beijing: Science Press, 1996. (in Chinese)

7

Niederreiter H., "Random number generation and quasi-Monte Carlo Methods", Philadelphia: Society for Industrial and Applied Mathematics, 1992.

8

Hua L.G., Wang, Y., "Application of Number-theoretic methods in approximate analysis". Beijing: Science Press, 1978. (in Chinese)

9

Shen, S.Z., Chen X., "Stability of reticulated shells", Beijing: Science Press, 1999. (in Chinese)

10

Rogers, J.L. "Simulating structural analysis with neural network", Journal of Computing in Civil Engineering, 8(2), 252 265, 1994. doi:10.1061/(ASCE)0887-3801(1994)8:2(252)

11

Hickernell, F.J., "A generalized discrepancy and quadrature error bound", Math. Comp., 67, 299 322, 1998. doi:10.1090/S0025-5718-98-00894-1

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £95 +P&P)