A Methodology for Developing Nonlinear Models by Feedforward Neural Networks

A1 Journal article (refereed)


Internal Authors/Editors


Publication Details

List of Authors: Saxen H, Pettersson F
Publication year: 2009
Journal: Lecture Notes in Computer Science
Journal acronym: LECT NOTES COMPUT SC
Volume number: 5495
Start page: 72
End page: 78
Number of pages: 7
ISSN: 0302-9743


Abstract

Feedforward neural networks have been established as versatile tools for nonlinear black-box modeling, but in many data-mining tasks the choice of relevant inputs and network complexity is still a major challenge. Statistical tests for detecting relations between inputs and outputs are largely based on linear theory, and laborious retraining combined with the risk of getting stuck in local minima make the application of exhaustive search through all possible network configurations impossible but for toy problems. This paper proposes a systematic method to tackle the problem where an output shall be estimated on the basis of a (large) set of potential inputs. Feedforward neural networks of multi-layer perceptron type are used in the three-stage modeling approach: First, starting, from sufficiently large networks an efficient pruning method is applied to detect a pool of potential model candidates. Next, the Akaike weights are used as to select the actual Kullback-Leibler best models in the pool. Third, the hidden nodes of these networks are available for the final network, where mixed-integer linear programming is applied to find the optimal combination of M hidden nodes, and the corresponding upper-layer weights. The procedure outlined is demonstrated to yield parsimonious models for a nonlinear benchmark problem, and to detect the relevant inputs.


Keywords

Information criterion, Neural networks, Non-linear black-box modeling, Structural and parametric optimization

Last updated on 2019-14-11 at 05:54