Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights

It is shown that feedforward networks having bounded weights are not undesirable restricted, but are in fact universal approximators, provided that the hidden-layer activation function belongs to one of several suitable broad classes of functions: polygonal functions, certain piecewise polynomial functions, or a class of functions analytic on some open interval. These results are obtained by trading bounds on network weights for possible increments to network complexity, as indexed by the number of hidden nodes. The hidden-layer activation functions used include functions not admitted by previous universal approximation results, so the present results also extend the already broad class of activation functions for which universal approximation results are available. A theorem which establishes the approximate ability of these arbitrary mappings to learn when examples are generated by a stationary ergodic process is given