Complexity of Shallow Networks Representing Finite Mappings

Authors: Věra Kůrková
Publication year: 2015
Publication type: Konferenční příspěvek (mezinárodní konf.)
Periodical: Artificial Intelligence and Soft Computing
Projects: Model complexity of neural, radial, and kernel networks
Bibliographic citation: KŮRKOVÁ, Věra. Complexity of Shallow Networks Representing Finite Mappings. In: RUTKOWSKI, L., KORYTKOWSKI, M., SCHERER, R., TADEUSIEWICZ, R., ZADEH, L.A., ZURADA, J.M., eds. Artificial Intelligence and Soft Computing. Vol. 1. Cham: Springer, 2015, S. 39-48. Lecture Notes in Artificial Intelligence, 9119. ISBN 978-3-319-19323-6. ISSN 0302-9743
Dostupné z: doi:10.1007/978-3-319-19324-3_4
DOI: 10.1007/978-3-319-19324-3_4
ID: 443724

Abstract

Complexity of shallow (one-hidden-layer) networks representing finite multivariate mappings is investigated. Lower bounds are derived on growth of numbers of network units and sizes of output weights in terms of variational norms of mappings to be represented. Probability distributions of mappings whose computations require large networks are described. It is shown that due to geometrical properties of highdimensional Euclidean spaces, representation of almost any randomly chosen function on a sufficiently large domain by a shallow network with perceptrons requires untractably large network. Concrete examples of such functions are constructed using Hadamard matrices.