Bounds on Sparsity of One-Hidden-Layer Perceptron Networks

Authors: Věra Kůrková
Publication year: 2017
Publication type: Konferenční příspěvek (mezinárodní konf.)
Periodical: Proceedings ITAT 2017: Information Technologies - Applications and Theory
Projects: Model complexity of neural, radial, and kernel networks
Bibliographic citation: KŮRKOVÁ, Věra. Bounds on Sparsity of One-Hidden-Layer Perceptron Networks. In: HLAVÁČOVÁ, J., ed. Proceedings ITAT 2017: Information Technologies - Applications and Theory. Aachen & Charleston: Technical University & CreateSpace Independent Publishing Platform, 2017, S. 100-105. CEUR Workshop Proceedings, V-1885. ISBN 978-1974274741. ISSN 1613-0073. Dostupné z: http://ceur-ws.org/Vol-1885/100.pdf
ID: 478625
Online record: View online record

Abstract

Limitations of one-hidden-layer (shallow) perceptron networks to sparsely represent multivariable functions is investigated. A concrete class of functions is described whose computation by shallow perceptron networks requires either large number of units or is unstable due to large output weights. The class is constructed using pseudo-noise sequences which have many features of random sequences but can be generated using special polynomials. Connections with the central paradox of coding theory are discussed.