Some Comparisons of Linear and Deep ReLU Network Approximation

Authors: Věra Kůrková
Publication year: 2024
Publication type: Konferenční příspěvek (mezinárodní konf.)
Periodical: Artificial Neural Networks and Machine Learning – ICANN 2024. Proceedings, Part X
Projects: AppNeCo: Approximate Neurocomputing
Bibliographic citation: KŮRKOVÁ, Věra. Some Comparisons of Linear and Deep ReLU Network Approximation. In: WAND, M., MALINOVSKÁ, K., SCHMIDHUBER, J., TETKO, I. V., eds. Artificial Neural Networks and Machine Learning – ICANN 2024. Proceedings, Part X. Cham: Springer, 2024, S. 231-240. Lecture Notes in Computer Science, 15025. ISBN 978-3-031-72358-2. ISSN 0302-9743
Dostupné z: doi:10.1007/978-3-031-72359-9_17.
DOI: 10.1007/978-3-031-72359-9_17
ID: 598159
Online record: View online record

Abstract

Influence of depth of ReLU networks on growth of their non-linearity is studied. Lower bounds on worst-case errors in linear approximation are derived for sets of highly-oscillatory functions that can be exactly represented by ReLU networks. Dependence of these errors on network depth is analyzed.