Iclr2020: Compression based bound for non-compressed network

Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network - Download as a PDF or view online for free
1) The document presents a new compression-based bound for analyzing the generalization error of large deep neural networks, even when the networks are not explicitly compressed. 2) It shows that if a trained network's weights and covariance matrices exhibit low-rank properties, then the network has a small intrinsic dimensionality and can be efficiently compressed. 3) This allows deriving a tighter generalization bound than existing approaches, providing insight into why overparameterized networks generalize well despite having more parameters than training examples.

Aman's AI Journal • Papers List

Co-clustering of multi-view datasets: a parallelizable approach

Learning group variational inference

ICLR 2020 Statistics - Paper Copilot

ICLR 2020

Higher Order Fused Regularization for Supervised Learning with Grouped Parameters

PDF) FEDZIP: A Compression Framework for Communication-Efficient Federated Learning

Jokyokai2

Stochastic Alternating Direction Method of Multipliers

Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network

Neural material (de)compression – data-driven nonlinear dimensionality reduction