Exploring the Impact of Additive Shortcuts in Neural Networks via Information Bottleneck-like Dynamics: From ResNet to Transformer
Deep learning has made significant strides, driving advances in areas like computer vision, natural language processing, and autonomous systems.In this paper, we further investigate the implications of the role of additive shortcut connections, focusing Horse Exercise Sheets on models such as ResNet, Vision Transformers (ViTs), and MLP-Mixers, give