An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures - IRT Saint Exupéry - Institut de Recherche Technologique
Pré-Publication, Document De Travail Année : 2025

An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures

Résumé

Orthogonal convolutional layers are the workhorse of multiple areas in machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitzconstrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptative Orthogonal Convolution), a scalable method for constructing orthogonal convolutions, effectively overcoming these limitations. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source library implementing this method, available at https://github.com/thib-s/orthogonium.
Fichier principal
Vignette du fichier
main.pdf (2.86 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04883528 , version 1 (13-01-2025)

Licence

Identifiants

  • HAL Id : hal-04883528 , version 1

Citer

Thibaut Boissin, Franck Mamalet, Thomas Fel, Agustin Martin Picard, Thomas Massena, et al.. An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures. 2025. ⟨hal-04883528⟩
0 Consultations
0 Téléchargements

Partager

More