Auto-Transfer: Learning to Route Transferrable Representations

No Thumbnail Available
Authors
Murugesan, Keerthiram
Sadashivaiah, Vijay
Luss, Ronny
Shanmugam, Karthikeyan
Chen, Pin-Yu
Dhurandhar, Amit
Issue Date
2022-03-15
Type
Article
Language
en_US
Keywords
Feature routing , Transferable Representations
Research Projects
Organizational Units
Journal Issue
Alternative Title
Abstract
Knowledge transfer between heterogeneous source and target networks and tasks has received a lot of attention in recent times as large amounts of quality labeled data can be difficult to obtain in many applications. Existing approaches typically constrain the target deep neural network (DNN) feature representations to be close to the source DNNs feature representations, which can be limiting. We, in this paper, propose a novel adversarial multi-armed bandit approach that automatically learns to route source representations to appropriate target representations following which they are combined in meaningful ways to produce accurate target models. We see upwards of 5% accuracy improvements compared with the state-of-the-art knowledge transfer methods on four benchmark (target) image datasets CUB200, Stanford Dogs, MIT67, and Stanford40 where the source dataset is ImageNet. We qualitatively analyze the goodness of our transfer scheme by showing individual examples of the important features focused on by our target network at different layers compared with the (closest) competitors. We also observe that our improvement over other methods is higher for smaller target datasets making it an effective tool for small data applications that may benefit from transfer learning.
Description
Full Citation
Murugesan, K., Sadashivaiah, V., Luss, R., Shanmugam, K., Chen, P. Y., & Dhurandhar, A. (2021, September). Auto-Transfer: Learning to Route Transferable Representations. In International Conference on Learning Representations.
Publisher
International Conference on Learning Representations
Journal
Volume
Issue
PubMed ID
DOI
ISSN
EISSN