Experiemental Results on Language Model?

[XiaoBuL]

Thanks for your great work. The proposed Adapter-plus has been verified in the VTAB benchmark. I'm curious about the effect of Language Model, is there the same conclusion?

[jmsteitz]

Pfeiffer et al. [1] explored adapters for NLP. They performed an architecture search for the optimal adapter and found a position that formally does the same data transformation as the Post-Adapter in our paper. However, the inner configuration differs as they did not use any scaling but used LayerNorm inside the adapter.

[1] Pfeiffer et al. "AdapterFusion: Non-Destructive Task Composition for Transfer Learning," in EACL, 2021.