Learning Interpretable Low-dimensional Representation via Physical Symmetry

[tatsuropfgt]

Learning Interpretable Low-dimensional Representation via Physical Symmetry [Liu+, NeurIPS'23]

Abst

• use physical symmetry as a self-consistency constraint for the latent space of time-series data
• the constraints lead the model to learn interpretable representation

Method

Overview

Loss

Latent embedding and S

• in the music problem, the latent embedding is 1-dim(pitch).
  • S means adding a random scalar to the latent code (R)
• in the video problem, the latent embedding is 3-dim(2-dim for the horizontal plane, 1-dim for the vertical height)
  • S means translations(adding a random scaler to the latent codes(R^2)) AND rotation the horizontal dimension
  • this means the ball in the 3D space has physical symmetry in the horizontal plane, but not in the vertical height axis

Result

Pitch(Audio)

• linear and noiseless mapping is ideal
• SPS(self-supervised learning with physical symmetry) leads the latent representation to have interpretable(pitch) embedding

Video

• more interpretable

Memo

• if the assumptions(in the latent embedding) are incorrect, the method can be helpful
• It can also be seen as the data augmentation in the latent space