Note Speeding up transformer based video models but reducing complexity of attention. from O((T*H*W)**2) to O((H*W)**2 + T**2)

Note Future embedding prediction from ViT model as latent world model

Note Decode all actions from same latent embedding with a transformer instead of linear / MLP

Note For smoother prediction of actions, predict past and future actions from a given latent embedding

Note Decode future frames from latent foresight token embedding. Decode actions from latent foresight

Note Control problem is a long sequence of State, Actions, Reward embedding from offline data

Note Control problem is a long sequence of State, Actions, Reward embedding from offline data

Note train multimodal causal transformer from different environments (atari etc) add a policy head for control given task