World Models in a Cognitive Architecture

LIDA is a cognitive architecture built on Baars’ Global Workspace Theory: it models cognition as a repeating cycle of perception, attention, and action. Modern reinforcement learning has its own answer to how an agent should model the world, in the line of work running from Dreamer to JEPA. This project, with Dr. Sean Kugele, puts the two together.

The world model encodes observations with a beta-VAE and predicts forward in latent space with a GRU. In PyTorch it improved next-state latent prediction by 18% across 50k+ simulated transitions. We then integrated a DreamerV3-style predictive agent with LIDA across 4 action spaces, which cut planning inference latency by 25%. A manuscript is in progress.

Earlier pieces of the same thread: a Gumbel-Softmax discrete beta-VAE on a polyomino dataset, and smaller latent-space experiments on MNIST and Pong next-frame prediction.

The question I keep coming back to is whether imposing architectural structure on a learning system makes it more sample-efficient, more interpretable, or neither. I’m not convinced yet.