Progressive Neural Networks (PNNs) are a fundamental architecture-based continual learning concept that offer explicit support for transferring knowledge across incremental tasks, creating dedicated sub-networks for each task and enabling lateral knowledge transfer.
The REPEAT continual learning algorithm is defined below: