Continual Learning in Deep Learning

Human brain learns new tasks while retaining prior knowledge, but artificial intelligence models suffer from catastrophic forgetting when trained sequentially. Catastrophic forgetting occurs when new information overwrites previously learned information. Effective continual learning requires a careful balance between the model's stability (ability to preserve existing knowledge) and plasticity (capacity to learn new information). This thesis develops methods to enhance this balance in deep neural networks. Drawing inspiration from the brain’s learning process, we introduce a bidirectional knowledge-transfer mechanism and propose parameter-efficient continual learning strategies that extend to large pre-trained models.