Over the past few years, AI systems have become mainstream. Among them are large language models (LLMs), which produce seemingly intelligent responses from chatbots. But one thing they all lack is the ability to continue learning as they are in use, a drawback that prevents them from growing more accurate as they are used more. They also are unable to grow any more intelligent by training on new datasets.
The researchers tested the ability of conventional neural networks to continue learning after training on their original datasets and found what they describe as catastrophic forgetting, in which a system loses the ability to carry out a task it was able to do after being trained on new material.
