The Anthropic red-team dataset is a significant open-access dataset aimed at improving AI safety through training preference models and assessing their safety.
Language Model Agents (LMAs) are emerging as a powerful primitive for augmenting red-team operations. They can support attack planning, adversary emulation, and the orchestration of multi-step activit