NIST Alerts AI Developers: Beware of Poisoning Methods and Cyber Threats

January 16, 2024
1 min read

In a new guideline paper, the National Institute of Standards and Technology (NIST) has highlighted the potential cyber threats that AI developers may face during the development and deployment of their models. The paper focuses on “poisoning” methods, where training data is tainted to manipulate the learning model, as well as “evasion” attempts to confuse AI already in use and prompts used by cyber threats to “jailbreak” the models. The paper also highlights the vulnerability of AI models during the learning phase, as they rely on large volumes of public data, which may contain misinformation. The paper suggests that while developers cannot completely secure their models, they should carefully consider potential attack sources and approaches to make trade-offs between capability and security.

The paper also discusses the risks of altering the source code of AI models, as many developers use open source components or third-party libraries. It highlights the difficulty of curating the large volume of information required for training AI models and the risk of unintentional self-poisoning as models generate synthetic content. The paper acknowledges that there are no foolproof methods for curbing these threats and recommends mapping out anticipated attack sources and approaches.

Latest from Blog

Top CISA official looks back on four years of cyber work

TLDR: Eric Goldstein, a top official at CISA, reflects on progress made in cybersecurity during his tenure. Key achievements include understanding cyber risks, collaboration with industry, and encouraging secure product development. Eric

Juggling AI cybersecurity highs and lows

TLDR: At the 2024 MIT Sloan CIO Symposium, industry leaders discussed the challenge of balancing AI’s benefits with its security risks, particularly focusing on generative AI. While generative AI can bring benefits

Get your free Cyber Security eBook now Valued at $169

“`html TLDR: Key Points: Claim your complimentary eBook worth $169 for free before May 22. The eBook covers practical applications of cyber security and network security for professionals, engineers, scientists, and students.