Today at the Microsoft STRIKE event: “STRIKE Live: Practical AI Safety and Security,” Eric Douglas, CVP, Security Research, Microsoft, gave the opening remarks, and Yonatan Zunger, CVP, AI Safety and Security, Microsoft, delivered the keynote to a large group of Microsoft engineers. They emphasized that safety must become as fundamental to our work as breathing.
So, what are the basic principles of safety engineering?
1. Know the ways your system might fail as intimately as the ways your system should work:
• Brainstorm failure scenarios and keep that list as fresh as your success scenarios.
• What you don’t know can hurt you – so use many eyes and plan for surprises.
2. For each scenario, have a plan:
• Eliminate it.
• Reduce its severity or frequency.
• Give users a way to solve it themselves.
• Have a response plan for when things go wrong.
How do you do that brainstorming? Eric and Yonatan recommend a three-pronged approach:
• System-first: What are the components? What happens if each one fails? What if it gets bad input? And the components include the users!
• Actor-first: What might someone want to achieve using this software? Under what circumstances are they using it?
• Target-first: Who might be affected by someone using this software? What might make them more or less vulnerable? How would they be able to respond?