At the 2023 Defcon hacker conference in Las Vegas, prominent AI tech companies collaborated with algorithmic integrity and transparency groups to engage thousands of participants in testing generative AI platforms to discover vulnerabilities in these crucial systems. This initiative of “red-teaming,” also backed by the US government, aimed at exposing these influential yet non-transparent systems to broader examination. Following up on this initiative, the ethical AI and algorithmic evaluation nonprofit Humane Intelligence recently announced a collaboration with the US National Institute of Standards and Technology. They opened a call for participants for a U.S. nationwide red-teaming exercise to assess AI office productivity software.
The initial qualification round will be conducted online, open to both developers and general public participants as part of NIST’s AI challenges under the Assessing Risks and Impacts of AI (ARIA) program. Those who qualify will attend a live red-teaming event at the end of October at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia, aiming to enhance capacity for thorough testing of generative AI technologies’ security, resilience, and ethical standards.
“Many users of these models are not equipped to judge if the model serves its intended purpose or not,” explained Theo Skeadas, CEO of the AI governance and online safety group Tech Policy Consulting, who are in collaboration with Humane Intelligence. “Therefore, we aim to democratize the evaluation process to ensure that all users can determine the efficacy of these models on their own.”
During the final CAMLIS event, participants will be divided into a red team attempting to exploit the AI systems and a blue team strategizing defense. The teams will utilize NIST’s AI risk management framework, dubbed AI 600-1, to gauge the red team’s success in undermining the systems’ anticipated operations.
“NIST’s ARIA is leveraging structured user input to gain insight into the practical usage of AI technologies,” states Rumman Chowdhury, founder of Humane Intelligence, who is also engaged with NIST’s Office of Emerging Technologies and part of the US Department of Homeland Security AI safety and security board. “The ARIA team primarily consists of experts in sociotechnical testing and evaluation, utilizing their expertise to propel the domain towards a more rigorous scientific assessment of generative AI.”
Chowdhury and Skeadas mention that this collaboration with NIST is only the beginning of a range of AI red team partnerships that Humane Intelligence will be revealing soon involving US government bodies, global governments, and NGOs. The initiative is designed to encourage entities that create currently opaque algorithms to provide transparency and accountability through initiatives such as “bias bounty challenges,” which offer rewards to those who identify flaws and biases in AI systems.
“The involved community should extend beyond software developers,” argues Skeadas. “It’s important that policymakers, journalists, elements of civil society, and people without technical backgrounds are all included in the scrutiny and evaluation of these systems. Additionally, it’s crucial that underrepresented groups, including speakers of minority languages or those from minority cultural backgrounds and perspectives, are given opportunities to engage in this evaluation process.”
