At the 2023 Defcon hacker conference in Las Vegas, prominent AI tech companies joined forces with groups focusing on algorithmic integrity and transparency to challenge generative AI platforms with the help of thousands of conference attendees. This initiative was to identify vulnerabilities in these vital systems, supported by the US government as part of a “red-teaming” effort. This approach aims to examine these influential but not easily understandable systems. Continuing this effort, the nonprofit Humane Intelligence, dedicated to ethical AI and algorithmic evaluation, introduced a partnership on Wednesday. They are collaborating with the US National Institute of Standards and Technology, opening invitations to all US residents to join a nationwide initiative assessing AI office productivity tools in a red-teaming event.
The initial qualifier round is accessible online to both developers and the wider public as part of NIST’s AI challenges titled Assessing Risks and Impacts of AI, or ARIA. Those who successfully pass the online qualifications will participate in a live red-teaming event scheduled for the end of October at the Conference on Applied Machine Learning in Information Security (CAMLIS), in Virginia. The purpose of this event is to enhance the ability to conduct thorough tests on the security, resilience, and ethical aspects of generative AI technologies.
“The average person using these models typically isn’t capable of determining their suitability for specific purposes,” explained Theo Skeadas, CEO of Tech Policy Consulting, a firm specializing in AI governance and online safety collaborating with Humane Intelligence. “Therefore, our objective is to make evaluation processes accessible to everyone so that all users can independently verify whether the AI models satisfy their requirements.”
The culmination of this event at CAMLIS will involve participants being divided into a red team, tasked with compromising the AI systems, and a blue team, assigned to defend them. The teams will use NIST’s AI risk management framework, known as AI 600-1, to determine the effectiveness of the red team in producing results that deviate from what is typically expected from the systems.
“NIST’s ARIA is drawing on structured user feedback to understand real-world applications of AI models,” says Humane Intelligence founder Rumman Chowdhury, who is also a contractor in NIST’s Office of Emerging Technologies and a member of the US Department of Homeland Security AI safety and security board. “The ARIA team is mostly experts on sociotechnical test and evaluation, and [is] using that background as a way of evolving the field toward rigorous scientific evaluation of generative AI.”
Chowdhury and Skeadas say the NIST partnership is just one of a series of AI red team collaborations that Humane Intelligence will announce in the coming weeks with US government agencies, international governments, and NGOs. The effort aims to make it much more common for the companies and organizations that develop what are now black-box algorithms to offer transparency and accountability through mechanisms like “bias bounty challenges,” where individuals can be rewarded for finding problems and inequities in AI models.
“The community should be broader than programmers,” Skeadas says. “Policymakers, journalists, civil society, and nontechnical people should all be involved in the process of testing and evaluating of these systems. And we need to make sure that less represented groups like individuals who speak minority languages or are from nonmajority cultures and perspectives are able to participate in this process.”
