OpenAI is reaching out to contractors, requesting them to upload actual work assignments and tasks from their past or current jobs. This initiative is part of their strategy to evaluate the performance of their upcoming AI models, as reported by WIRED based on internal documents.
The aim is to establish a human performance baseline to benchmark against AI capabilities. In September, OpenAI introduced a new evaluation process to assess AI performance against that of human professionals in various fields, which it considers crucial for its goal of achieving Artificial General Intelligence (AGI).
According to a confidential document, OpenAI encourages contractors to submit concrete examples of their work, such as Word documents, PDFs, PowerPoint presentations, and more. They have been instructed to provide both the task requests—what their supervisors asked them to do—and the deliverables produced in response.
One guideline stipulates the necessity to anonymize or remove any private, proprietary, or sensitive information from the documents before submission. OpenAI emphasizes the importance of sharing “real, on-the-job work” that contractors have genuinely completed.
For instance, a task from a Senior Lifestyle Manager involved creating a PDF itinerary for a family’s yacht trip, showcasing the type of tasks OpenAI seeks.
Legal experts have raised concerns regarding the risks contractors face when sharing potentially confidential information, even after scrubbing sensitive details. The reliance on contractors to determine the confidentiality of data presents a significant risk to both the contractors and OpenAI.
AI labs have depended on firms like Surge, Mercor, and Scale AI to source data contractors. However, as the quality of training data becomes increasingly important, companies are investing more in skilled workers. This shift has led to a generation of lucrative opportunities within the AI training space, with valuations skyrocketing for companies like Handshake AI and Surge.
Furthermore, OpenAI has also explored alternative avenues for acquiring real company data from businesses going through liquidation, again raising concerns about privacy and data integrity.
For more detailed insights:
