Databricks, a company focused on helping businesses develop custom AI models, has introduced an innovative method to enhance AI performance despite the common challenge of using imperfect data. Jonathan Frankle, the company’s chief AI scientist, highlights a prevalent issue: organizations often struggle with "dirty data". Many businesses possess data but face difficulties fine-tuning AI models without access to clean, organized data sets.
To address this, Databricks has developed a technique that allows companies to eventually deploy AI agents for various tasks without being hindered by data quality issues. This method employs a combination of reinforcement learning and synthetic training data, thus allowing models to improve through practice and generate necessary training data using AI-generated inputs.
The company’s approach is built on the concept of "best-of-N", where even a weak model can achieve satisfactory results given repeated attempts. By training a model to predict which outcomes human testers would prefer, Databricks has created a reward model, known as the DBRM, that refines the performance of existing models without requiring new labeled data.
Databricks has referred to this technique as Test-time Adaptive Optimization (TAO). This method integrates lightweight reinforcement learning to enhance models, making them more effective without the need for extensive clean data. It has shown improved performance with larger models, indicating the technique’s scalability and effectiveness.
For instance, when tested on FinanceBench, which evaluates how well AI models answer financial queries, the method improved the performance of the Llama 3.1B model, allowing it to outperform OpenAI’s GPT-4o and o3-mini models. Experts, including computer scientist Christopher Amato, recognize the potential of the TAO approach, highlighting its ability to tackle the ongoing challenge of training AI models with sparse or low-quality data.
Real-world applications of this technology are already emerging. One health app developer improved its AI model’s reliability using TAO, illustrating the method’s practicality in sensitive contexts where accuracy is essential.
Overall, Databricks is paving the way for businesses to unlock the full potential of AI models, even when faced with the complexities of data quality.
