New framework and set of capabilities for benchmarking and red teaming AI evaluation

DataRobot, the enterprise AI platform leader, today announced the integration of LLM evaluation measures aligned with a new initiative from the Singapore Government Agency, Infocomm Media Development Authority (IMDA). The “Project Moonshot” initiative unveiled at the Singapore Asia Tech x Summit offers new capabilities that help AI practitioners and system owners manage LLM deployment risks by providing a common framework for benchmarking and red teaming evaluation.

“At DataRobot, our focus is addressing the confidence gap and helping organizations scale responsible use of generative AI,” said Jay Schuren, Chief Customer Officer, DataRobot. “We’re excited to announce that our latest product release incorporates Project Moonshot’s testing toolkit and its benchmarking and evaluation tests. The result is that LLM evaluations are more accessible and help scale the responsible use of generative AI, enabling practitioners to turn on and configure guard models to change the behavior and responses of LLMs.”

Project Moonshot delivers three core capabilities for AI practitioners and system owners:

1. Automated evaluation tools for generative AI solutions that easily integrate into CI/CD pipelines.
2. A benchmark repository allowing teams to run evaluations relevant to their applications by curating the right benchmarks.
3. A one-stop tool for AI red teaming, from jailbreaks to customized attacks.

“The development of Project Moonshot, one of the world’s first open-source tools to bring red teaming, benchmarking and baseline testing together in an easy-to-use platform, would not have been possible without the contribution of partners such as DataRobot,” said, Dr Ong Chen Hui, Chair of the Governing Committee at AI Verify Foundation. “Project Moonshot will provide developers with an intuitive toolkit to test their LLM applications. This new toolkit signals Singapore’s continued commitment to advance the global open-source efforts toward addressing generative AI safety concerns.”

“We are proud to support our portfolio company, DataRobot, on its growth journey in Southeast Asia,” said Paul Ng, Chief Executive Officer, EDBI. “The company has not only expanded its footprint in Singapore but has also fostered collaborations that benefit the local innovation ecosystem, such as this partnership with IMDA. Project Moonshot provides local enterprises with the right tools to deploy generative AI technologies confidently. As a strategic investor, we are committed to creating value for our portfolio companies while enhancing Singapore’s innovation capabilities.”

DataRobot is an IMDA accredited company and a member of the AI Verify Foundation which launched the world’s first AI Governance Testing Framework and Toolkit in 2022.

To learn more about DataRobot’s latest AI observability and intervention capabilities, read our latest product announcement or visit datarobot.com.

About DataRobot

DataRobot is the enterprise AI platform leader, empowering organizations to accelerate AI from idea to impact. With over a decade at the forefront of AI innovation, we know what it takes to make a real difference – to your bottom line, to your business vision, and to the world around us. Our end-to-end AI platform allows your organization to quickly build, securely operate and confidently govern your entire AI landscape from a single, unified experience. Organizations across industries and geographies trust DataRobot to help solve their biggest challenges with AI, leveraging generative and predictive capabilities today while providing the flexibility to adapt to the innovations of tomorrow. Learn more at datarobot.com and connect with us on LinkedIn.

About IMDA

The Infocomm Media Development Authority (IMDA) leads Singapore’s digital transformation by developing a vibrant digital economy and an inclusive digital society. As Architects of Singapore’s Digital Future, we foster growth in Infocomm Technology and Media sectors in concert with progressive regulations, harnessing frontier technologies, and developing local talent and digital infrastructure ecosystems to establish Singapore as a digital metropolis. For more news and information, visit www.imda.gov.sg or follow IMDA on Facebook (IMDAsg) and Twitter (@IMDAsg).

About EDBI

Founded in 1991, EDBI is a leading Singapore-based global investor focusing on high-growth technologies across Information & Communication Technology, Emerging Technology and Healthcare sectors. With the aim of creating long-term value and enhancing capabilities of Singapore’s key industries, we partner with companies poised for expansion in Asia and beyond, leveraging our extensive network, resources, and expertise. Explore our portfolio of successful investments and discover how we can accelerate growth together, through Singapore. Learn more at www.edbi.com.

View source version on businesswire.com: https://www.businesswire.com/news/home/20240530035257/en/