Baseten Launches Chains: The Framework Built for Compound AI Systems

SAN FRANCISCO, June 27, 2024 /PRNewswire/ -- Baseten, the leader in high-performance AI inference, is excited to announce the beta release of Chains. Built on the foundations of Baseten's open-source Truss framework, Chains enhances the performance of products using multiple AI models for compound AI systems, offering unmatched efficiency and scalability for businesses deploying complex AI inference workflows.

Compound AI Systems Require Multi-Model Inference

The landscape of artificial intelligence is expanding at an unprecedented rate, with companies increasingly reliant on multiple AI models running on heterogeneous hardware. Chains addresses the complexities of managing and optimizing these models at scale, ensuring businesses can maximize their AI investments without the usual performance bottlenecks.

"Chains represents a leap forward in our mission to help companies ship great AI products," said Tuhin Srivastava, CEO of Baseten. "The next generation of products will use many different AI models combined with developer code, and Chains provides the framework and infrastructure needed to make those products excellent. With Chains, you can select the right GPU or CPU for every model or code component in your Chains workflow and scale them independently for the highest performance at the lowest cost."

Key Benefits of Chains:

• Heterogeneous GPU Resourcing: Allows customers to select GPU and CPU resources for each component of a Chains workflow and auto-scale them independently.
• Business Logic Integration: Enables developers to orchestrate business logic with their ML models within a single Python program.
• Improved Developer Experience: Provides code-checking and type-checking by default to eliminate typical mistakes within complex workflows.
• Comprehensive Monitoring: Offers real-time insights into performance, resource utilization, and operational metrics, empowering businesses with actionable intelligence.
• Reduced Latency: Chains removes boilerplate code and unnecessary network hops to increase throughput and low latency at each step of a multi-model workflow.

Evolving the Truss Framework for Multiple Models

Building on the principles of the open-source Truss framework, Chains is its natural evolution, specifically engineered to address the heightened demands of multi-model, compound AI system deployments. Chains improves on Truss to provide a unified framework for orchestrating complex AI workflows spanning multiple component models, types of hardware, and arbitrary code in a way that prioritizes performance and efficiency at scale.

"It's a fact that the more moving pieces you have in a product, the more critical it is that all of their interactions are fast and reliable. For companies like our customers that are using multiple AI models to power their core products, it becomes existential," noted Amir Haghighat, CTO of Baseten. "We built Chains to help our customers power their multi-model use cases cost-efficiently, with low latency and high throughput. Chains builds on all the great work we've done for customers to improve inference performance and applies it to multi-modal use cases."

Get Started With Chains Today

Chains is currently available for beta testing. Interested parties can sign up for a free account today or request a personalized demo at baseten.co.

"Products using multiple AI systems are the future," said Tanay Kothari, Founder and CEO of Wispr AI. "Handling the orchestration of these models in a way that pushes latency as low as possible is critical for building seamless user experiences. We're excited that Chains is addressing this problem head-on."

About Baseten

Baseten is the leader in infrastructure software for high-scale AI products, offering the industry's most powerful AI inference platform. Committed to delivering exceptional performance, reliability, and cost-efficiency, Baseten is on a mission to help the next great AI products scale. Top-tier investors, including IVP, Spark, Greylock, Conviction, Base Case, and South Park Commons back Baseten. For further details, please visit baseten.co or follow us on X (fka Twitter) at @basetenco.

View original content to download multimedia:https://www.prnewswire.com/news-releases/baseten-launches-chains-the-framework-built-for-compound-ai-systems-302184383.html

SOURCE Baseten