Interest in AI among the enterprise continues to rise, with one recent one survey found that nearly two-thirds of companies plan to increase or maintain their spending on AI and machine learning this year. But often, these companies encounter roadblocks in deploying different types of AI in production.
Inspired to tackle the challenges, Salman Avestimehr, founding director of the USC-Amazon Center on Trustworthy Machine Learning, co-founded a startup to allow companies to train, deploy, monitor and improve AI models on the cloud or on the edge. Called FedMLraised $11.5 million in seed funding at a valuation of $56.5 million led by Camford Capital with participation from Road Capital and Finality Capital.
“Many businesses are eager to train or fine-tune custom AI models on company- or industry-specific data, so they can use AI to address a range of business needs,” Avestimehr told TechCrunch in email interview. “Unfortunately, custom AI models are prohibitively expensive to build and maintain due to high data, cloud infrastructure and engineering costs. In addition, the proprietary data to train custom AI models is often sensitive, controlled or siled.”
FedML overcomes those barriers, Avestimehr claims, by providing a “collaborative” AI platform that allows companies and developers to work together on AI tasks by computing data, models and resources.
FedML can run several models or custom AI models from the open source community. Using the platform, customers can create a collaborative group and synchronize AI applications across their devices (eg PCs) automatically. Collaborators can add devices to use for AI modeling training, such as servers or even mobile devices, and track training progress in real time.
Recently, FedML implemented FedLLM, a training pipeline for building large “domain-specific” language models (LLMs) and OpenAI’s GPT-4 on proprietary data. Compatible with popular LLM libraries such as Hugging Face’s and Microsoft’s DeepSpeeds, FedLLM is designed to improve the speed of custom AI development while preserving security and privacy, Avestimehr says. (To be clear, the jury is out on whether or not it is.)
In this way, FedML doesn’t differ much from the other MLOps platforms out there – “MLOps” referring to tools for streamlining the process of bringing AI models to production and then maintaining and monitoring them on them. Galileo and Arize are there as well as Seldon, Qwak and Comet (to name a few). Holders like AWS, Microsoft and Google Cloud offer MLOps tools in some form or another (see: SageMaker, Azure Machine Learning, etc.)
But FedML has ambitions beyond tooling AI and developing a machine learning model.
As Avestimehr tells it, the goal is to build a “community” of CPU and GPU resources to host and serve models when they are ready to deploy. The details haven’t worked yet, but FedML plans to incentivize users to calculate contributions to the platform through tokens or other forms of compensation.
Distributed, decentralized computing for serving the AI model is not a new idea – Gensys, Run.AI and Petals are among those who have tried – and are trying – it. However, Avestimehr believes FedML can achieve greater reach and success by combining this computational paradigm with a set of MLOps.
“FedML enables custom AI models by empowering developers and enterprises to build large-scale, proprietary and private LLMs at a lower cost,” Avestimehr said. “What sets FedML apart is the ability to train, deploy, monitor and improve ML models anywhere and collaborate on the combined data, models and computation – significantly reducing the cost and time to market.”
To date, FedML, which has a workforce of 17 people, has paid about 10 customers, including a “tier one” automotive supplier and a total of $13.5 million in the highest amount, including the new financing. Avestimehr claims that the platform is being used by more than 3,000 users worldwide and doing more than 8,500 training jobs across more than 10,000 devices.
“For the data or technical decision maker, FedML makes customized, affordable AI and large language models a reality,” Avestimehr said, confidently. “And thanks to its foundation in federated learning technology, its MLOps platform and collaborative AI tools that help developers train, serve and observe custom models, building custom alternatives is an accessible best practice.”