News

OpenAI works on Foundry, a platform to run your ML models on dedicated instances

Open AI, the creator of ChatGPT, is already working on its next product. Is about Foundry, a platform for developers who will allow the execution of the company’s machine learning modelslike GPT-3.5, on dedicated instances. For now, only a small number of system users have access to the platform, which is described as “designed for innovative customers running larger workloads«.

In the documentation of the service, according to Techcrunch, it appears that the platform allows “inference at scale, with complete control of model configuration and performance profile«. For now there is not much more information about the service, except for several screenshots that its users have shown.

It follows from these that, if Foundry ultimately has the features and functions that appear in it, it will offer a “static location” of computing capacity for a single customer. Maybe in Azure, which is the most popular public cloud platform in OpenAI.

Foundry users will be able to monitor specific instances with the same tools and dashboards they use in OpenAI to develop and optimize models. Additionally, Foundry will offer some type of version control, allowing its customers to decide whether or not to upgrade to new releases of updated models. It will also provide a more robust fine-tuning for the latest OpenAI models.

On top of all this, Foundry will offer instance uptime service level commitments. Also scheduled engineering support. Rentals will be based on dedicated computing units, with a three-month or one-year contract commitment. Running an instance with a single model will require a certain number of compute units, which will depend on several factors.

Of course, renting and using the instances with Foundry is not going to be exactly cheap. Running a light version of GPT-3.5 will cost $78,000 every three months, or $264,000 a year. These prices, and the launch of Foundry, could respond to growing pressure from Microsoft, after its investment in OpenAI, for the company to make a profit, complicated above all by the high computing costs they have to face to train their software. AI models, as well as to carry out their developments.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *