“For example, you might have a game developer using an LLM to generate game character descriptions or parameters on the fly. For that to work, the LLM simply ‘must’ generate that information reliably in valid JSON format. Right now, developers have to rely on third party tools like Pydantic, Zod, LangChain, etc. to do things like iteratively run the same prompt until the inference returns data that’s usable, which is not a great solution,” Shimmin said.
What developers want is both in-model function calling, where the model’s syntax incorporates this functionality, and a model that itself knows how to work with APIs and formatting languages like JSON properly, the analyst explained, citing the example of Phi-based model NuExtract.
Why Salesforce is ‘open sourcing’ these products?
Analysts believe that Salesforce is releasing these products as “open source” in order to gain market share for capabilities such as Agentic actions.
“By making this tool available to developers and researchers, Salesforce aims to accelerate the development and refinement of function calling models, potentially leading to more robust and reliable AI applications,” Hinchcliffe said, adding that the open-sourcing of APIGen can also lead to more specialized and efficient AI solutions that are better tailored to the unique needs of various industries, enhancing business operations and customer experiences.