Feb 12, 2023

Augmenting LLMs with APIs

Libraries like LangChain show how to use the output of LLMs to drive actions through external tools and APIs. I did a quick video on that last week:

Just a few days ago, the folks over at Meta AI released a paper that takes that much further, by incorporating external API calls into the training and inference steps of the LLM itself. Their system is called ToolFormer, and here’s a short video on it:

Their topline result is that when augmented with APIs in this way, smaller models like GPT-J (6B parameters) can outperform models like GPT-3 (175B parameters), that are an order of magnitude larger, on a variety of tasks.