[Web summary, paraphrased by the fetch tool; not the original abstract] Introduces AgentBench, a benchmark of eight environments testing how well LLMs act as agents in reasoning and decision-making tasks. Top commercial models perform strongly while many open-source models up to 70B lag well behind. Failures are attributed mainly to weak long-term reasoning, decision-making and instruction following; better instruction following and high-quality multi-round alignment data may help, and training on code has mixed effects across agent tasks. Datasets, environments and the evaluation package are publicly released.
[Web summary, paraphrased by the fetch tool; not the original abstract] Language models solve new tasks from few examples but struggle with arithmetic and factual lookup. Toolformer teaches itself, self-supervised with a handful of demonstrations per API, to decide which tools to call, when, with what arguments, and how to use results. Tools include a calculator, a Q&A system, two search engines, a translation system and a calendar. It improves zero-shot performance on many downstream tasks, often matching much larger models, while keeping language modeling ability.