Comparative Study of Human and LLM-Based Surrogate Modeling Using Tool-Integrated ReAct Agents

This paper presents a comprehensive comparison between a human expert and a tool-augmented Large Language Model (LLM) agent in fitting surrogate models using three metamodels: Gaussian Process, Neural Networks and QLattice. The LLM agent operates autonomously, leveraging different tools to facilitate model fitting, error verification, hyperparameter optimization, and visualization. We evaluate the performance of both a human expert and the LLM agent across multiple metrics, including model performance and sample efficiency. The findings of this study provide valuable insights into the capabilities and limitations of LLM agents in surrogate model fitting, on both synthetic datasets and real datasets for production of the integrated circuits industry.