Evaluating the Reliability of Large Language Models in Political Forecasting: A Case Study of Nepal's 2026 Parliamentary Election

This article conducts a thorough examination of four prominent language models—GPT-4, Claude 3.5 Sonnet, Google Gemini 1.5 Pro, and DeepAI—right before Nepal's parliamentary election set for March 5, 2026.The researchers employed a series of standardized prompts to prompt these models to generate predictions regarding seat distributions, examine various scenarios, assess reasoning depth, identify biases, evaluate coalition commitments, and even critique their own predictions. They collected all this information prior to the election and then contrasted it with the actual outcomes at the voting stations. In this election, the Rastriya Swatantra Party (RSP) achieved a significant victory by securing 182 out of 275seats, marking a historic moment as Nepal’s first single-party majority under the 2015 constitution. What’s surprising is that all four models forecasted the Nepali Congress (NC) would lead and significantly undervalued the RSP by approximately 121 to 152 seats, In terms of accuracy, their forecasts were significantly inaccurate—the mean absolute error ranged from 46.0 to 77.8 seats, and Brier scores fell between 0.144 and 0.423.None of these models were able to accurately predict the coalition results either. These results prominently underscore a predisposition towards traditional parties; it appears that the models relied excessively on past trends, overlooking the new dynamics present in Nepal's political landscape. Ultimately, this highlights considerable constraints in employing language models for election predictions and prompts crucial inquiries regarding their more effective application in journalism, policy development, and assisting voters in making knowledgeable choices.