Evaluating small and large language models for natural language inference : a performance–efficiency analysis

Abstract

Large Language Models (LLMs) have demonstrated strong performance in Natural Language Processing tasks, but their computational and environmental costs remain significant challenges for scalable deployment. This study evaluates the trade-offs between performance and efficiency by comparing Small Language Models (SLMs) and a Large Language Model (LLM) baseline in Natural Language Inference (NLI) tasks. Experiments were conducted on the SNLI dataset using instruction-tuned decoder-only models under zero-shot, few-shot, and parameter efficient fine-tuning configurations. The evaluation considered both predictive performance metrics, including accuracy and F1-score, and efficiency-related measures such as inference latency, memory consumption, energy usage, and estimated carbon emissions. Additionally, robustness analyses were performed through semantically equivalent input perturbations and repeated inference consistency tests. The results show that compact language models achieve competitive performance in NLI tasks. Specifically, following parameter-efficient fine-tuning, all evaluated SLMs surpassed the zero-shot baseline of the selected 120B-parameter LLM, achieving accuracy gains of up to 12.7%, with models like Mistral 7B improving from 55.32% to 87.00%. Furthermore, fine-tuned architectures in certain cases offered significant reductions in computational and environmental costs, such as the Phi-3 3.8B model requiring nearly four times less energy than its base version. These findings suggest that smaller architectures represent a practical avenue for exploring computational efficiency in NLI applications when resources are limited.