From MLOps to LLMOps: A Position on Engineering Discipline for AI-Native Applications

Abstract

Artificial intelligence (AI) has rapidly become an integral part of the engineering world, moving from traditional software development towards increasingly intelligent, adaptive, and autonomous applications. Machine Learning Operations (MLOps) has evolved as an established engineering approach for deploying, monitoring, and maintaining machine learning systems in production, but the advent of large language models (LLMs) has brought new operational characteristics that go beyond what is currently known in MLOps. LLM-based systems inherently depend on the design of prompts, the external foundation models, the retrieval processes, the contextual information that is needed to be dynamic, and the probabilistic results that cannot always be guaranteed. These properties pose novel engineering problems in terms of lifecycle management, reliability, cost of operations, execution of autonomous activities, and assessment of the non-deterministic responses. This paper proposes to treat LLMOps as a new discipline, not just a new buzzword for MLOps. It provides a conceptual position for LLMOps, explains what it means to incorporate the scope of engineering into LLMOps, and outlines five engineering practices which are essential for the successful operation of LLMOps compared to traditional MLOps: prompt versioning, fidelity grounding, cost control, action authorization, and assessment for non-deterministic outputs. The paper also brings up implications for the future transition of the software engineering research and industrial practice and proposes a research agenda for AI-native applications. By treating LLMOps as a standalone engineering discipline, there is a more solid conceptual basis for AI systems that are reliable, secure, scalable, and economically viable to support more and more autonomous software applications.