LLM-Assisted Code Review in DevOps Workflows: Accuracy, Developer Trust, and Integration Patterns

Abstract

Large Language Models integrated into code review pipelines offer the promise of faster feedback cycles and broader coverage than human reviewers alone. This paper presents a 9-month field study of LLM-assisted code review deployed across 6 engineering teams totalling 210 developers. We evaluate review comment accuracy, false-positive rates, developer acceptance rates, and downstream defect escape rates. LLM suggestions were accepted at a rate of 61% and reduced escaped defects in production by 27%. Developers expressed significant trust calibration challenges: over-reliance on LLM suggestions was observed in 18% of reviewed pull requests. Guidelines for human-in-the-loop review governance and LLM prompt engineering for code quality are contributed.