AI vs. Lawyer: Who Wins the Contract Analysis Bout?

The legal landscape is undergoing a dramatic shift as AI takes centre stage. The introduction of AI-driven legal support has massively impacted the way procurement and supplier management teams approach contract management and analysis. Once reliant on time-consuming manual analysis by in-house or third-party legal teams, Contract managers are now looking to be replaced or augmented by advanced AI tools such as Large Language Models (LLMs).

In this whitepaper, we put legal professionals head-to-head with an advanced AI solution: Ask Brooklyn, our platform’s built-in AI assistant. We test its ability to deliver the kind of legal insight that was once the sole domain of human lawyers. As AI continues to gain traction in the legal sector, understanding how it performs against traditional methods is critical for legal departments, procurement teams, and technology providers alike. So, when both contenders step into the ring, who will emerge as the contract analysis champion?

Introduction

Amongst many occupations, the legal profession is experiencing a significant transformation, driven by the innovations of Artificial Intelligence (AI).  This has impacted their daily admin and resource requirements as businesses begin to use Large Language Models for legal support opposed to seeking human resources.

An area that has been impacted by this transformation is Supplier and contract management. Before AI, reviewing contracts was a labour-intensive process requiring specialised legal expertise that organisations would use internal legal teams in-house or outsource to third-party legal entities for support. This process would become time-consuming and labour-intensive for all stakeholders involved. With the advancement of Large Language Models, there is growing evidence that these AI systems can perform specific legal tasks with comparable or superior performance to human lawyers in particular contexts.

Ask Brooklyn, our AI Assistant, is powered by Claude LLM Model, built into our Customer-Supplier Management Platform. Built into the platform to assist users in effectively managing their vendors and navigating complex contracts. The AI Assistant enables users of the platform to review specific clauses, summarise entire documents and interpret legal language into plain English, helping users make sense of everything from data processing agreements to compliance requirements under regulations. It is a practical tool to make contract review and analysis more manageable and transparent.

But how is AI changing the way businesses use qualified lawyers for contract analysis, and is it just as effective? Within this whitepaper, we will comprehensively analyse the Large Language Model (LLM) Ask Brooklyn to assess how this AI Assistant benchmarks against a qualified lawyer in contract analysis. This benchmark assessment will be through key performance indicators: Accuracy, speed, cost-efficiency, and practical applications.

So, Ask Brooklyn and a Qualified Lawyer are in the ring, but who is going for the knockout?

About the Author

Nick Francis, Chief Technology and Marketing Officer at Brooklyn Solutions

Nick Francis is a well-established and experienced CxO delivering Digital & Security-focused Transformation through the design, build, and deployment of cost-effective, highly automated industry-leading solutions. Nick has experience working across the private and public sectors in industries such as Financial Services, Insurance, Legal, Utilities, Retail, Public Sector and Government. Specialised in transformation activity to optimise processes, operational expenditure, and increase productivity. Significant experience in compliance, risk & control activities in highly regulated industries, standardisation of technologies, streamlining of internal processes and continuous improvement driving consistency and efficiency across an organisation whilst holding Customer, Colleague and Partner experience at a premium.