Can you trust AI to Draft your Commercial Contracts?

In the first of a series of posts about AI, I am going to start by taking a look at the AI generated commercial contracts freely available from LLM (Large Language Model) based generative AI models.

If you are thinking about asking AI to produce a commercial contract for you, is it a good idea? There is definitely something strangely alluring about the confident simplicity of an AI answer. The blithe assertion that ‘you’re doing a great job’ as it supplies you with an instant response in a few short sentences is very appealing. It’s undeniably fast and perhaps looks ‘good enough’ to the untrained eye. I sometimes wonder if AI is in cahoots with one of the senior partners I worked with as a junior lawyer; I can still hear his words ‘Perfect is the enemy of good, Jonathan, don’t forget that. We don’t need War and Peace.’ 

While I expect most users are not looking for perfection in AI, they no doubt want something that is good enough, even if they are not paying for it. With this in mind, I have been tracking the effectiveness of AI generated contracts for a few years now. It has been interesting to watch its development.

Early Testing of AI: Where it fell short 

About six months ago I ran a request for a short form mutual non disclosure agreement (NDA) through one of the well known AI models. On the face of it, it looked like it would just about do the job, but on looking more closely I noticed that either party could terminate at any time and there were no post termination obligations. This made the NDA useless. I requested other more complex contracts, including a food manufacturing outsource agreement and this time it really struggled to produce anything near good enough. In fact, it was so bad, I felt confident that my job would be safe for some time yet. Next, a client approached me with a commercial agency agreement produced by AI. It initially seemed like it had got some clauses just about right, but those clauses were so short they risked being void for uncertainty. Of more concern was that it failed to cover some really key issues, including the question of indemnity v compensation for the agent on termination, a massive risk factor of which the client was blissfully unaware. So, six months ago I decided AI was not good enough, nowhere near.

Has AI improved since then? 

But, has it improved? Just yesterday I ran the same requests through the same AI model and two others. There were some interesting developments. With one AI model, all the clauses in every contract were much shorter this time, in many cases a mere aide memoire, and not detailed enough to be a legally binding provision. But on the flip side, the coverage of the range of issues had improved. In the NDA, all the AI examples had dealt with the post termination hole beneath the waterline. In the food manufacturing outsource agreement, one AI example had improved slightly on last time, but still suffered from inadequate coverage of key issues and insufficient detail on the issues it did cover. However, another example markedly improved its coverage of the issues, but at the same time had clearly retreated from trying to draft legally binding clauses, merely citing the issue to be considered. Most of the contracts were now prefaced with clearer warnings that these were ‘starting points’, and in some cases included a recommendation to seek legal advice.

So, it’s a mixed bag. Some of the AI generated contracts have improved as an information tool, while others may be shying away from trying to serve as a definitive and enforceable contract with legally binding clauses. As they stand though, they are still not good enough. If a solicitor produced the same contract, that solicitor would be negligent beyond doubt.

Perhaps I am merely defending my patch? Who else to ask but AI?  ‘Are AI generated commercial contracts comprehensive and reliable?’ I typed in. The answer back was ‘No. AI generated commercial contracts are not reliably comprehensive on their own. While AI can draft contracts quickly and efficiently, industry analysis shows they frequently contain significant flaws that make them unusable without human review.’ I was a little disappointed that it didn’t tell me it was a brilliant question, that I was brilliant to think of it, but I was mildly reassured that I was on the right lines. More to follow in my next post on this subject.

If you want a commercial contract that’s robust, compliant and tailored to your business, and has not been generated by AI, get in touch with me, Jonathan Heskia, Director in our experienced Corporate and Commercial team. 

The contents of this article is a general guide only at the date of publication. It is not comprehensive, and it does not constitute legal advice. Specific legal advice should be sought in relation to the particular facts of a given situation.