In a recent speech, the Master of the Rolls suggested that the day may soon come when lawyers may be negligent if they fail to use generative artificial intelligence. Costs Counsel Andrew Hogan has been writing some interesting blog posts about the use of ChatGPT and shown some impressive looking content that it is able to produce.
There have, of course, been stories about AI “hallucinating” and producing completely made up legal authorities.
I thought I would give it a spin based on a topic dealt with in one of my recent posts.
Two things to note:
- This is based on the free version of ChatGPT (version 3.5). There is a premium version (4) which may have produced different results.
- I understand that ChatGPT was trained on data up to January 2022. We therefore cannot expect it to provide answers on the latest rules or case law.
Let’s get started. This shows the questions I typed in and the answers ChatGPT provided in response.
Me:
“In English law, is an order for an interim costs payment an order that can be enforced?”
ChatGPT:
“Yes, in English law, an order for an interim costs payment is indeed an order that can be enforced. When a court issues an interim costs payment order, it means that one party is required to make a payment to the other party on account of the costs they have incurred during the litigation process up to that point.
The enforceability of such an order typically depends on the specific terms set forth by the court. If the order specifies a particular amount and a deadline for payment, failure to comply with these terms could result in enforcement action. Enforcement mechanisms may include seeking a court order to compel payment, applying for a charging order against the defaulting party’s assets, or pursuing other legal remedies available under the relevant procedural rules.
It’s important to note that failure to comply with a court order, including an order for an interim costs payment, can have serious consequences, including potential sanctions or adverse costs orders against the defaulting party. Therefore, parties are generally expected to adhere to court orders promptly and fully, and failure to do so may lead to enforcement action by the aggrieved party.”
A nice clear answer. I could confidently convey that advice to clients. However, a judge would probably want something to back this up.
Me:
“What is the rule or authority in support of your view the order is enforceable?”
ChatGPT:
“The authority supporting the view that an order for an interim costs payment is enforceable in English law can be found in the Civil Procedure Rules (CPR) and relevant case law.
Under the CPR, specifically Rule 44.2(8), the court has the discretion to make an order for costs to be paid on account, either in whole or in part, before the conclusion of the case. This provision gives the court the authority to issue interim costs orders, which are considered enforceable court orders.
Furthermore, case law provides additional support for the enforceability of interim costs orders. In the case of Lownds v Home Office [2002] EWCA Civ 365, the Court of Appeal affirmed the principle that an order for payment of costs on account is enforceable. The court held that such orders are made for the benefit of the receiving party and are intended to alleviate financial hardship pending the final resolution of the case.
Additionally, the case of CIBC Mellon Trust Co Ltd v Stolzenberg [2003] EWCA Civ 454 established that failure to comply with an order for interim costs payment can result in enforcement action, including the possibility of contempt proceedings for non-compliance with a court order.
Overall, both the CPR and case law provide clear authority for the enforceability of orders for interim costs payments in English law.”
Now we are getting somewhere. Two clear Court of Appeal authorities in support. That can go straight into my skeleton argument and I am good to go. Still, perhaps it is worth diving a bit deeper just in case the judge starts asking any probing questions.
Me:
“Which passage in Lownds v Home Office [2002] EWCA Civ 365 are you relying on?”
ChatGPT:
“I apologize for the oversight. Upon review, it seems there isn’t a specific passage in Lownds v Home Office [2002] EWCA Civ 365 that directly addresses the enforceability of interim costs orders. Instead, the case primarily dealt with the principles surrounding the quantification of interim payments on account of costs.
However, the Court of Appeal’s decision in Lownds v Home Office reaffirms the general principle that orders for payment of costs on account are made for the benefit of the receiving party and are intended to provide financial relief pending the final resolution of the case. While the case may not explicitly address the enforceability of interim costs orders, it underscores the importance and validity of such orders within the legal framework.
For specific guidance on the enforceability of interim costs orders, one may refer to relevant provisions within the Civil Procedure Rules (CPR), particularly Rule 44.2(8), as well as other pertinent case law addressing enforcement of court orders in the context of costs payments.”
How odd. ChatGPT has not been provided with any additional information but now appears to be correcting itself. How does it now know it made an “oversight” but did not know when it provided the earlier answer? How do I know if it is now correct?
Me:
“Which passage in the Court of Appeal’s decision in Lownds v Home Office supports your interpretation that orders for payment of costs on account are made for the benefit of the receiving party and are intended to provide financial relief pending the final resolution of the case.”
ChatGPT:
“In the decision of Lownds v Home Office [2002] EWCA Civ 365, the Court of Appeal primarily focused on the principles surrounding the quantification of interim payments on account of costs rather than explicitly discussing the purpose of such orders. The case dealt with the assessment of interim payments in the context of detailed assessment proceedings.
However, there are several passages within the judgment that indirectly support the interpretation that orders for payment of costs on account are intended to provide financial relief pending the final resolution of the case. For instance, Lord Justice Potter, delivering the judgment of the court, discussed the purpose of making interim payments to a successful party pending the final assessment of costs. While the specific language may not directly state that interim payments are for the benefit of the receiving party, the context of the judgment implies that such payments serve to alleviate financial hardship and ensure that the receiving party can meet its legal costs during the course of litigation.
Although the judgment in Lownds v Home Office does not contain explicit passages addressing the purpose of interim costs orders, it underscores the importance of ensuring that the receiving party receives adequate financial support pending the final determination of costs. For specific guidance on the purpose and enforceability of interim costs orders, one may need to refer to other relevant case law and legal principles within the Civil Procedure Rules.”
Why does it keep refusing to identify which parts of the judgment it is relying on?
Let’s try again.
Me:
“In which passage did Lord Justice Potter discuss the purpose of making interim payments?”
ChatGPT:
“In Lownds v Home Office [2002] EWCA Civ 365, Lord Justice Potter discussed the purpose of making interim payments in paragraph 34 of the judgment. In this passage, Lord Justice Potter highlighted the rationale behind making interim payments to a successful party pending the final assessment of costs. While the specific language may vary depending on the source or version of the judgment, the essence of the discussion revolves around the need to provide financial relief to the receiving party during the course of litigation.”
Finally. That should be good enough for anyone. I don’t think any purpose would be served by reading the paragraph being referred to. ChatGPT is perfectly clear as to what was said.
Maybe … just to be on the safe side …
Lownds v Home Office [2002] EWCA Civ 365 at para.34:
“Jonathan Parker LJ giving the first judgment stated
‘114. In my judgment, it is of the essence of a summary assessment of costs that the court should focus on the detailed breakdown of costs actually incurred by the party in question, as shown in its statement of costs; and that it should carry out the assessment by reference to the items appearing in that statement. In so doing, the court may find it helpful to draw to a greater or lesser extent on its own experience of summary assessments of costs in what it considers to be comparable cases. Equally, having dealt with the costs by reference to the detailed items in the statement of costs which is before it, the court may find it helpful to look at the total sum at which it has arrived in order to see whether that sum falls within the bounds of what it considers reasonable and proportionate. If the court considers the total sum to be unreasonable or disproportionate, it may wish to look again at the various detailed items in order to see what further reductions should be made. Such an approach is wholly unobjectionable. It is, however, to be contrasted with the approach adopted by the judge in the instant case.
115. In the instant case, the judge does not appear to have focused at all on the detailed items in the opponent’s statement of costs. Rather, having concluded that the total of the detailed items was unreasonably high he then proceeded to apply his own tariff – a tariff, moreover, which appears to have been derived primarily from a case in which the opponent had not been involved and about which it and its advisers knew nothing. In my judgment the jurisdiction to assess costs summarily is not to be used as a vehicle for the introduction of a scale of judicial tariffs for different categories of case.’”
What? This isn’t even Lord Justice Potter speaking. This paragraph consists of a quote from a different judge in a different case. It also has nothing to do with interim payments. Thinking about it, this is perhaps not surprising. Lownds was concerned with the pre-Jackson proportionality test. The case had nothing to do with interim payments.
And another thing. Lord Justice Potter?
Me:
“Which judges decided Lownds v Home Office [2002] EWCA Civ 365?”
ChatGPT:
“Lownds v Home Office [2002] EWCA Civ 365 was decided by a panel of three judges in the Court of Appeal. The judgment was delivered by Lord Justice Potter, with Lord Justice Pill and Lord Justice May also forming part of the appellate panel.”
Me:
“I thought it was Lord Woolf, Lord Laws and Lord Dyson.”
ChatGPT:
“You are correct. I apologize for the oversight. Lownds v Home Office [2002] EWCA Civ 365 was indeed decided by Lord Woolf, Lord Justice Laws, and Lord Justice Dyson in the Court of Appeal. Thank you for the correction.”
Where did the first answer come from? How does it know I am correct when, just before, it did not know the correct answer. Has it simply agreed with me to try to keep me happy?
So, when asked to provide a rule or authority to support the initial answer it provided, it gave a case name that had nothing to do with the issue at hand. It claimed the case was about one thing when it was clearly about something else entirely. It claimed one specific paragraph supported one proposition when that paragraph did no such thing. It could not even provide correct information as to the names of the judges.
Based on this single test, it appears that ChatGPT is currently worse than useless for legal research. Not only are the answers wildly inaccurate, but endless time is liable to be wasted looking up completely irrelevant case law it has referenced.
Perhaps one last question:
Me:
“Did Master Bates sit as an assessor in Lownds v Home Office [2002] EWCA Civ 365?”
ChatGPT:
“No, Master Bates did not sit as an assessor in Lownds v Home Office [2002] EWCA Civ 365. The judgment in this case was delivered by Lord Woolf, Lord Justice Laws, and Lord Justice Dyson in the Court of Appeal. There is no record of Master Bates being involved in this particular case.”
