As artificial intelligence rapidly transforms various industries, the legal profession is no exception. However, a recent evaluation of Lexis+ AI™️, a new "generative AI-powered legal assistant" from LexisNexis, raises serious concerns about its reliability and effectiveness for legal research and drafting.

In a comprehensive review, University of British Columbia, Peter A. Allard School of Law law Professor Benjamin Perrin put Lexis+ AI™️ through its paces, testing its capabilities across multiple rounds. The results were disappointing, revealing significant limitations that should give legal professionals pause before incorporating this tool into their workflow.

Key issues identified include:

Citing non-existent legislation
Verbatim reproduction of case headnotes presented as "summaries"
Inaccurate responses to basic legal questions
Inconsistent performance and inability to complete requested tasks

Perhaps most concerning was the AI's tendency to confidently provide incorrect information, a phenomenon known as "hallucination" that poses serious risks in the legal context. For example, when asked to draft a motion, Lexis+ AI™️ referenced a non-existent section of Canadian legislation. In another instance, it confused criminal and tort law concepts when explaining causation.

These shortcomings highlight the critical need for human oversight and verification when using AI tools in legal practice. While AI promises increased efficiency, the potential for errors and misinformation underscores that these technologies are not yet ready to replace traditional legal research methods or professional judgment.

For lawyers considering integrating AI into their practice, several best practices emerge:

lawyers need to be weary when using generative ai! 😮

Understand the technology's limitations
Verify all AI-generated outputs against authoritative sources
Maintain client confidentiality by avoiding sharing sensitive information with AI tools
Stay informed about AI developments and ethical guidelines
Use AI as a supplement to, not a replacement for, human expertise

Just like in the United States, Canadian law societies and bar associations are beginning to address the ethical implications of AI use in legal practice. The Law Society of British Columbia has published guidelines emphasizing the importance of understanding AI technology, prioritizing confidentiality, and avoiding over-reliance on AI tools. Meanwhile, The Law Society of Ontario has set out its own set of similar guidelines. Canadian bar ethics codes may be structured somewhat differently than the ABA Model Rules of Ethics and some of the provisions may diverge from each other, the themes regarding the use of generative AI in the practice of law ring similar to each other.

Canadian law societies and bar associations, mirroring their U.S. counterparts, are actively addressing the ethical implications of AI in legal practice. The Law Society of British Columbia has issued comprehensive guidelines that underscore the critical importance of understanding AI technology, safeguarding client confidentiality, and cautioning against excessive reliance on AI tools. Similarly, the Law Society of Ontario has established its own set of guidelines, reflecting a growing consensus on the need for ethical AI use in the legal profession.

While the structure of Canadian bar ethics codes may differ from the ABA Model Rules of Ethics, and specific provisions may vary between jurisdictions, the overarching themes regarding the use of generative AI in legal practice are strikingly similar. These common principles include:

Maintaining competence in AI technologies
Ensuring client confidentiality when using AI tools
Exercising professional judgment and avoiding over-reliance on AI
Upholding the duty of supervision when delegating tasks to AI systems
Addressing potential biases in AI-generated content

Hallucinations can end a lawyers career!

This alignment in ethical considerations across North American jurisdictions underscores the universal challenges and responsibilities that AI integration poses for the legal profession. As AI continues to evolve, ongoing collaboration between Canadian and American legal bodies will likely play a crucial role in shaping coherent, cross-border approaches to AI ethics in law.

It is crucial for legal professionals to approach these tools with a critical eye. AI has the potential to streamline certain aspects of legal work. But Professor Perrin’s review of Lexis+ AI™️ serves as a stark reminder that the technology is not yet sophisticated enough to be trusted without significant human oversight.

Ultimately, the successful integration of AI in legal practice will require a delicate balance – leveraging the efficiency gains offered by technology while upholding the profession's core values of accuracy, ethics, and client service. As we navigate this new terrain, ongoing evaluation and open dialogue within the legal community will be essential to ensure AI enhances, rather than compromises, the quality of legal services.

MTC

The term "hallucination" refers to a phenomenon where an AI model generates or interprets information not grounded in its input data. Simply put, the AI is making stuff up. This can occur in various forms across different AI applications:

Remember just like you can’t complain to the judge when your clerk makes a factual or legal error in your brief, you can’t blame ai for its errors and hallucinations! 😮

Text Generation: In NLP, hallucination is often observed in language models like ChatGPT. Here, the model might generate coherent and fluent text, but this text is factually incorrect or unrelated to the input prompt. For instance, if asked about historical events, the model might 'hallucinate' plausible but untrue details. Another example is when attorneys rely on ChatGTP to draft pleadings only to learn the hard way that its cited cases do not exist. (Remember, always check your work!)

Image and Speech Recognition: In these areas, AI hallucination can occur when a model recognizes objects, shapes, or words in data where they do not actually exist. For example, an image recognition system might incorrectly identify an object in a blurry image, or a speech recognition system might transcribe words that were not actually spoken.

I’ll spare you a deep, complex discussion of the problems with AI in this context. But the three takeaways for attorneys are: 1. The programming for AI is not ready to write briefs for you without review, 2. Attorneys are not being replaced by attorneys (but attorneys who do not know how to use AI in their practice correctly will be replaced), and 3. Always check your work!

Happy Lawyering!