Solving hallucinations

Solving hallucinations is crucial because legal documents cannot have mistakes. We have tried to mitigate this in a few different ways. We have provided access to 2-3 top legal databases, and all legal research and planning are solely based on research conducted using these legal databases.

Mira V1 also runs the same research and planning tasks in parallel using multiple agents. This approach has helped us uncover missing information, discover new information, and identify incorrect information using cited sources and more.

We identified that increased collaboration at certain levels helps us understand what's missing and what issues are present. We then repeat this process until no issues remain. Overall, this approach has increased the research quality while significantly reducing or eliminating hallucinations.

Safety and accuracy

One of the important features (still in early development) of Mira V1 is its ability to understand the complexity of tasks and their potential impact on the business.

If Mira V1 recognizes a major issue or a critical decision-making point, it will pause the task and reach out to the relevant person. If it cannot provide a proper solution, it will stop the task and clearly communicate that it cannot proceed due to the issues present.

We expect and encourage small teams without legal expertise to use Mira V1 for documents and contracts. We recommend checking the output (the documents in this case) with a licensed lawyer for an extra layer of security if needed. This approach can still save 70% to 80% of your legal costs.

Either way, this is a challenging and industry-related problem to solve, and it has become more important than ever for legal tasks. We are still in the early stages and are working on research and custom jurisdiction-specific models that can help us resolve this issue.

Comparison

We tested state-of-the-art models alongside Mira V1 and PRO. We provided all the models with equal information to draft a complex legal document.

Legal documents and contracts drafted using Mira V1 were highly comprehensive, personalized, accounted for the latest laws and regulations, and, most importantly, had little to no hallucinations or incorrect information present.

Note that the number of pages in a legal document may not always be the best way to predict its accuracy and legal reasoning, yet it shows how detailed the agreement is based on legal research and parallel validation that were conducted to create the document. We hope to publish more legal specific benchmarks, such as LegalBench, LawBench, BLT and LEXTREME, against Mira V1.

We also believe that standard Legal LLM benchmarks might not be the best way to test the legal intelligence of Mira V1. We think the best way to test Mira V1 would be to focus on: communication skills and collaboration, accuracy of legal research, situational awareness, and legal documents' accuracy and enforceability.

With our current limited resources, we couldn't create custom tests and benchmarks for Mira V1. We hope you will create standard benchmarks for Agentic Legal LLMs in the coming months.

Limitations

Misinterpret when to stop/continue.

Even though Mira V1 is built to stop/pause when used for really complex or decision-making tasks with limited information, the ability to assess whether it should stop or continue can sometimes be inaccurate. For example, Mira V1 might stop/pause for a normal task and might even continue working on tasks and make complex decisions on its own for complex tasks.

We plan to improve this by developing custom critic models that can assess whether Mira should pause or continue.

Long context issues:

More often, legal research involves reading multiple documents and finding relations for the current situation of the user. Because the goal of Mira V1 is to provide accurate results, it may repeat the process of legal research and keep researching and might not know when to stop. This might lead to confusion as Mira V1 has a load of information to process and might confuse certain things.

We have currently addressed this issue by implementing improved step-by-step reflection with a maximum limit on the number of research iterations. This approach helps the models focus on specific tasks and complete them by understanding their previous actions and the remaining quota of moves (in this case, research steps) left.

We certainly understand that we can never put a number on the maximum times a research could happen, nor can reflecting on the past actions taken always lead to solid research. This is a challenging two-way problem to solve; we need to solve both 'context' and improve the understanding/ability of the models on legal research. We intend to improve them in the upcoming versions.

Data privacy and security

While developing Mira V1, we have put a tremendous amount of focus on data privacy and security. We understand that your business information and your contracts are highly confidential information. All your business information, emails, and documents are highly encrypted - we use AES 256 (bank-grade security).

You also have the option to delete critical business information from the dashboard itself, but note that if you do so, Mira V1 will forget about this information.

You also have the option to instruct Mira V1 to send your final agreements using DocuSign or Dropbox for security, meaning the final agreements will never be saved in our database. Finally, we do not use any of your personal information, business information, documents, contracts, or emails to train our models.

Get access to V1

We are providing Mira V1 for limited businesses and teams. Mira V1 will have 100% availability and access by early September, so if you are a startup, business, or legal team interested in knowing more, please send an email to team@useairstrip.com