ALT The image is a bar chart comparing the proportion of hallucinated and incomplete answers across different legal research tools. Hallucination: Lexis: ~0.2 Westlaw: ~0.3 Practical Law: ~0.3 GPT-4: ~0.4 Incomplete: Lexis: ~0.25 Westlaw: ~0.3 Practical Law: ~0.6 GPT-4: ~0.1 Caption: Comparison of hallucinated and incomplete answers across generative legal research tools. Hallucinated responses are those that include false statements or falsely assert a source supports a statement. Incomplete responses are those that fail to either address the user’s query or provide proper citations for factual claims.

ALT Promises and pitfalls of artificialintelligence for legal applicationsSayash Kapoor∗, Peter Henderson∗∗, Arvind Narayanan∗∗∗AbstractIs AI set to redefine the legal profession? We argue that this claim is not supported by the current evidence.We dive into AI’s increasingly prevalent roles in three types of legal tasks: information processing, tasksinvolving creativity, reasoning, or judgment, and predictions about the future. We find that the ease ofevaluating legal applications varies greatly across legal tasks based on the ease of identifying correctanswers and the observability of information relevant to the task at hand. Tasks that would lead to themost significant changes to the legal professional are not only harder to evaluate; they are also most proneto overoptimism about AI capabilities. We make recommendations for better evaluation and deploymentof AI in legal contexts.