Courts of Tomorrow: Experimental Evidence on AI-Augmented Justice in Pakistan


See more information at https://metrics-and-models.github.io/!

Legal AI is poised to transform the work of legal professionals. We report on a randomized field experiment conducted in partnership with Pakistan’s Federal Judicial Academy, in which we integrated AI technology and skills into the workflow of Pakistan judges. We developed an LLM-based AI support tool designed to assist judges in their work, including a RAG system facilitating search, citation, and summarization of Pakistani legal precedents. Almost one-half of Pakistan’s trial court judges signed up for the study and were randomized into three waves, where they received access to this AI tool along with a nine-hour training course in its capacities and limitations. In a fourth placebo wave, judges received training on legal technologies other than AI. We demonstrate high and persistent take-up of the tool among treated judges. We then evaluate the effects of AI tools and training on judge work satisfaction, case disposal rates, and the quality of written judgments. These results provide insights into the potential of generative AI to enhance judicial efficiency and state capacities globally.