Law professors prefer AI over peer answers - Marginal REVOLUTION

Thank-you! You've been successfully added to the Marginal Revolution email subscription list.

Large language models (LLMs) are increasingly promoted as educational tutors, yet most evaluations focus on domains with a single ground truth. Many disciplines, however, hinge on judgment: reasoning, weighing ambiguity, and reaching defensible conclusions. Law provides a sharp test. We conducted a blinded evaluation of short-answer tutoring in contracts courses with sixteen U.S. law professors. Participants created 40 representative questions, wrote answers, and judged 2,918 anonymized comparisons between human and LLM responses. Professors rated LLMs far higher than their peers (average win rate = 75.33%), with models performing similarly to the best instructor. LLM responses were also rarely flagged as harmful (3.53%, vs 12.06% for professors). Preferences for LLM answers were consistent across evaluators and reflected shared professional standards. Our evaluation can be reliably extended to additional models by employing a separate LLM as a judge, rendering expert agreements an effective, scalable method to evaluate AI tutors in judgment-rich domains.

"far".  That is from a new paper by Alejandro Salinas, et.al.  Via Andrew Curran.  And via John Chamberlain:

Artificial intelligence (AI) and large language models (LLMs) tools are capable of mass-producing academic finance papers that are nearly indistinguishable from human-authored research, according to a new study published in the Journal of Economic Literature.

C’mon people, get ready.  I know it is difficult to admit when your human capital has been devalued, but that time is upon us.  In particular, being prolific is no longer such a comparative advantage in academia.  You might run to the "but I know what questions to ask" cope, but I implore you to solve for the equilibrium.  What is the equilibrium wage for merely asking questions?

Of course academic life and projects will continue, but the real rewards will go to people doing new, innovative, and hitherto impossible projects with AI.

Marginal Revolution University

See Courses

Learn more about Mercatus Center Fellowships

Learn More

Subscribe via Email

Enter your email address to subscribe to updates.

Email Address

Subscribe

RSS Feed

Contact Us

Alex Tabarrok

Email Alex

Follow @atabarrok

Tyler Cowen

Email Tyler

Follow @tylercowen

Webmaster

Report an issue

Blogs We Like

Interesting People & Sites

Our Web Pages

Alex Tabarrok's Home Page

Alex's TED talk, how ideas trump crises

Conversations with Tyler

FDAReview.org

Tyler Cowen's Personal Web Page

Tyler's ethnic dining guide

Apply to Emergent Ventures

Books

Modern Principles of Economics

Tyler Cowen & Alexander Tabarrok