The 95% Problem: Why AI Land Data Still Needs a Landman’s Review

The biggest risk of AI in the land business isn’t that it gets things wrong.

It’s that it gets things almost right, and nobody experienced is checking.

Obviously wrong output is a manageable problem. Somebody notices, it gets fixed, life goes on. Almost-right output is different. It passes review. It goes in the file. It gets relied upon. And then eighteen months later somebody discovers the problem at the worst possible moment.

What “almost right” looks like in land management

A JOA summary that’s 95% accurate sounds great. Until the 5% it missed is the non-consent penalty clause, and you find out during a dispute what you actually agreed to.

A division order deck that’s 95% clean sounds efficient. Until the 5% becomes an interest dispute in year three, with accumulated underpayment and a very unhappy royalty owner who now has an attorney.

A title opinion where 95% of the chain checks out is not a title opinion. The whole value is in the exceptions.

Notice that in each case, the failure isn’t random. AI tends to miss the unusual: the non-standard clause, the odd conveyance, the exception to the pattern. Which is precisely where the money is.

This is not an anti-AI argument

I use AI every single day, across almost everything we do at Prosper Natural Resources. I’m a believer. The efficiency gains are real and I’m not going back.

But I believe in it the way a pilot believes in autopilot: it flies most of the route. You still want somebody in the seat who’s landed in bad weather.

Twenty-eight years in this business taught me where the bad weather is.

What review actually needs to look like

Effective review isn’t spot-checking a random 10% of AI output. It’s targeted at the places where “almost right” is expensive:

  • Non-standard provisions: anything an attorney clearly touched

  • Exceptions and carve-outs: where the document departs from the form

  • Anything with a penalty, forfeiture, or non-consent mechanism

  • Ownership decimals that will be relied on for payment

  • Curative items where the AI concluded something was resolved

That’s a Certified Professional Landman’s job. Not because the software is bad, but because knowing where to look is the expertise.

The question worth asking your vendor

If you’re evaluating an AI land data solution, ask who reviews the output and what their credentials are. If the answer is “the model is very accurate,” that’s not an answer.

Want CPL-level review on your land data? Get in touch.

Next
Next

From Courthouse Title Runs to AI: What 28 Years in Land Management Taught Me