The Straw Man
So there I was...
Staring at a solid month of writing in my spare time at work and every evening. It was a white paper for a global SaaS powerhouse. The feedback had come back: you need to drop the entire thesis about the future of unstructured data, we only deal in structured data. Rewrite it.
The thing is, we were both right, but they were paying the bill. The thesis addressed future data structures and analytic opportunities, and that is where I thought we were going. It wasn't. So I had to pivot to considerations of data maturity. Weeks of additional wiring, but the client was happy and I was able to make a strong, albeit slightly different point.
I think back on that time and how today the problem could have been easily solved with a Claude prompt. But the work would not have been mine. The research and insights would not have been mine, nor the conclusions. What is the point of hiring someone for their insights only to have them outsource it? There is a genuine issue of authorship in the age of AI, and the judgment that comes with it.
In a Houston courtroom a couple hours south of me, a man learned what it costs to answer the authorship questions the wrong way. The case revolved around the January 2020 explosion at Watson Grinding, a blast that killed three people and leveled around two hundred homes. 3M was a defendant, so they hired an expert witness, a gas-detection engineer with more than twenty years in the field. He was paid to put his own independent judgment from years of experience, under oath, into the witness chair to help the jury and the judge understand the science. That was his sole purpose in filling the empty chair.
But he handed it to ChatGPT.
Not to help him think, but to do his thinking for him, and not to arrive at an evidence-based conclusion. He did not ask AI what caused the explosion or where the fault lay. He told it the answer he was being paid to reach and asked it to figure out the argument: "show how 3M is 0% at fault for the explosion at Watson Grinding." It did as it was told and produced the line "3M is 0% responsible." If you've ever been in litigation, you are probably familiar with a legal tool called discovery, and it became his undoing. He ended up acknowledging at trial that roughly 85% to 90% of his final report was all ChatGPT, and like a good procrastinator a good deal of it generated the night before his deposition. His rate was $475 an hour.
This is not a bad tool story but rather a bad character story. ChatGPT flagged "0% responsible" as being too much of a stretch, and that it would invite exactly the criticism he later received. The very thing he was using as a shortcut disagreed with the conclusion he was paid to deliver, so he overrode it and submitted the report anyway. AI was not the failure of judgment in that room, he was. When it turns out the only thing in the building with a shred of caution is the LLM, the chair that matters is already painfully empty.
For all you sleuths out there, opposing counsel found a stray five-page "Citation Overlay" that smelled like AI, so they asked for the ChatGPT logs and 3M's own lawyers were forced to hand over all three hundred and fifty pages of them. Then, in a truly ballsy move, opposing counsel called the defense's expert as his own witness, just to walk the jury through the logs. His summary of the whole fiasco was an epic courtroom moment: "ChatGPT was the expert in the case. There's just no question about that." The jury put 30% of the liability on 3M, costing them more than $61 million in damages.
Now I think back to my white paper. I discovered I was being paid to reach a conclusion the client wanted, so I did the weeks of additional writing. But the argument was mine, and my name on it meant something because it was actually me behind it and all of it was true and defensible. Our expert took a different route. The court priced the difference in our approaches at $61M, because the moment a human vacates the chair of judgment, everyone downstream can see it, and the credibility you were hired to provide evaporates.
AI can certainly do the heavy lifting. It can research, draft, summarize, and even argue. I built a whole council of ethicists that does that to keep me on my toes. What it cannot do is be accountable for the answer, because accountability only affects something that can be harmed by being wrong. A model cannot be sworn to tell the truth, the whole truth, and nothing but the truth so help it God. It cannot be cross-examined, cannot lose its professional license, or nights of sleep. The chair labeled judgment is not about who does the typing. It is about who owns the conclusion, right or wrong. Our expert wanted the fee for sitting in that chair, and then tried to blame the technology when it all went south.
So here is my question this morning, from a guy who once spent a month rewriting something he really did not want to.
You can use AI to sharpen every insight you have. I do, every day. It is a part of how I find many of these stories. When someone hires you, they are not just buying a bunch of words, they are buying your experience and judgement. So before you send the next thing out with your name on it, ask the question that would matter in a courtroom. If a lawyer put your chat logs on a screen in front of a jury, would they see you using a tool, or would they see the outsourcing of your judgment?
#AI #Ethics #Accountability #Judgment #Integrity #TheEmptyChair #STIW