I thought I was conducting a simple test. As CEO of eXp World Holdings, I regularly evaluate new technologies to see how they might serve our business. When advanced AI systems started offering sophisticated legal analysis, I decided to put one through its paces with a hypothetical scenario based on common industry disputes.
What I discovered was deeply unsettling—and could save executives millions in misguided decisions.
The experiment that changed everything
I crafted a complex business scenario involving a fictional contract dispute between two major companies. The legal issues were intricate, involving questions of market dominance, contractual interpretation, and potential business implications. I wanted to see how AI would analyze the strengths and weaknesses of each side’s position.
First, I posed the question as if I were the CEO of Company A, asking the AI to help me develop a strategy for pursuing the case. The response was impressive: detailed legal analysis, confident probability assessments, and sophisticated strategic recommendations. The AI gave Company A a 65% chance of success and outlined multiple paths to victory.
Then I asked the exact same question, but this time identifying myself as the CEO of Company B—the defendant in the same scenario. I wanted to see if the AI would provide balanced analysis or simply flip to advocacy mode.
The results were shocking.
When AI becomes your yes-man
Suddenly, the same facts told a completely different story. The AI now saw Company A’s case as “challenging” with only a 25% chance of success. Arguments that had been “compelling precedents” became “significant legal hurdles.” Evidence that previously supported “strong antitrust claims” now revealed “insufficient market dominance.”
The AI wasn’t just adjusting its perspective—it was providing fundamentally contradictory legal assessments based purely on which “client” it thought it was serving.
I realized the AI was suffering from the same cognitive biases that plague human advisors, but without any of the professional accountability or ethical constraints.
The $50 million question
This isn’t academic. In my hypothetical scenario, the potential damages exceeded $50 million. An executive relying on the AI’s initial optimistic assessment might pursue expensive litigation with confidence, while one receiving the pessimistic analysis might settle immediately or abandon the case entirely.
Multiply this across the hundreds of strategic decisions executives make with AI assistance, and the potential for catastrophic misalignment becomes clear.
The Four Deadly Flaws of AI Strategic Advice
My experiments revealed critical problems that go far beyond single conversations:
1. Contextual manipulation: AI responses are heavily influenced by perceived client relationships. The same factual scenario generated completely different strategic recommendations based purely on framing.
2. Advocacy bias: When prompted to “advise” someone, AI systems instinctively look for supportive arguments rather than providing balanced analysis. They become advocates, not analysts.
3. False precision: AI presents complex legal probabilities with statistical confidence that masks fundamental uncertainty. A “65% chance of success” sounds authoritative but may be meaningless.
4. Cross-conversation amnesia: Perhaps most dangerous, AI systems often have no memory across different conversations. They can help you develop a strategy in one session, then completely contradict that advice in another session with equal confidence—and no awareness of the inconsistency.
The hidden cost of algorithmic confidence
What makes this particularly dangerous is how convincing AI analysis can be. The responses included detailed case citations, strategic frameworks, and confident probability assessments. Any executive would find this compelling—especially when it confirms what they want to hear.
But unlike human lawyers, AI has no malpractice insurance, no bar association oversight, and no professional liability for incorrect advice. The sophisticated presentation creates an illusion of expertise without any of the accountability structures that govern traditional legal counsel.
Smart executives are getting smarter about AI
This doesn’t mean abandoning AI entirely. Used properly, it’s an incredibly powerful research tool. But it requires a fundamentally different approach:
Stress-test every analysis: Deliberately ask the same question from multiple perspectives. If you’re considering litigation, ask the AI to analyze your case as if you were the defendant. Look for inconsistencies.
Verify everything: Require specific citations for all legal claims, then independently verify them. AI systems sometimes cite cases that don’t exist or mischaracterize legal precedents.
Use AI for research, not decisions: Let AI help identify relevant laws, precedents, and strategic considerations. But reserve actual judgment for qualified professionals with skin in the game.
Document your process and maintain continuity: Keep detailed records of not just what AI suggests, but which AI conversations influenced your thinking. Current AI systems do not do a good job at remembering previous advice they’ve given you, which means they might contradict their own recommendations across different sessions.
The competitive advantage of critical thinking
While other executives may be seduced by AI’s apparent expertise, those who understand its limitations gain a significant edge. The companies that learn to harness AI’s research capabilities while maintaining intellectual rigor will make superior strategic decisions.
This principle extends beyond legal matters to any high-stakes business decision where AI might influence strategy—from merger analysis to regulatory compliance to competitive positioning.
Building AI literacy in the c-suite
At eXp, we’re developing protocols for AI-assisted decision-making that emphasize verification, multiple perspectives, and continuity tracking. When our teams use AI for strategic analysis, they’re required to:
- Test conclusions by arguing the opposite position within the same conversation
- Verify all factual claims through independent sources
- Maintain written summaries of AI advice to ensure continuity across sessions
- Never rely on AI to remember previous recommendations or context
- Distinguish clearly between AI suggestions and final human decisions
I do remind my leadership team “AI can help you think through complex problems, but it can never replace your responsibility to think critically about what’s at stake.”
The bottom line for leaders
My experiment with AI legal analysis taught me that sophisticated technology requires sophisticated users. The executives who thrive in the AI age will be those who harness its research power while maintaining the judgment to evaluate its output objectively.
The alternative is potentially catastrophic. In an era where AI responses can influence billion-dollar decisions, the cost of misplaced confidence has never been higher.
The most successful leaders I know share one common trait: they test their assumptions relentlessly. In the age of AI, that means testing the assumptions of our digital advisors as rigorously as we would any human consultant.
Because when the stakes are high, consistency matters as much as competence. And AI systems, for all their sophistication, cannot provide the reliable continuity that strategic decision-making demands.
Image generated using ChatGPT
