Hey, fintech fam 💜

It’s almost wedding day (SO excited), so I asked my friend (and Academy of Fintech member!) Christina Rea-Baxter, if she would write today’s newsletter.

Christina’s continuing the conversation we’ve been having for months about AI in fintech, but she’s coming at it from a governance and compliance perspective.

I think you’ll really like today’s read!

Let's get into it. ✨

I've had dozens of conversations with compliance leaders, founders, and executive teams about AI over the past year.

Almost every one of them starts the same way: we talk about hallucinations, explainability, model validation, evolving (AI’s favorite word)  regulations, and whatever the latest headline happens to be.

I've noticed, however, that there's one question that rarely comes up:

How will AI change the way your employees make decisions?

Every time I ask this question, the conversation suddenly shifts. Because most organizations are still thinking about AI as a technology problem.

They're focused on whether the model works, whether it's accurate enough, or whether it can be trusted.

Very few people are thinking about what happens after people begin trusting it.

I don't think the biggest AI failures we're going to see over the next five years will happen because a model suddenly becomes wildly inaccurate.

The technology will continue improving. Governance frameworks will mature. Regulators will publish more guidance. Those are all positive developments.

What worries me more is something much quieter.

AI changes not only how work gets done, but how people think about work. It changes what people question, what they overlook, how confident they feel, and ultimately how they make decisions.

Those changes happen gradually, which is exactly why they're so easy to miss.

We've become very good at asking how we should govern AI. I'm not convinced we're spending enough time asking how AI changes the people we're asking to govern.

People are sometimes surprised when I say this, but I don't think compliance has ever really been about regulations.

Regulations establish the rules, but they don't make decisions. People do.

When I think back on enforcement actions, remediation projects, and some of the biggest compliance failures I've seen over the course of my career, I rarely remember them as stories about policies or procedures.

What I tend to remember are the stories about people.

Someone ignoring a red flag because they were under pressure to hit a deadline, someone convincing themselves an exception was low risk because nothing bad had happened before, and eventually, someone stopped asking questions.

That's why I've found the AI conversation so interesting.

We're investing enormous amounts of time figuring out how to govern the technology, but we're spending comparatively little time thinking about how the technology changes the people using it every day.

I've noticed this repeatedly through my work at RayCor Consulting, where my team and I spend a lot of time helping financial institutions think through AI governance with readiness assessments and evidence & defensibility implementation.

The conversations almost always begin with model risk, documentation, and regulatory expectations.

They almost always end with people.

How do we preserve independent judgment? How do we prevent overreliance? How do we make sure AI supports better decisions instead of quietly replacing them?

Spoiler alert: technology has always influenced human behavior.

GPS changed how many of us navigate without thinking.

Search engines changed what we bother remembering because we know we can look it up later.

Even spellcheck changed the way many of us write because we stopped catching mistakes ourselves.

AI is doing the same thing, but the consequences are much bigger because we're using it to support decisions that can affect customers, financial crime investigations, fraud reviews, and regulatory compliance.

The more often AI produces a reasonable answer, the more natural it becomes to trust it.

We all look for ways to reduce cognitive effort, especially when we're busy, under pressure, or trying to work through a queue of repetitive tasks.

One of the reasons I keep coming back to this question is that I've caught myself doing it, too.

There have been moments where an AI response sounded so polished and so reasonable that my instinct was to move on without questioning it.

I had to consciously stop and ask whether I agreed with the recommendation or whether I was simply reacting to how confident it sounded.

That experience made me realize this isn't just a governance challenge for institutions. It’s much bigger than that because it's a very human challenge.

Imagine you're an analyst reviewing alerts.

The AI recommends closing one, and the explanation seems reasonable.

You've seen the model make good recommendations hundreds of times before, and you still have dozens of cases waiting for you.

At that point, the real question is whether you still feel responsible for reaching your own conclusion.

The first major AI governance failures may not come from a model producing a terrible answer.

They may come from thousands of perfectly capable people gradually becoming less curious, less skeptical, and less willing to challenge a recommendation because the machine has been right often enough that questioning it no longer feels necessary.

The more I think about this, the more I believe AI governance has a measurement problem.

Right now, organizations are investing enormous amounts of time evaluating the technology itself.

They're validating models, documenting assumptions, testing for bias, reviewing outputs, and building governance committees to oversee implementation.

Those are all worthwhile exercises; I just don't think they're enough. I think we need to broaden the conversation around AI governance.

We shouldn't only be asking whether the model is reliable, but also whether our people are becoming more thoughtful decision-makers or more passive reviewers.

I'm much more interested in questions like these:

Those aren't traditional AI governance metrics, but I wouldn't be surprised if they become some of the most important ones over the next several years.

In fact, those questions became one of the driving forces behind AIRE (AI Risk & Explainability), the AI governance framework we've developed at RayCor.

We care about model inventories, validation, documentation, and governance structures because those are all essential.

But we also believe organizations need practical ways to preserve human judgment as AI becomes part of everyday decision-making.

That's a conversation I think the industry is only beginning to have.

One thing I hope we don't lose in all of this excitement is the reason most of us got into compliance in the first place.

Our job has never been to create perfect processes (if you’re a regulator, pretend I didn’t say that).

I believe we’re mainly here to help organizations make better decisions.

AI has the potential to make that job dramatically easier.

It can eliminate repetitive work, identify patterns humans would almost certainly miss, and give experienced professionals more time to focus on the complicated decisions that actually require judgment.

But none of those benefits happen automatically. Organizations have to build cultures where employees understand that AI is there to support their thinking, not replace it.

That means encouraging people to challenge recommendations, making it acceptable to override the system when something doesn't feel right, and rewarding thoughtful decision-making instead of simply rewarding speed.

One of the questions I think leaders should start asking is surprisingly simple: If I took the AI away tomorrow, would my team still know how to make this decision?

If the answer is no, you probably don't have an AI governance problem: you have a workforce development problem.

The organizations that succeed will be the ones that continue investing in curiosity, skepticism, accountability, and independent judgment.

Even as AI becomes embedded in everyday work. Because when the next major compliance failure makes headlines, I doubt the story will simply be that an AI model got something wrong.

My guess is it will be that a room full of smart, well-intentioned people slowly stopped believing it was their job to ask whether the AI was right.

That's a much harder problem to solve than model validation. It's also, in my opinion, where the future of AI governance really begins.

Christina Rea is the Founder & CEO of RayCor Consulting, where she advises banks, fintechs, payment companies, and digital asset firms on financial crimes compliance, governance, regulatory strategy, and AI governance.

She is the creator of AIRE, RayCor's AI governance framework for regulated financial institutions, a former Chief Compliance Officer of Binance.US, a New York attorney, and a frequent expert witness in fintech and crypto-related litigation.

This has been one of my favorite songs for years.

It's one of those songs that instantly slows my brain down, reminds me it's actually summer, and makes me want to roll the windows down and take the long way home.

📰 Share this newsletter with a friend and start growing your network.

🔗 Connect with me on LinkedIn for daily insights on leadership.

🤝 Grow your business through content & community by partnering with me.

📣 Promote yourself to 50,000 subscribers by sponsoring this newsletter or the Humans of Fintech podcast.

🎤 Host an epic event by booking me as a speaker, moderator, or emcee.

📚 Increase your expertise by ordering your copy of my book, Fintech Feminists: Increasing Inclusion, Redefining Innovation, and Changing the Future for Women Around the World.

Thanks for spending time with Christina and me today!

Let me know how you enjoyed the first guest newsletter.

Love,

Nicole 💜