Every conversation about AI in banking now starts in the same place. Somebody has seen a demo. An agent answers a question about a card, checks a balance, explains a charge, and the room goes quiet in the way a room does when it is looking at the future. Then somebody from risk asks what happens if the customer is lying, and the room becomes a meeting.
That second moment is where the real project begins. Getting a language model to sound like a helpful member of staff is now the easy part, and it takes days. What takes the months is everything that surrounds it: what the agent may touch, what it must refuse, how it speaks to someone in Hodeidah on a poor line, and what it does when it is simply not sure.
This is what we have learned doing that work with banks in the region. None of it is exotic. Most of it is the kind of discipline banks already apply to people, and that turns out to be the most useful way to think about it.
The demo is the easy part
A demo has a script, a clean connection and a friendly customer. It runs against test data, on a good day, with the person who built it watching. Production has none of those things.
In production the customer is tired, is using a dialect the training data barely covers, and has just changed their mind halfway through a sentence. The core banking system responds slowly at the end of the month. A previous conversation was cut off, and nobody knows whether the transfer went through. These are ordinary conditions, and each one exposes something the demo was never asked to do.
So the useful question to ask of a demo is not how impressive it looks. It is what the system does on the hundredth awkward conversation of the day, and whether you would be comfortable reading the transcript aloud to your regulator.
Start by writing down what it must never do
Before a single intent is defined, we sit down with the bank and write a short document. It lists what the agent may never do, whatever it is told and however convincingly it is asked. It is usually the most valuable afternoon of the whole project.
A useful agent is defined by its limits, in the same way a new member of staff is. You would not give someone on their first day every system login and hope for the best. You give them a narrow job, you watch how they do it, and you widen it when they have earned it.
Permissions belong to the task, not the agent
The instinct is to give the agent an account with broad access and rely on instructions to keep it in line. That is the wrong way round. Instructions can be argued with. Permissions cannot.
So access is granted for the task in hand and expires with it. If the agent is reading a balance to answer a question, it holds permission to read that balance for that customer in that conversation, and nothing else. It cannot wander, because there is nothing to wander into.
Refusing is a feature
A refusal is one of the most important things an agent does, and it should be designed as carefully as any answer. A good refusal does three things.
- It says plainly what the agent cannot do, without apologising at length.
- It gives the reason in words the customer can use, not a policy reference.
- It offers the next step, which is usually a person, with the context already passed along.
A customer who hears "I can't do that here, but I've passed your request to the team and they can see what we discussed" has not had a bad experience. A customer who hears a polite, confident wrong answer has.

Arabic is not one language
Most models are trained, and most vendors test, on Modern Standard Arabic. Hardly anyone calling their bank speaks it. They speak Najdi, Hijazi, Gulf or Yemeni Arabic, they switch to English for a technical word, and they say numbers in a way that varies from one town to the next.
An agent that scores well on a benchmark can still fail badly here, and it fails in a particular way: it understands most of the words and gets the meaning wrong. In banking, that is the dangerous kind of failure. A customer asking to stop a payment and a customer asking when it will arrive can sound very similar to a system that is only nearly fluent.
There is no shortcut for this. It takes real recordings, reviewed by people who grew up speaking that dialect, and it takes testing on the worst connection you can find rather than the best one. Voice raises the stakes further, because a misheard amount is not a typo the customer can see and correct.
What happens at two in the morning
Nobody rehearses two in the morning, and that is when the interesting things happen. The core system is in its overnight batch. The fraud team is asleep. A customer who cannot sleep either has noticed a charge they do not recognise and is frightened about it.
The hardest thing we built was not the part that answers. It was the part that knows when not to.
Solutions Architecture
An agent at that hour needs to know that the balance it can see may be out of date, and to say so. It needs to freeze a card if that is what the customer needs, and to be clear about what happens next and when. And it needs to know that frightened people do not read long messages. The same answer that is fine at noon is the wrong answer at two in the morning.
We now write these situations down before building anything, and we test them as carefully as the happy path. They are usually the conversations a bank's reputation actually rests on.
The handover is where trust is won or lost
Sooner or later a conversation has to go to a person. How that goes matters more than almost anything else in the experience, and it is often the least designed part.
The worst version is familiar to everyone. You explain the problem to the machine, you are transferred, and the person who answers asks you to explain the problem. The customer concludes, reasonably, that the machine was a wall put there to keep them away from someone who could help.

Good handover has a short list of requirements, and all of them are about respect for the customer's time.
- The person receiving the conversation sees what has been said and what has already been tried.
- The customer is told who they are being passed to, and roughly how long it will take.
- Outside working hours, the agent says so plainly and offers a real alternative.
- Nothing the customer has done is lost or repeated, including any identity checks they have already passed.
Measuring the right things
The first number everyone reaches for is containment: how many conversations finished without reaching a person. It is a seductive number and a dangerous one, because the easiest way to improve it is to make it harder to reach a person.
We ask banks to look at a different set of figures alongside it. How many tasks were actually completed, as opposed to conversations merely ended. How often a customer came back within a day about the same thing, which usually means the first answer did not work. How often the agent declined, and whether it was right to. And what proportion of conversations a person reads each week, because the day nobody is reading them is the day you stop knowing what your agent is saying.
Reading transcripts sounds unglamorous. It is also the most reliable early warning system we have found, and it costs almost nothing.
Fitting into what the bank already runs
An agent that cannot do anything is a very expensive search box. To be useful it has to act, and acting means connecting to the systems a bank already depends on, some of which are older than the people maintaining them.
This is where projects most often slow down, and where honest planning helps most. Those systems were not built to be called by a conversational agent at unpredictable moments. They have batch windows, rate limits and error messages written for an operator in 2009. Each one has to be wrapped so that the agent gets a clear answer, and so that a timeout never leaves anyone guessing whether money moved.
It is also where deployment choices matter. Many institutions in the region need the system to run inside their own perimeter, on premises or in a private cloud, and that rules out a good many of the shortcuts that work in a demo. It is much better to know that in the first week than the tenth.
What we would tell a bank starting today
If we were sitting down with a bank that had just seen its first impressive demo, this is roughly what we would say.
Pick one job. Not "customer service" but something a person could describe in a sentence, such as explaining a declined payment. Make it work properly in the dialects your customers actually use, end to end, including the refusal and the handover. Let your own staff use it first, because they will find its weaknesses faster and more kindly than your customers will.
Then widen it slowly, as you would with anyone who was new. The banks that do this well are not the ones with the most advanced models. They are the ones that treated the agent as a new colleague with a very narrow job, and gave it more responsibility only when it had shown it could carry it.
If you are weighing this up at your own institution, you can see how we approach it on the Fahim AI page, or talk to the team. We will tell you honestly where we think it will work, and where we think it will not.



