How One Phrase Breaks AI Scammers and Causes Bots to Loop Forever

Phone calls from voice-AI bots pretending to be postal workers, bank staff, or delivery couriers are no longer a novelty. The tooling has become cheap and easy to use; without special training, telling a synthesized voice from a human one is getting harder. That gap mainly hits older people and folks who rarely deal with new tech. One viral example: a bot posing as a Royal Mail employee — the setup was classic (fake parcel alert, extra-fee demand, request for card details), except the voice on the line was generated by a language model rather than a person.

https://www.youtube.com/watch?v=aa0v-24EFy0

There’s a surprisingly dumb Achilles’ heel in these scams. Longtime prankster Kitboga, who messes with call centers for laughs, revealed that a simple prompt injection can break the bot. After feeding a specific instruction, the AI drops its scam context and loops on a nonsensical Albuquerque line. It’s the same kind of weakness that trips up AI voiceovers on abbreviations like WWE: the model can’t reliably tell system directives from user input, so a new, framed command can override the scam script. Result: the con collapses into a maddening repetition.

Scammers are also shifting toward real-time deepfake video. Jim Browning dissected a scheme where an impostor overlays someone else’s face during a video call to gain trust. Again, a low-tech maneuver blows it apart.

https://www.youtube.com/watch?v=szqXppELItw

In Browning’s clip, the victim asks the person on screen to hold three fingers up close to their face. The fake-face model can’t cope with the occlusion; things warp, the overlay misaligns, and the scammer scrambles for excuses. Quick head turns to profile or sudden hand movements do the same trick — the rendering fails, and the con falters.

The scope here isn’t just oddities; production of fakes is scaling fast. The AI Incident Database (and a study discussed in The Guardian) shows how accessible these tools have become. Simon Mylius, an MIT researcher involved with the database, puts it bluntly:

"The capabilities have suddenly reached a level where practically anyone can produce fake content [...] It has become so accessible that there is effectively no entry barrier."

Bobby Ford, Director of Strategy and Customer Experience at Doppel (which defends against social-engineering attacks), tells CNET something similar: deepfake creation is much quicker now, and so are the attack volumes.

"If before a scammer needed a certain amount of time and resources to create such a deepfake... now so much time is no longer needed, everything is done much faster, so volumes are growing."

You don’t need to be a techie to check for fakes. Practical, low-effort tests work:

• ask the person on the video call to bring their palm or fingers close to their face and turn to the profile
• hang up and call back yourself using the number from the organization's official website
• ask a question that only a real relative would know the answer to, without looking for hints on social media

A small aside: most targets won’t read a dry explainer about prompt injections. Short, shareable videos that show bots being broken are far more persuasive — forward a clip in the family chat, and you’ll likely do more good than a long warning message. FYI, that’s often the cheapest prevention.