The Hidden Costs of 'Free' AI on Security

Back in January 2024, an employee fell for a scam and handed over twenty-five million dollars during a video call. The person saw what looked like their boss and coworkers on screen, but it turned out they were all AI-generated images. It used to be easy to verify someone's identity. You’d just see their face and hear their voice, and you’d know who they were without a doubt. Now, that same verification is pretty much useless. To actually confirm someone's identity these days, you gotta go through multiple steps with code phrases and callback methods, which all take time and money. Trust just got way more expensive for everyone. The financial sector is starting to factor this into their costs. Companies lacking solid security measures are seeing their insurance rates rise. The typical hit from these sorts of scams is around six hundred grand, not even counting the massive losses like that twenty-five million incident. But for everyday folks? We’re dealing with this cost in different ways: the time spent being suspicious of everything, the money we lose to scams we can’t fight, and the chances we miss out on because we just can’t trust what we’re seeing. I’ve started to doubt every financial request unless I’ve got strong proof otherwise. This is a drain on my time and mental energy that didn't even exist a couple of years ago. And I’m realizing this is just the tip of the iceberg. As these technologies become cheaper and more accessible, the burden of keeping our financial security in check will just keep climbing for all of us. So who’s footing the bill for this? Right now, it’s mostly individuals and small businesses that can’t shell out for top-notch security systems.

This is a serious issue but in a formal setting, there is a mix of careless in this story because twenty-five million is not a small amount to give out without enough supporting authorization and verification. Authorization and verification via email and phone call is necessary to get anything of this magnitude moved, at least I'm in the system so I know how some of these things work. Maybe the employee have some questions to answer regarding the whole thing. AI is definitely changing a lot of things and even increasing the doubt of people regarding what is real and what is not. It brought a lot of advantages but those did not come at no cost. Eventually we will adapt to this new way of life where everything in the digital world have to be questioned and verified.

Transferred 25 million dollars over the video call? For that big amount there must be a legal process involved with all the paper and documentation so it is not that easy unless someone is stupid enough and given that much money to manage. But for the smaller amounts, it is possible, I heard a few stories that someone over the phonecall sounded exactly like a father or husband asked for an emergency need and asked to send money to an bank account but only later the person realized that they have been scammed. SO keep the numbers saved in your contact and don't asnwer unless they call from that number.

When it comes to financial or asset security, I always believe that thieves are always one step ahead of the legal institutions that regulate them. It can be concluded that criminal innovations always outpace innovations in protection or security. This is an undeniable phenomenon, a manifestation of our world's daily advancements. As users and innovators, we cannot simply focus on the positive aspects (efficiency, convenience, and the "wow" effect of technology) without experiencing the real impact of our adoption of technology in our lives. When security becomes a top priority, and thieves constantly find new methods, the only thing left is for system owners to react by adding layers of security to protect users. Users are left in the back seat, having to comply with the system by adapting to new rules with all their complexities. Money, effort, and time are what we pay, but value and security are what we get. Problems arise for small and medium-sized businesses when their capital is limited to invest in additional layers amidst the demand for zero-fault security. Where experiments can't be conducted or capital is exhausted to keep up with the thieves, those who are unwilling and unable to adapt are the ones left behind. So, if asked who bears the greatest cost, it's users who refuse to adapt to multilayer security standards. In an era where proving honesty is more important than honesty itself, we (users, law enforcement, institutions, and business entities) must start thinking about how to not just react but take the initiative to counter thieves' moves.

This is indeed becoming very serious. With artificial intelligence becoming ubiquitous and potentially misused, there will be more problems like this in the future. I remember reading a news article about two years ago about Turkish security forces arresting someone who used  AI to impersonate President Recep Tayyip Erdoğan's voice, attempting to deceive select companies and high-ranking government officials. https://www.trtworld.com/article/14515799 That was two years ago. Now, the situation is much more complex and dangerous, as it's now possible to create realistic videos of people who look real, not just voices.

Fast forward to today, we are now more aware of AI and the signs accompanied.  that something might be AI-generated. AI might become more sophisticated in the future so we really have to be always suspicious of everything. Better to look like you are overreacting than regret being too trusting.

I think they take advantage of adults who are in a hurry to solve problems right away. Nowadays, you have to be careful with phone numbers, which is why I don't trust anyone until I see them in person and they give me their phone number. For online transactions, I understand that there are secure ways outside of social media to make it safer. However, before making any transaction, it is always better to use p2p on secure platforms that hold the money until the corresponding service or person verifies their identity. Cheer up, friend, the world is for those who prepare themselves

Criminals are not always ahead of some key organisations like Central Banks and other top financial institutions. These institutions have some of the best security systems and have a large budget for security. Sometimes these security breaches are enabled by insiders who are driven by financial and other gains. There will never be a perfect security system. Every institution would have to use available funds to invest in research and development of improved security systems. I still believe that there are affordable but still sound security systems that small and medium-scale businesses could adopt. Every institution, including the government, transfers costs to individuals, so it's not strange. Organisations that cannot afford the cost will not kill themselves or go out of business. They will seek the security systems they can afford.

He who falls for those tricks deserves the fate. The one trusting 25million to an employee without control mechanism is also to blame. Another kind of AI trick: Self-delusions source: https://x.com/i/status/2003239718478033147

I find it extremely hard to believe. I see a lot of AI-generated videos on Social media, and I can detect them within a few seconds. We are talking about a video conference here where AI had to match the voice, face, and everything, which is nearly impossible. Even if AI can do this, it should not be that hard to notice the difference. If this is a real news, I would expect some referance link so I can have a read. Even if your AI knows a lot of information about you, it should be between you and the AI. If a 3rd person can get the data, that means you didn't care about your privacy and you are liable for that.

But someone can be in trouble and call with another line, maybe they got involved in an accident and lost their phone or they were attacked and lost their phone and they needed to call and inform their family about the incident so at that point they can't call from the phone numbers that are being saved on our phone. This thing is a problem and I we have to be careful, but sometimes we don't have to neglect picking calls from unknown numbers.  It could actually be someone that knows us that's calling and they need our help or want to pass urgent information that could save life. So my suggestion is this, if someone calls and pretend to be another person by using AI, and request for financial help, ask them few personal questions that only the real person can actually provide the answer, questions that can't be found online.

Anti-AI software is going to become big, I think. I am quite tired of the AI slop and advanced scamming. We need a way to filter these things. AI has its limitations and those limitations should be poked and prodded at until something gives way.

Honestly, security cost is quite alarming that's why individuals and big businesses are replacing human technology with AI Applications, since security check up is another vent of making the poor look helpless in every financial system, verification process with difficult task has made many loss opportunities and means of surviving because average cost is not affordable but we just hope there's is simplest means in future.

That is why I said for smaller amounts, it can be very danger but the amount in that situation might be less than $1000 or even smaller than that. Keep a secondary number too and your family member might know that number in case of such incidents because we get caught in those situation with very closed ones not just every family relative right?

Unfortunately the world is still full of stupid people, there are still many people who haven't fallen for scams like this and they have no reason to raise their guides up until they become a victim. Ordinary home invasion looks like something that is so unreal in the face of many people today because they are living in a country where they can leave their doors open at night and nothing will happen. About the number, may we not become a victim to evil situations, what if the person whom the phone number belongs to is already under captivity? Depending on the situation it will be very hard to tell, because they can still use the phone number to make that call, if that's your pass sign you still fail.

This is totally different situation, the person under the control of the one who is asking for money then we mght not have option other than paying the ransom that they were asking for because the other option is to contact the police which may work or not but we can't take chances with a closed one's life. And if we are capable of paying the money then we could do and secure the one that should be the priority because we can earn back the money later but not the life once it's lost.

What you are talking here is pure kidnapping. There is not much we can do instead of sending the money in this situation because no matter what, we can't take the risk of our loved ones getting hurt. For that, we will do anything possible and people mostly even prefer not to involve cops and resolve the matter quickly by giving the money. This is a much safer option but sometimes even the police can be of much help here depending on their resolution time. These kinds of kidnappings were quite common back then but as the rules and regulations have tightened, we rarely see any such scenario. Scammers can still use AI to fake the kidnapping by spoofing the cell phone or by creating fake pictures and videos. Most people will be trapped here and will think this is for real and will send the money without having any extra thoughts. This is quite dangerous to be honest, so I believe in whatever OP said.

Twenty five million dollar is a really big amount. I wonder why someone would make sure a big transaction over the phone. This might be the result of reckless behaviour of that employee. In the first place, why did he/she think that his/her colleagues or boss would ask for such a big amount on a single call. I do not really know the entire story yet so can't make any assumptions here. Is there any official link to this story? I would love to read the entire scenario and maybe try to learn from that story. For me, no matter how real it looks, I would never make any transaction just because we had a virtual meet. I would first physically meet the person before making this big transaction. Yes, AI is quite dangerous and there are people who are always looking for ways to scam others. AI is like a magic genie for them. They can customise their scams and find more advanced ways to scam people. It's actually quite difficult to control and all we can do is be vigilant and try to be as safe as possible.

It's a good point, but I'd still be wary about believing some of the news stories that come out. There are a lot of silly people who will fall for scams quite easily, even with plenty of training given, but we can all be susceptible if caught at the wrong time with the right combination of factors. We don't actually know the full details of what was involved in this particular scam, sometimes it will be glamorized by the victim because otherwise they might lose even more business and reputation. There will need to be an evolution in protocols as you say, but unless there are offline or separate secure channels for communicating, then all the same vulnerabilities will continue to exist. It does make for interesting discussions though, as AI has come along rather quickly and can be very realistic even in these early stages.

$25M is a huge deal and should never be executed by confirmation from a video call. Even you're too rich, that is still a big enough money that needs to be handled carefully and such big deal can be splitted to different transactions in order to reduce risk of both scam and mistakes when broadcasting transactions. If you complete your business deal with a single transaction, if you make mistake when broadcasting that transaction, you lose your bitcoin forever while the deal will be failed. Big or small, it depends on financial and wealth status of a company or a person, but generally $25M is not a small fund with anyone or any entity. How many Bitcoin confirmations is enough? Number of confirmations for sealing a transaction and a business deal depends on its value too.