With the current boom in AI-generated content, the risk management world faces unprecedented challenges in keeping e-commerce safe and usable. Through machine learning, users can create digitally manipulated images and videos of human subjects that still look authentic to the untrained eye. These forgeries are called deepfakes.
The unsavory side of this technology, namely sexually explicit deepfakes, has caught some platforms flat-footed. This January, we saw content moderators across social media scrambling to remove pornographic deepfake material of Taylor Swift—and the targets aren’t just celebrities. Some deepfake sites advertise services that can digitally undress people in ordinary photos.
Over the past few years, G2 Risk Solutions has monitored this industry and protected our clients from its reach. Most deepfake porn sellers stay underground and launder payments through peer-to-peer payment apps, cryptocurrencies, and seemingly benign fronts. For example, G2 Risk Solutions investigators found that one prominent deepfake porn site funneled their transactions through a merchant that sells jumbo bean bag chairs.
The deepfake site’s payment page promises that “No mention of the adult website will be on your bank statement.”
They do this through trickery—bank statements will show charges for bean bag chairs.
In a similar case, another deepfake site used a front that purported to sell dancing classes.
G2 Risk Solutions has discovered that bad actors may cycle through multiple fronts within a network to avoid raising red flags with fishy transactions. A fully automated system is unlikely to find cases like this. They require a human touch to investigate, and that’s where we excel. G2 Risk Solutions analysts are knowledgeable and adaptable enough to tackle new industry trends, including AI-generated content.
If you have questions or would like to learn more about deepfakes, contact us.