abigail phoebe // 8% of images on Discord are scams: automatically detecting mr. beast scams on Discord<br>my other websites

abigail<br>.sh

abigail phoebe<br>software engineer & scientific researcher

e-mail[email protected]github@ijsbollinkedinin/abigailphoebe

e-mail[email protected]github@ijsbollinkedinin/abigailphoebe

8% of images on Discord are scams: automatically detecting mr. beast scams on Discord

Have you ever seen those crypto-scams on Discord where they post 4 images, usually starting with Mr. Beasts Twitter profile?

They look something like this (though this is a parody, obviously).

I kinda went somewhat viral on Twitter recently over my solution to these scams, so by popular demand, here's a write up on how it works!

I got annoyed of having to delete these in my communities (I run a Discord partnered server, as well as around 10 others), so I looked around for the available solutions people have already made. There were many, though none quite felt good enough for me.

One of the most popular available solutions to these scams are the use of "honeypot" channels. These are really simple: a channel is created by the bot, and if any user sends a message in them, they are banned. This works well when bots just spam in every single channel they have access to, though it also catches innocent users who don't read the channel's information, not to mention how easy it is for scam developers to bypass, as all they would need to do is develop a simple check if a channel is a honeypot channel and then avoid it.

Another popular tool I came across relied on perceptual hashing to detect the scams, this similarly had some shortcomings. There are several types of perceptual hashing algorithms, though all we tried had the same issues; at its core a perceptual hash comparison will only match if the images are either exact matches, or very very similar matches.

Since the Mr. Beast scam images use photos of monitors with different brightness, cropping, angle of the photo being taken, etc, to a computer they look entirely different. While we as humans can easily see that two different images of a monitor are the same, a computer cannot if it is using a simple algorithm such as perceptual hashing. Instead it can only match to images in the same scam-batch, which once flagged by a human, is already too late and has been sent hundreds of times across Discord.

This is not to mention other issues with common perceptual hashing algorithms such as a fairly small bit-space, which leads to two completely different images being matched as the same image (something we ran into a few times during testing).

As both of these popular detection methods were lackluster at best, and since I am a computer scientist, I figured I'd try and make my own (why spend 5 seconds doing a task which I can spend 5 days automating?).

Introducing Scrump Anti-scam: A roughly 10-stage detection pipeline that not only has a near 100% match rate, but is yet to have any credible false positives, can adapt in real-time to new scam images, and can also run entirely locally on a fairly small Hetzner server while handling over 200 messages per second.

Before I get into the details of this post, I want to quickly highlight that none of your images are ever saved by Scrump, nor are they sent to third-parties. Scrump follows EU privacy laws, and is hosted locally in Europe.

I will not be going into detail on the exact algorithms and detection pipeline Scrump uses, as doing so would help scammers bypass its detection. However I will outline at a high level what we are doing so that anyone with a bit of background knowledge on the subject can reasonably figure it out.

As I already mentioned, perceptual hashing is just not good enough for our task, so we needed something different. We initially thought about running OCR (optical character recognition, i.e. read text from an image) on every image and then checking the text, which works perfectly!..... for small servers. When you only need to scan a few images per minute, OCR is great! However, when building at Scrump's scale, without using a paid, privacy-issue raising, and unknown external API to perform the OCR, it ends up eating the entire CPU of our server, since uh... well, we get "a few" images per second.

What if we used perceptual hashing to detect similar enough images, and then did OCR? This was another idea we tried, though this would still miss some known images and also sent too many irrelevant images to OCR (thus blowing up the CPU...).

Eventually, we settled upon a custom implementation of an algorithm developed by Meta to detect CSAM as our first-pass filter (to clarify: this is all running fully on-device, absolutely no data is ever shared with Meta or any other third parties). This filter simply tells us "this is definitely not a scam" or "i don't know".

From there, any image flagged "i don't know" is sent into a further pipeline to check against known images using a variety of hashing...