XGBoost beat LLMs at finding civilian-harm posts in Ukraine war Telegram data

Jimmc4141 pts0 comments

How to Use AI to Help Find Civilian Harm - bellingcat

How to Use AI to Help Find Civilian Harm - bellingcat

Search for:

Investigations

Resources

Ukraine

Workshops

Donate

Miguel Ramalho

Miguel is an Investigative Technologist for Bellingcat. He uses data and code to investigate and communicate stories, he experiments and builds research tools with and for the online investigations community.

Nick Waters

Nick is an expert who specialises in the examination of conflict using online open source information, including as evidence in justice and accountability processes. He began contributing to Bellingcat in May 2016 and was a staff member from May 2019 to March 2025. His previous work has focused on the use of chemical weapons in Syria, violence against migrants on the borders of the EU and civilian harm during Russia’s full scale invasion of Ukraine.

How to Use AI to Help Find Civilian Harm

June 25, 2026

AI

Civilian Harm

Ukraine

Between February 2022 and September 2025, Bellingcat staff and volunteers collected, geolocated, and shared more than 2,500 incidents of civilian harm following Russia’s full-scale invasion of Ukraine.

As part of this effort, Bellingcat tested a new machine learning model intended to rank Telegram social media posts on their likelihood of containing incidents of civilian harm.

This novel methodology dramatically reduced the search and selection time required, freeing researchers to focus on verifying incidents of civilian harm – not just searching for them.

This piece documents our methodology, ethical considerations and lessons learned in the hope that others researching similar topics can benefit from our work.

Open source research into civilian harm is still a relatively new field and it presents many challenges – one of the biggest is organising and sorting through the huge volume of user generated content being produced to find what is relevant.

Machine learning, a form of artificial intelligence that uses algorithms to identify patterns from large amounts of data and make predictions, can make this task more efficient.

With ongoing conflicts involving large amounts of civilian harm occurring in Sudan, and much of the Middle East, this guide aims to offer those covering these conflicts an example of how machine learning can be used to help find and sort incidents. You can also access the Code Notebook for our model here.

We defined “civilian harm” not just as civilian deaths or injuries resulting from armed conflict, but also the broader and delayed effects on civilians from mental trauma, loss of livelihood, displacement, destruction of infrastructure and more. This definition was informed by the Protection of Civilians book on civilian harm.

Initial Telegram Dataset

Each Telegram post containing civilian harm which had already been manually verified by researchers was used to build an initial dataset of confirmed cases of civilian harm, which data scientists call positive instances. We collected a total of 5,848 unique URLs for these Telegram posts. For our manual collection we reviewed posts on relevant Telegram channels, working through oldest to newest posts each day. Assuming that a given post made it to our geolocated incidents list, it meant the researcher who flagged it also looked at the posts that appeared before and after it on Telegram and did not flag those ones, so we selected the 10 posts surrounding the verified civilian harm post as our additional dataset of posts that did not contain civilian harm. After excluding any deleted or duplicate posts, we ended up with 48,545 non-civilian harm posts, our negative instances.

Support Bellingcat

Your donations directly contribute to our ability to publish groundbreaking investigations and uncover wrongdoing around the world.

Donate

The choice to overrepresent negative instances aims at better reflecting the real world and increasing data available for model training.

We enriched each URL with metadata from the Telegram API, such as the time of publication, reactions or textual content. As some of these posts had been deleted, we completed the missing data points with previously preserved versions from our Auto Archiver database, only available for the positive instances.

Feature Engineering

Training a machine learning model requires numerical data, as these models compute a prediction score based on mathematical operations.

We built these by converting raw data from our initial dataset, such as keywords signalling potential civilian harm, into numerical scores (or “features”) that the model could interpret, with the aim of increasing the model’s ability to identify patterns. This process, known as feature engineering, can significantly improve model results because it allows data scientists to suggest explicit context knowledge.

A full list of features we used to train the model can be found in the code notebook accompanying this piece. Many features were directly inspired by...

civilian harm posts data telegram from

Related Articles