How We Built a Self-Calibrating POI Map from Human Input, Data Agents and AI | Foursquare

Skip to content

Search

Log In

Studio

Attribution

Audience and Proximity

Developer Console

Placemaker Tools

Speak to Sales

Resources / Blog / Products

How We Built a Self-Calibrating POI Map from Human Input, Data Agents and AI<br>Inside the FSQ Places Engine: Part 1

December 18, 2025 by Fourquare

One year ago, we introduced Foursquare’s new Places Engine, a unique crowdsourcing platform that brought together humans and agents to create a comprehensive POI (point-of-interest) dataset. We built this system to find consensus  from conflicting inputs and anchored it with a strong spatial foundation  to ensure every POI record in our database matches the physical reality. The result is something fundamentally different from a traditional POI system: a self-calibrating, living representation of POIs that continuously reasons about every input it receives. Rather than simply storing the data, the engine weighs new evidence against existing knowledge, calibrates the reliability of every contributor based on outcomes, and dynamically refines place records, all without manual intervention.

In this blog post, we explore how our central consensus engine  operates as a meritocracy driven by inputs from three distinct types of contributors (Humans, Data Agents, and AI Agents) , and examine the workflows that enable continuous calibration and feedback  between them.

The Three Types Of Contributors And The Consensus Engine

Our system receives proposals for new places or changes to existing places from three different types of contributors: a) Humans who use our apps and Placemaker tools to directly report changes to locations they know or vote on edits proposed by other users, b) Data Agents which monitor digital sources like websites, data feeds, and partner systems, match them against existing places in our database to understand differences and propose new places or edits to existing matched places and c) AI Agents which encapsulate various machine learning and large language models to either proactively report closures of places or vote on edits made by other contributors.

The central consensus engine evaluates these proposals (also known as ‘woes’) and decides whether to accept them based on alignment between different contributors. Rather than relying on a simple majority, the engine, built on a modified version of the Dawid-Skene algorithm, operates as a meritocracy: each contributor carries a trust score calibrated at the attribute level, giving greater weight to those with a proven track record of reliability. Once a decision has been made, the system calibrates every participant involved to either boost or penalize the trust score associated with that participant.

Let’s say, a partner-feed reports that a neighborhood gym is “Open,” but a long-time Placemaker reports it as “Closed.” In a simple majority system, this might be a tie, but the resolution is a continuous loop of verification and grading:

First, the system acts as a judge . It weighs the conflicting testimony based on history. It recognizes that the Human Placemaker has a 99% accuracy rating for venue closures, while the automated feed has a history of lagging behind real-world changes. Based on this weighted evidence, the system determines that the gym is, in all probability, closed.

Once that truth is established, the system acts as a teacher . It effectively “grades” the contributors based on the ruling it just made. Since the Superuser was on the side of the truth, their history is reinforced, slightly boosting their influence for the next time. Conversely, the automated feed, having provided incorrect information against the consensus, takes a penalty to its reliability score. This ensures that if this specific feed continues to be wrong about closures, its voice will matter less and less in future decisions.

This judge-and-teacher loop serves as the foundation of the consensus engine. It also layers in additional calibration techniques to handle more nuanced scenarios detailed below:

Contextual Trust:  First, it recognizes that trust is contextual rather than binary. Consider a business owner. They are the ultimate authority on their own Opening Hours, so the system assigns them a near-perfect trust score for that attribute. However, they are often unreliable when inputting their Venue Name, frequently cluttering the field with marketing slogans like “Best Pizza in NY” instead of the official registered name. The model accounts for this by maintaining separate trust scores for each contributor per attribute type, trusting the owner on logistics but verifying their strings against other sources.

Cold Starting Data Agents:  Second, the system solves the “cold start” problem for new Data Agents using Bayesian Priors. When a new data partner, such as a new Listing Syndicator, joins the network, we do not force them to start with a score of...