Building a Real-Time Chord Recognizer

Under the Hood: Building a Real-Time Chord Recognizer | WhatChord

The problem is not a lookup

The first intuition when building a chord recognizer is to build a dictionary. There are only 12 pitch classes, which means there are only 2^12 = 4096 possible pitch-class sets. Store a name for each set, and when a user plays C-E-G, look up {C, E, G} and return “C major.”

The problem is not memory. Four thousand entries is trivial. The problem is meaning. A pitch-class set does not contain enough information to decide what musicians will call it.

Piano players often leave out notes that a dictionary entry might expect. Extended chords add notes that no fixed dictionary entry anticipates. And the same set of pitch classes, as discussed in the companion article, can legitimately be described as multiple different chords depending on musical context.

What you actually need is a scoring model. It has to evaluate how well any given set of notes fits each chord type, rank all plausible interpretations, and apply musical judgment when scores are close.

Overview: a four-stage pipeline

Before diving into each component, here is the overall shape of the algorithm. A snapshot of sounding notes enters at the top; a ranked list of chord interpretations comes out at the bottom.

Input: set of sounding pitch classes + lowest (bass) note

Pitch-class bitmask

12-bit integer: one bit per semitone in the octave

Candidate generation

Each sounding note becomes a candidate root, scored against every chord template, extensions extracted

Score normalization

Raw scores are normalized for fair comparison across chord complexities

Ranking

Musical heuristics resolve ambiguous scores; hard structural rules override when the score alone would pick the wrong answer

Output: top ranked chord candidates, result cached in LRU

The rest of this article walks through each stage in detail, ending with a discussion of known limitations.

Pitch classes and bitmasks

WhatChord models the common 12-tone equal temperament (12-TET) pitch-class framework used by MIDI keyboards, which divides each octave into equal semitone positions. A pitch class is the note’s position within that octave, ignoring which octave it’s in, so middle C, the C above it, and the C three octaves below all share pitch class 0. In this engine, pitch classes are numbered 0 (C) through 11 (B).

For analysis, the engine collapses the sounding notes into a set of pitch classes plus the lowest sounding note as bass. The pitch-class set is represented as a 12-bit integer mask where bit n is set if pitch class n is present. C major (C=0, E=4, G=7) looks like this:

11 10

A♯ G♯ F♯ D♯ C♯

// Pitch classes: C=0, E=4, G=7 int pcMask = (1 0) | (1 4) | (1 7); // pcMask == 0b000010010001 == 0x091

This representation is compact and fast. Checking whether a pitch class is present is a single bitwise AND. Counting present pitch classes is a popcount. Rotating the set relative to a candidate root is a loop over bits with modular arithmetic. All of these operations are cheap.

A key design decision: only pitch classes actually present in the voicing are tested as candidate roots. There are no “ghost roots” and the algorithm never proposes an interpretation where the chord is rooted on a note that is not being played. This keeps the candidate count small (bounded by the number of sounding notes, typically 3–7) and avoids obviously wrong readings.

This is a deliberate “solo keyboard” assumption. The current engine is optimized for the common case where the same MIDI stream contains both the harmony and the bass note. A future ensemble mode could relax that rule for settings where another instrument is carrying the bass, allowing rootless voicings to imply roots that are not literally present in the keyboard part.

Chord templates

Chord qualities are also defined as bitmask templates. Each one describes three sets of intervals relative to the root:

Required: tones that must be present to identify this quality. Missing more than one required tone causes the template to be skipped entirely.

Optional: tones frequently omitted in real voicings (almost always the perfect 5th). Present when played, unremarkable when absent.

Penalty: tones that actively contradict this quality. Having a major 3rd present when you are trying to identify a minor chord hurts the score.

The 22 templates, organized by complexity:

Quality Required intervals Optional Key penalty tones

Major R, M3 P5 m3, m7, M7

Minor R, m3 P5 M3, m7, M7

Diminished R, m3, ♭5 M3, P5

Augmented R, M3, ♯5 m3, P5

Sus2 R, M2, P5 m3, M3, m7, M7

Sus4 R, P4, P5 m3, M3, m7, M7

Major 6 R, M3, M6 P5 m3, m7, M7

Minor 6 R, m3, M6 P5 M3, m7, M7

Dominant 7 R, M3, m7 P5 M7, m3

7sus2 R, M2, m7 P5 m3, M3, P4, M7

7sus4 R, P4, m7 P5 m3, M3, M7

7♭5 R, M3, ♭5, m7 P5, M7, m3

7♯5 R, M3, ♯5, m7 P5, M7, m3

Major 7 R, M3, M7 P5 m7, m3

Major 7sus2 R, M2, M7 P5 m3, M3, P4, m7

Major 7sus4 R, P4, M7 P5 m3, M3, M2, m7

Major 7♭5 R, M3, ♭5,...

Building a Real-Time Chord Recognizer

Related Articles

Elevated error rates on requests to multiple models

Donald Trump and sons to be 'forever' exempt from tax audits

PopuLoRA: Co-Evolving LLM Populations for Reasoning Self- Play

Old Reddit Is Down

The ultimate female fantasy – A feminist critique of Beauty and the Beast