Generalized Audio-Driven Synthesis of Precise Drummer Motion | Disney Research Studios

Select Page

Generalized Audio-Driven Synthesis of Precise Drummer Motion

In this work, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics.

August 18, 2026

arXiv (2026)

Authors

Álvaro G. Iñesta (The Blavatnik School of Computer Science and AI, Tel-Aviv University)

Mattia Ryffel (DisneyResearch|Studios)

Amit H. Bermano (DisneyResearch|Studios/The Blavatnik School of Computer Science and AI, Tel-Aviv University/ETH Zurich)

Robert W. Sumner (DisneyResearch|Studios/ETH Zurich)

Martin Guay (DisneyResearch|Studios)

Generalized Audio-Driven Synthesis of Precise Drummer Motion

Download Publication PDF

Abstract

Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.

Research at Disney

Home

Research

People

Careers

Publications

Outreach

About Us

Legal

Privacy Policy

California Privacy Rights

Children's Online Privacy Policy

Terms of Use

MORE

News