OpenTSLM: Teaching Large Language Models to Understand Time | Aionic Labs<br>Skip to content

Loading…<br>SUPPORTED BY

© AIONIC AG 2026

GET IN CONTACT→OpenTSLM

Teaching Large Language Models to Understand Time<br>OpenTSLM: Time Series Language Models for Reasoning over Multivariate Medical Text- and Time Series Data<br>Patrick Langer1, 2, 3, Thomas Kaar1, 3, Max Rosenblattl1, 3, Maxwell A. Xu4, 5, Winnie Chow6, Martin Maritsch7, Robert Jakob3, Ning Wang3, Juncheng Liu8, 9, Aradhana Verma10, Brian Han11, Daniel Seung Kim12, Henry Chubb11, Scott Ceresnak11, Aydin Zahedivash1, Alexander Tarlochan Singh Sandhu10, Fatima Rodriguez10, Daniel McDuff5, 13, Elgar Fleisch2, 3, 14, Oliver Aalami1, Filipe Barata2, *, Paul Schmiedmayer1, *

AUTHOR AFFILIATIONS→1STANFORD MUSSALLEM CENTER FOR BIODESIGN, STANFORD UNIVERSITY, USA, 2CENTRE FOR DIGITAL HEALTH INTERVENTIONS, ETH ZURICH, SWITZERLAND, 3AGENTIC SYSTEMS LAB, ETH ZURICH, SWITZERLAND, 4UNIVERSITY OF ILLINOIS URBANA-CHAMPAIGN, USA, 5GOOGLE RESEARCH, USA, 6STANFORD UNIVERSITY, USA, 7AMAZON, USA, 8MICROSOFT, 9NATIONAL UNIVERSITY OF SINGAPORE, SINGAPORE, 10DIVISION OF CARDIOVASCULAR MEDICINE, STANFORD UNIVERSITY, USA, 11PEDIATRIC CARDIOLOGY, STANFORD UNIVERSITY, USA, 12DIVISION OF CARDIOLOGY, UNIVERSITY OF WASHINGTON, USA, 13UNIVERSITY OF WASHINGTON, USA, 14CENTRE FOR DIGITAL HEALTH INTERVENTIONS, UNIVERSITY OF ST. GALLEN, SWITZERLAND, *EQUAL CONTRIBUTION<br>ACCEPTED AT

READ THE PAPERGITHUB REPO★1.2k

What are Time Series Language Models?<br>Time Series Language Models (TSLMs) are language models that can take raw time series as input alongside text. Instead of converting signals into plots, summaries, or long strings of numbers, a TSLM reads the signal directly and answers in natural language.<br>This makes them flexible in the same way multimodal chat models are flexible for images: you can attach one or many signals, combine signals of different lengths and sampling rates, ask open-ended questions, request explanations, compare windows, or ask for a forecast without training a separate model for each task.

How OpenTSLM works<br>With OpenTSLM, we present a multimodal LLM that treats time series as a native modality. It can reason over raw multivariate signals alongside text, and it is designed to scale to multiple very long time series without forcing the signals through the language-model token stream.<br>The paper compares two ways to fuse text and time series, OpenTSLM-SoftPrompt and OpenTSLM-Flamingo , on Gemma3 and Llama3.2 backbones at 270M, 1B and 3B scale. SoftPrompt places time series tokens in the language-model input. Flamingo uses cross-attention, allowing the language model to query compact time series representations instead of processing every signal point as a text token.<br>OPENTSLM-FLAMINGO ARCHITECTURE

Experiments<br>We compare OpenTSLM against the main ways people try to make language models handle time series today: serialize the values as text, fine tune on serialized values, show the model a plot, or use GPT-4o with image inputs.<br>OpenTSLM outperforms every baseline across time series question answering, activity recognition, sleep staging, and ECG question answering. Even 1B-parameter OpenTSLM models surpass GPT-4o, showing that time series capability comes from the model interface, not just from making a general model larger.<br>BEST MACRO-F1 PER METHOD FAMILY · HIGHER IS BETTER ↑<br>OpenTSLM<br>Image-based<br>Tokenized<br>GPT-4o

TSQATIME SERIES QUESTION ANSWERING<br>OpenTSLM

99.71<br>Image-based

94.94<br>Tokenized FT

84.54<br>GPT-4o

59.24<br>Tokenized

16.24

HAR-CoTACTIVITY RECOGNITION<br>OpenTSLM

69.31<br>Tokenized FT

60.44<br>Image-based

44.01<br>GPT-4o

10.83<br>Tokenized

0.00

Sleep-CoTSLEEP STAGING<br>OpenTSLM

69.88<br>Image-based

18.56<br>GPT-4o

15.47<br>Tokenized FT

9.05<br>Tokenized

5.66

ECG-QA-CoTECG QUESTION ANSWERING<br>OpenTSLM

41.92<br>Image-based

26.17<br>GPT-4o

24.95<br>Tokenized FT<br>OOM<br>Tokenized

0.00

BARS SHOW BEST MACRO-F1 WITHIN EACH METHOD FAMILY · OOM = OUT OF MEMORY | LANGER ET AL., 2025

ECG-reasoning validated with 7 cardiologists<br>We evaluated 84 rationales of our best model, OpenTSLM/Llama3.2-3B, through expert review by 7 Stanford Hospital clinicians, finding the reasoning correct or partially correct 97% of the time. They used a rubric based on ACC/AHA ECG competence guidelines and the RIME framework, focusing on ECG feature identification, reasoning quality, and use of patient context.<br>ECG REASONING VALIDATED BY SEVEN CARDIOLOGISTS | STANFORD HOSPITAL<br>POSITIVE<br>PARTIAL<br>NEGATIVE

97%<br>of 84 reviewed rationales from OpenTSLM/Llama3.2-3B rated correct or partially correct, using a rubric based on ACC/AHA ECG competence guidelines and the RIME framework.

PATTERN RECOGNITION

CLINICAL REASONING

CONTEXT INTEGRATION

Scalability unlocked<br>We, for the first time, present an architecture for TSLMs that enables scaling to multiple multivariate and long time series. While prior methods typically scale quadratically in memory with longer time series, our implementation achieves comparable performance with...