How to Validate Against an Unlabeled Kaggle Test Set

How to Validate Against an Unlabeled Kaggle Test Set | by Alan Scott Encinas | Jun, 2026 | MediumSitemapOpen in appSign up Sign in

Medium Logo

Get app Write

Alan Scott Encinas

3 min read· Just now

Listen

Picture a final exam where you hand in your answers and the professor never gives you a grade. You get three guesses for the entire term and a single sealed result at the very end. Study as hard as you want. You will not learn whether you were right until it no longer matters. That is the leaderboard I am climbing. This is the second entry in a log I’m keeping while I actually compete. The first wall I hit was not the modeling. It was the discovery that the scoreboard I am chasing cannot tell me my score. A quick reset If you just landed here: I’m competing in the Hyperspectral Object Tracking Challenge 2026, a Kaggle contest tied to an academic conference. You get one object, boxed in the first frame of a video shot in colors the eye was never built to see, and you have to follow it through every frame after. The last entry covered why that is hard. This one is about the part that comes before any of the tracking matters. The mirror that doesn’t work The test set they score you on is unlabeled. You get the first-frame box and nothing else. No answer key, no ground truth, no way to check your own work. The public leaderboard is the only mirror you have, and it is a cruel one. You see your number only when you spend a submission, and submissions are rationed. Develop that way and you are tuning a race car by feel, in the dark, allowed to glance at the speedometer twice a day. There is a way out, and building it was the most important thing I did this whole stretch. The training data is labeled. Four hundred and five sequences, every frame boxed, ground truth included. So you stop chasing the leaderboard and build your own scorer at home, one that grades against that labeled set whenever you want, as many times as you want. A scorer is worthless until it agrees with the real one The work was not writing the scorer. The work was proving it matched. A homemade metric that disagrees with the official one is worse than nothing, because it lies to you with confidence. So I made my local scoring code byte for byte identical to Kaggle’s official scorer, then cross-checked the whole thing against the official scorer written in a completely different language. They agree. My laptop now tells me my competition score before I ever submit anything. Why this reorders the whole project That one fact changes what a submission is for. A submission used to be how I found out my score. Now it is a scarce confirmation of something I already measured at home. The leaderboard stopped being my compass and became a checkpoint. The compass lives on my own machine. Here is the part no after-the-fact writeup ever shows you: in a competition like this, most of the real work happens before you touch the model. You are building the instrument that tells you the truth, and you are making sure it does not lie. A faster tracker means nothing if you cannot measure it. Measurement is the foundation the rest of the project stands on, and it is completely invisible unless someone shows it to you while it is being poured. Where I’m standing right now Still 0.524. Still about three and a half hundredths under the podium line. The number has not moved. But I can see it now without spending a submission, which means the real work, the part where the number actually moves, can finally begin. There is a darker comedy I skipped over. Before this harness could help me with anything, my very first submission to Kaggle did not score at all. It was rejected before the scorer even looked at it, for a reason I never saw coming. That is the next entry. More in this series This is part of an ongoing builder’s log written from inside live competitions. You’re reading where I was, not where I am.

Originally published at https://alanscottencinas.com on June 24, 2026.

Machine Learning

Kaggle

Data Science

Model Evaluation

Build In Public

Written by Alan Scott Encinas 13 followers ·18 following

AI Engineer & Systems Architect. I turn complex ideas into working systems: cognitive AI, autonomous systems, robotics, defense.

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech

How to Validate Against an Unlabeled Kaggle Test Set

Related Articles

US Government directive to suspend access to Fable 5 and Mythos 5

Is AI ruining our skills? Early results are in – and they're not good

The Anatomy of an AI-Native Org

Apertus – Open Foundation Model for Sovereign AI

Britain Became as Poor as Mississippi