CSV Is Never Just CSV – Maschinenraum

When I started building CSV Unwrap, I wasn’t particularly worried about parsing CSV files.

That’s a solved problem.

What interested me was something else.

Could software look at an unfamiliar dataset and help explain what you’re actually looking at?

Not just display rows and columns.

Help build a mental model.

That turned out to be a much harder problem than I expected.

It Started With A Simple Question

CSV is probably the most boring file format we use.

Rows.

Columns.

Commas.

Almost every programming language already knows how to read one.

And yet opening an unfamiliar CSV still feels surprisingly awkward.

You scroll around.

Read a few column names.

Guess what they contain.

Search for something recognizable.

Only after a few minutes do you start understanding what you’re actually looking at.

That made me wonder:

Could software help with those first five minutes?

Not by running sophisticated analyses.

Just by making unfamiliar data feel less unfamiliar.

CSV Isn’t Complicated

One assumption disappeared almost immediately.

I expected datasets to be reasonably clean.

Instead I found columns that were almost entirely empty.

Dates stored as Unix timestamps next to ISO strings.

IDs that looked like numbers.

Numbers that were actually categories.

Columns called value.

Columns called data.

Columns that seemed obvious until they weren’t.

CSV isn’t complicated.

People are.

Every export reflects years of product decisions, naming conventions, business rules and forgotten edge cases.

The file is just where all of that ends up.

The Viewer Became The Real Project

I honestly thought the viewer would take a weekend.

Display rows.

Add scrolling.

Done.

Instead it became one of the most interesting parts of the project.

Large files can’t simply be rendered all at once.

Rows need to appear only when they’re visible.

Data has to be loaded on demand.

Memory usage needs to stay predictable.

Scrolling has to feel effortless.

The funny part is that none of this should be noticeable.

A good table viewer shouldn’t feel clever.

It should feel obvious.

Which usually means somebody spent a lot of time making difficult things disappear.

Empty Cells Tell Stories

One discovery surprised me more than anything else.

Columns that are almost entirely empty often contain the most interesting information.

Maybe they’re only populated when a payment fails.

Maybe they represent an exceptional workflow.

Maybe they’re only used by one customer.

A profiler reports:

98% null values.

A person reads something different:

This field is almost never used.

Those aren’t the same thing.

Statistics describe the data.

Context explains it.

Types Explain Less Than You Think

Detecting primitive types is easy.

String.

Number.

Boolean.

Date.

Useful.

But rarely sufficient.

These are all numbers.

Age

Revenue

ZIP code

Customer ID

Temperature

Technically identical.

Semantically completely different.

At some point I stopped asking:

What type is this?

and started asking:

What is this column trying to represent?

That turned out to be the much more interesting question.

Confidence Is A UX Decision

One of the trickiest problems wasn’t detecting semantic types.

It was deciding when to show them.

Internally every prediction has a confidence score.

That sounds objective.

It isn’t.

Use a very conservative threshold and the tool is almost never wrong.

It also recognizes almost nothing.

Lower the threshold and suddenly it becomes much more useful.

But every now and then it’ll confidently call an identifier a year.

Or mistake a code for a currency.

There isn’t a perfect threshold.

Only a trade-off.

The interesting part is that this isn’t really a machine learning problem.

It’s a UX decision.

Every feature has a price.

Every New Feature Adds A Decision

One experiment I’m still thinking about is the card view.

I actually like it.

Especially for datasets containing long text, nested JSON or many descriptive fields.

Rows become something you can actually read instead of something you scan horizontally.

The interesting part wasn’t the implementation.

It was what happened afterwards.

Now there were two equally valid ways to explore the data.

The table.

And the cards.

Both useful.

Both solving different problems.

But every new feature also asks users one more question.

Which one should I use?

Adding functionality is easy.

Keeping software simple is much harder.

DuckDB Quietly Solved A Lot Of Problems

Every project has one dependency that quietly does most of the heavy lifting.

For CSV Unwrap, that’s DuckDB.

I originally picked it because it looked like a convenient way to query CSV files.

It ended up solving several problems I thought I’d have to solve myself.

Sometimes the biggest engineering decision isn’t what you build.

It’s what you don’t have to build.

Where Things Get Interesting

The more I worked on the project, the...