Steeping through data and brewing a database — PV;WTF

Steeping through data and brewing a database

2026-08-01

I like drinking tea. I have fond memories of the little bags of lemon tea that my father drank when we were young. Around the same time period a wayward cup of coffee somehow made its way down my stomach where it proceeded to cement itself (based on the stomachache), and coffee went way down the list of preferred drinks. Doesn't help that the rest of the family always drank it strong enough that the spoon would stand up unassisted. Nowadays I can go for a late night Irish Coffee or Scanian Kaffegök, but that's it.

Later, around the time I went to university there was this wonderful little shop with metal tins lining the walls, each containing its own interesting blend of tea and smells that would follow you out after leaving. It was a great visit every time and is where my tea drinking started to really pick up.

Though, I've never really got into more of the pure single origin teas, the T.G.F.O.P. and Pu-Ehrs of the world. My favorites are all black tea blends, a good Earl Grey with the citrusy zest, something a bit more floral, or maybe with berries? Ideally with natural flavor and not towards the super synthetic end of the flavor spectrum.

Case in point, I was on vacation earlier this year, and strolling through a market I found a nice little stand selling teas. The guy behind it was great and he let me sniff out a few of the fancy single origin teas he had for sale, but alas, no black blends in sight. So on the ride back my brain was catching on a potential new side project, how hard would it be to make your own tea blend? Just need to source some tea, and some ingredients, which shouldn't be that hard. But the quality is important, so would need to get ahold of some Calabrian bergamot oil for example. And if we're doing all of that, why not be a bit more creative. What else could I put in?

What else have people already tried to put in their tea blends?

So as always, instead of focusing on the core problem of possibly, maybe, making a tea blend, I thought about building a small database of the existing ones. What has been tried? What do people like? What flavor combinations have actually not been tried?

Plucking up leaves of data

To understand the different combinations of tea bases and ingredients I had my sights set on scraping a few different suppliers.

Tehuset Java, the Swedish one where I already know they are up to no good, wildly mixing and matching to make interesting blends. They also carry Kusmi tea, a french tea house with similar proclivities so it becomes our second candidate. Interestingly France seems to have a knack for this style of tea so we're throwing Dammann Frères in the mix as well. While the Germans are more into their fruit infusions we're giving them a shot through Paper & Tea and TeeGschwendner. At the end, the danish Østerlandsk also landed to round out the nordics.

While I'm in for the special blends, being respectable purveyors of tea they all have single origin pure tea as well, so if we're building a library of tea why not grab them all.

Writing all of this down and asking Claude to very kindly create an artifact to understand how far in over our heads we are led to this first prototype.

Teas on one side, details on the other. The pages seem to have enough information for this to be viable, they might be hidden in the description, in json data or in accordions, but they are there. Check on plausibility, water not even up to the ankles.

At this point our data pipeline consists of asking a magic chat to make me a website with teas. It feels magical, but being a single step is neither a data pipeline nor a trustworthy one.

Tea?

Magic

Website

No, we should to save the mana/tokens/spell slots for later. Instead let's have it build its own replacement data pipeline, one where we can re-run parts of it incrementally as we learn all the weird and varied ways that you can describe teas on the internet.

It would be nice if we could just throw a http request at the websites of our chosen suppliers and get everything back in one go, the whole list of teas and all of their details? Actually, Paper & Tea is Shopify based so they do serve a big JSON blob of all products. The rest need a bit more work, though Kusmi and TeeGschwendner have a helpful sitemap we can use. Java and Dammann Frères have neither so we need to navigate through all of their category listings to find all teas. I don't think any of them were fully client side rendered, otherwise we'd have to prep the headless harness as well.

Each site thus gets a different little spider, collecting a list of URLs to all the potentially tasty teas. Then we can filter it to remove things like tasty chocolate (not tea), pretty gifts sets (is tea, but not loose), different size packaging (100g vs 500g of Earl Gray might make a difference if you've run out of essential oil for your bath, but my tea cup is...