I (er Claude) wrote an RSS de-duper

scumola1 pts1 comments

rsshose — the firehose, deduped

Export

Grab the OPML export from your reader — tt-rss, Feedly, Inoreader,<br>Miniflux, NewsBlur all have one. It's just your list of feeds.

Upload

Drop it on the upload page. No account, no password — you get a<br>private link that is your feed. Up to 150 feeds, free.

Drink

Read your deduped river on the web or subscribe to it as a single<br>RSS feed in the reader you already use. Updated every 15 minutes.

Happening in the hose right now

Real stories the hose collapsed in the last few hours —<br>each of these was multiple near-identical articles before it got here.

&times;2

More than 1,000 arrested as part of global human trafficking crackdown

Interpol says the vast majority of the 2,070 victims or potential victims identified were women being trafficked for sex.

BBC News<br>7h

also covered by 1 other

More than 1,000 arrested as part of global human trafficking crackdown<br>BBC News

&times;2

Scotland could freeze datacentre projects in challenge to UK’s AI strategy

Scottish government to consider SNP national council motion for moratorium on all new datacentres The Scottish government is about to consider a sweeping moratorium on building new datacentres, putting a key plank of the UK’s AI strategy at risk. Last Sunday the Scottish National…

World news | The Guardian<br>2h

also covered by 1 other

Scotland could freeze datacentre projects in challenge to UK’s AI strategy<br>Technology | The Guardian

&times;2

Katalyst's satellite rescue mission is now in pursuit of NASA's Swift

It will take several weeks for the Link spacecraft to rendezvous with NASA's Swift observatory.

Ars Technica - All content<br>7h

also covered by 1 other

Katalyst's satellite rescue mission is now in pursuit of NASA's Swift<br>Ars Technica - All content

Blunt answers

How does it decide two articles are the same story?<br>A cheap fingerprint pass catches verbatim syndicated copies, then<br>semantic embeddings catch same-story-different-words. It's tuned to<br>never merge two different stories — a surviving duplicate is<br>annoying, a swallowed story is unforgivable.<br>Which article wins a merge?<br>A boring, honest score: source quality, first to publish,<br>completeness. And in your private river the winner always comes from<br>your subscriptions — we never redirect you somewhere you<br>didn't subscribe.<br>What do you do with my feeds?<br>Fetch them politely (conditional GET, backoff) and cluster the<br>articles. Feeds you bring may join the shared pool everyone benefits<br>from. No tracking, no account, no email.

feeds from articles times news covered

Related Articles