Nathan VanBenschoten on X: "https://t.co/fPFd5Vid8h" / X<br>Post

Log inSign up

Post

Nathan VanBenschoten<br>@natevanben

How to build a 256 TB search index

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0">3:57 PM · Aug 5, 202610.7KViews

37<br>50

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0 min-w-0 overflow-hidden">Yash Agarwal

@yashagw

5m

great read man @natevanben

11

span:not(:empty)~span:not(:empty)]:before:content-['·'] [&>span:not(:empty)~span:not(:empty)]:before:px-1 [&>span:not(:empty)~span:not(:empty)]:before:shrink-0 min-w-0 overflow-hidden">Richard Artoul

@richardartoul

31m

this is neat

does the shared WAL with ID-based sharding mean that every indexer has to read every part of every WAL segment and ignore what it doesn't care about, or is the internal WAL format organized in chunks based on the shards so indexers can avoid Show more

68

Log in or sign up for X<br>See what’s happening and join the conversation<br>Continue with phoneContinue with AppleContinue with Google<br>or<br>Log in with username or email

Relevant people

Nathan VanBenschoten@natevanbenFollow<br>chief architect @turbopuffer, previously distributed systems @CockroachDB

Trending now