Last week, I faced an interesting problem at work. I received a critical warning from AWS regarding our PostgreSQL RDS cluster, it was approaching transaction ID wraparound, something that can cause a full database outage if not handled in time.⁣ ⁣ This was a good reminder of how important it is to understand the internals of the systems we rely on every day.⁣ ⁣ For those unfamiliar, PostgreSQL uses a 32-bit transaction counter (XID) that wraps around after ~2.1 billion transactions. If old rows aren’t vacuumed and frozen in time, PostgreSQL can no longer determine which rows are visible, forcing it to stop accepting new writes to prevent data corruption.⁣ ⁣ In our case, it turned out that autovacuum wasn’t keeping up due to high write throughput and a few misconfigured thresholds.⁣ ⁣ What helped:⁣ ⁣ 👉🏻 Identifying large tables with unvacuumed rows.⁣ 👉🏻 Manually running 𝖵𝖠𝖢𝖴𝖴𝖬.⁣ 👉🏻 Tuning autovacuum workers settings and vacuum cost limits.⁣ ⁣ It’s one of those things that often goes unnoticed until it becomes urgent. If you're running PostgreSQL on RDS (or self-managed), keep an eye on 𝖺𝗀𝖾(𝖽𝖺𝗍𝖿𝗋𝗈𝗓𝖾𝗇𝗑𝗂𝖽) and ensure autovacuum is running efficiently.⁣ ⁣ Invisible debt in infrastructure has a way of surfacing at the worst possible time. This one got our attention early, worth sharing in case it helps others avoid a painful surprise.

Just crossed 100k applications for the 100 introductory Temple devices. To the 100 who get one: you're the founding cohort. We'll be grateful for what you teach us. To the rest: we're building as fast as we can without compromising what @temple needs to be. Follow @temple for updates.

Hello world. The first 100 Temples are ready to ship. We're now inviting athletes, scientists, founders, doctors, creators, and individuals who care deeply about their physical and cognitive health to be the founding users of Temple. Apply for early access at temple.com

Heart Rate is just one of the simpler things. Temple sees more. Follow @temple for more updates.

We're hiring 🚀

Another #Hiring post 🚨 We're also looking for Backend Engineering Interns at @temple. Skillset preferred (again, not limited to): Go, Python, Kafka. Please note that we're only looking for people who're okay with working on-site. Location: Gurgaon, India. Do reach out with your Resumes / Portfolios / GitHub profiles, or tag someone you might know who can be a good fit.

Update: Have received a lot of applications, going through all of them. Will take time.

Coming soon. Follow @temple for more updates.

The worst part about today's @Cloudflare outage was that the dashboard was inaccessible. There was no way we could go ahead and change the DNS records. And to make things worse, even our domain registrar’s site depended on Cloudflare… so changing nameservers wasn’t an option either.

I’m not sharing this as the CEO of Eternal, but as a fellow human, curious enough to follow a strange thread. A thread I can’t keep with myself any longer. It’s open-source, backed by science, and shared with you as part of our common quest for scientific progress on human longevity. Newton gave us a word for it. Einstein said it bends spacetime. I am saying gravity shortens lifespan.  Read on, and tell me what you think.

Stay tuned folks ✨

So many people keep asking me about Continue. What is it? What are you up to? Here you go... Continue started as a research effort two years ago, with the belief that if the human body is a system, it should also have its leverage points. The simple levers that, when adjusted, could fundamentally alter how we age and live. Along with a team of initially skeptical researchers, we've been investigating a penny-drop insight about human aging. Something that's been hiding in plain sight for eternity. But more on that in a few weeks. We are at the tail-end of the research on this hypothesis, which if true, could fundamentally change our understanding of biology and aging. But today isn't about that. It's bigger than that. Today, we are expanding Continue Research to include a $25 million fund (entirely personally backed) to support researchers across the world who dare to ask simpler questions than anyone else. Who believe biology might be far simpler than we've made it. For over a decade, I have believed that most of the world’s problems stem from our short human lifespans. Continue Research’s goal is to extend healthy human function long enough that humans stop making short-term decisions. Read more about Continue’s purpose at – continue.com This will be a multi-decadal journey. Our goal here is to become a small catalyst in humanity's journey of conscious evolution. To lead us into the Post-Darwin era. Researchers, please look us up and apply for funding/grants at – continue.com/researchers ——— PS: Continue is not a part of Eternal

Finally upgraded my setup, thanks to @IKEAIndia 🤌🏻 What else should I add? 👀

Another day of digging interesting facts from open-source codebases! 💡 I just discovered how ClickHouse stores Arrays efficiently using values offsets ClickHouse has a reputation for being ridiculously fast at analytical queries. Part of that speed comes from its columnar storage design, but one detail that surprised me is how it stores arrays and nested data structures. Instead of keeping each row’s array as is, ClickHouse explodes arrays into two separate columns: 👉🏻 Values column: a flat sequence of all elements from all arrays. 👉🏻 Offsets column: an array of integers marking where each row’s array ends. 📋 A Naive Approach (Why Not Just Store JSON?) Imagine you have a table with arrays: CREATE TABLE example ( id UInt32, tags Array(String) ) ENGINE = MergeTree(); And you insert some rows: id tags 1 [ "db", "metrics" ] 2 [ "ai", "ml", "genai" ] 3 [ "infra" ] A naive database might: 👉🏻 Store each row’s array as a blob (["db", "metrics"]), bad for compression. 👉🏻 Store each row in a separate subtable, bad for query speed. ✅ The ClickHouse Way: Exploding Arrays ClickHouse stores the same data in this format: tags.values = [ "db", "metrics", "ai", "ml", "genai", "infra" ] tags.offsets = [ 2, 5, 6 ] 👉🏻 tags.values holds all elements in sequence, concatenated across rows. 👉🏻 tags.offsets tells you how many elements belong to each row, by marking the ending index. So how do you reconstruct the original row arrays? ✓ Row 1: take values from index 0 to 1 (offset 2) → [ "db", "metrics" ] ✓ Row 2: take values from index 2 to 4 (offset 5) → [ "ai", "ml", "genai" ] ✓ Row 3: take values from index 5 to 5 (offset 6) → [ "infra" ] That’s it! Two flat columns, but the ability to map back to nested arrays. ✅ Why This Is Brilliant 1. Append-friendly Adding a new row is just appending more values and a new offset. No reshuffling required. 2. Cache-friendly scans Columnar queries (like arrayJoin(tags)) only need to scan the flat tags.values, which is contiguous in memory. 3. Compression wins Flat columns compress better than row-level blobs because similar data (like repeated tags) sit close together. 4. Vectorized execution Queries operate on contiguous memory regions, enabling SIMD acceleration and fast parallel scans. 📋 Attaching the exact source code for reference: github.com/ClickHouse/ClickH… #DatabaseInternals #Clickhouse

So I was going through Prometheus Internals and learned something interesting.💡 When we think about Prometheus, we often imagine it as "the metrics database" scraping, storing, and querying endless time series from our apps. But under the hood, it’s doing something surprisingly clever that makes it much more efficient than you'd expect. ✅ Prometheus assumes that most metric data looks boring and predictable, then exploits that fact ruthlessly. It is inspired by a 2015 paper published by Facebook titled "Gorilla: A Fast, Scalable, In-Memory Time Series Database" (vldb.org/pvldb/vol8/p1816-te…). Here’s the trick: Prometheus doesn’t actually store full timestamps or raw values for every single metric sample. Instead, it uses a smart compression technique based on deltas and XOR encoding. ➡️ The Problem with Naive Storage Let’s say Prometheus scrapes a metric every 15 seconds. A naive approach would store each sample like this: [timestamp: 64 bits] [value: 64 bits] That’s 16 bytes per sample. For millions of time series scraped every few seconds, the storage cost explodes. ➡️ The Delta XOR Magic Prometheus takes advantage of two simple observations about metrics: 👉🏻 Timestamps are Predictable: Scrapes usually happen at fixed intervals (e.g., every 15s). Instead of storing the full timestamp, Prometheus stores the difference (delta) between consecutive timestamps. Example, for timestamps like 1694000000, 1694000015, 1694000030 → Store: 1694000000 (first one raw), 15, 15 👉🏻 Values don’t change drastically Prometheus stores the first value as is, but every subsequent value is XOR’d with the previous one. If values are close (like CPU usage moving from 0.51 to 0.52), the XOR result has lots of leading zeros. Those zeros compress very well. ✅ The Payoff Because of this scheme, Prometheus can often store 2-3 bytes per sample instead of 16. That’s why we can comfortably run Prometheus on a single node with millions of metrics without needing a giant cluster behind it. 📋 References - Prometheus TSDB Format Docs: github.com/prometheus/promet… - Prometheus Source Code (XOR Compression: github.com/prometheus/promet… - Facebook Gorilla Paper: vldb.org/pvldb/vol8/p1816-te…

If I could give one piece of advice to my college self: Don’t ignore Math. Computer Science and Math go hand in hand. Master Probability & Statistics! They’ll open more doors than you can imagine.