Co-Founder / CTO @spiraldb | Creator of @vortexdotdev
Joined February 2021
- Tweets 45
- Following 357
- Followers 203
- Likes 119
7 Photos and videos
May 14
SmithDB is the perfect example of how far performance can be pushed by having full control over the storage layer. The DataFusion @vortexdotdev stack seems to be emerging as THE way to build next generation databases.
#ParquetIsForFloors
May 13
We built SmithDB: the database purpose built for agent observability workloads that now powers many parts of LangSmith.
Agent observability presents a challenging data problem. Agent traces can contain tens of thousands of intermediate spans and large, unbounded payloads. These characteristics are a direct result of agents running for longer time horizons and LLM context window sizes growing.
Traditional data infrastructure was not built to handle the complexities associated with storing and querying this data.
SmithDB brings LangSmith up to 12x performance improvements across access patterns most important for agent observability. I’ve been working on SmithDB directly with an amazing team over the past few months, and I’m incredibly proud of the results we’re seeing.
I wrote a bit more about the story and engineering challenges behind SmithDB in this blog.
Additionally, if you’re a systems engineer interested in building the future of agent observability, please reach out!
2
2
46
5,501
Nicholas Gates retweeted
May 4
Love seeing Naomi Osaka honor the CLRS Algorithms textbook at this year's Met Gala
109
1,735
18,119
614,317
Feb 26
Vortex is BOTH 38% smaller and has 10–25x faster scans than Parquet ZStd for TPC-H SF10
We implemented BtrBlocks-style cascading compression in Vortex that recursively tries codecs like ALP, FSST, and bit-packing, letting the data pick the best encoding
spiraldb.com/post/cascading-…
7
40
2,805
30 Apr 2025
🚀
Checkout our latest post by @andreweduffy about bridging our Rust code to Java to accelerate Apache Iceberg queries 🏎️
spiraldb.com/post/vortex-on-…
1
474
16 Apr 2025
Slightly late to the party here, but could the billion dollar #ErasTour really not figure out how to center a digital clock…
5
411
31 Jan 2025
I bet there aren't any other file formats that come with a slick terminal explorer 😎
1
21
1,468
25 Oct 2024
And so it begins… a shame to see Iceberg caving in to DataBricks on technical design decisions whose only benefit is better compatibility with Delta.
lists.apache.org/thread/wyon…
12 Jun 2024
It will be interesting to see what the Apache Iceberg community thinks about merging its format with Delta Lake instead of innovating independently.
5
465
24 Oct 2024
They said no one would get my @olivianj reference
New blog post from CTO @ngates_ about the important decision to separate logical and physical types in the Vortex file format
blog.spiraldb.com/logical-vs…
5
425
13 Sep 2024
Fsst & Furious
In our next post, @andreweduffy writes about state-of-the-art string compression with FSST and #Rust 🦀
blog.spiraldb.com/compressin…
5
284
7 Jul 2024
After a decade of fighting spell checkers at work, I have chosen this Fourth of July weekend to finally surrender 🇺🇸
1
2
218
27 Jun 2024
It's kind of ridiculous that OIDC authentication is not more widespread. PyPi leading the way. If I have to copy one more god token...
4
194
27 Jun 2024
Does no one except @auth0 support OAuth 2.0 Device Auth Flow?
I've checked @stytchauth, @WorkOS, @FronteggForSaaS, @HeyKinde, @supabase
Thinking of creating a CloudFlare worker to MITM this...
2
3
254
Check out our inaugural blog post by @ngates_ describing the FastLanes compression layout and how you too can decode >100 billion integers per second!
blog.spiraldb.com/life-in-th…
1
8
45
6,678
13 Jun 2024
🤣
12 Jun 2024
It will be interesting to see what the Apache Iceberg community thinks about merging its format with Delta Lake instead of innovating independently.
2
261
1 Dec 2023
When you need two USB-C ports on your MacBook and don’t want to lose your YubiKey…
2
4
368
23 Apr 2023
Quick! Grab your real estate!
23 Apr 2023
Today, we are rolling out the first step in our plan to build financial support and long-term sustainability, while simultaneously giving our users one of our most requested features: organization accounts.
blog.pypi.org/posts/2023-04-…
1
1
356
22 Apr 2023
This is great! More tools should offer support for OIDC tokens and stop asking me to generate API tokens with no expiry 👀
20 Apr 2023
Starting today, PyPI package maintainers can adopt a new, more secure publishing method that does not require long-lived passwords or API tokens to be shared with external systems.
blog.pypi.org/posts/2023-04-…
1
2
276