Product @Cloudflare working on R2 Data Catalog, Pipelines, and R2 SQL 🧊. Databases, streaming, distributed systems, tech, games, & 🎸

Joined January 2010
213 Photos and videos
Hugest of opportunities. I'm having the best time here - if you're curious about what it's like to work here, shoot me a DM and don't sleep on this one.
i'm hiring a pm for @cloudflare r2 if you know anyone who would be a good fit lmk
1
6
550
Easily my favorite social/communication tool of all time. And it just keeps getting better!
the biggest voice / video chat app you know runs on @cloudflare compute discord.com/blog/how-we-move…
165

ALT Fruit Loops And Milk GIF

27
Marc ⛅️ retweeted
Been building Traks(Web Analytics Platform) on @Cloudflare pipelines r2 sql and honestly. This combo is stupid good. ingest firehoses into r2 as parquet, query it straight with sql. no warehouse, no etl glue, no ops. The Serverless data stack is finally here. Using it for my personal projects as i'm still digging into the exact r2 sql limits. concurrency, scan size, query duration and other limitations.
Cloudflare continues to build up its zero-egress-charges data warehouse capabilities with R2 Pipelines Ingest data into an Data Catalog-enabled R2 bucket, using Pipelines or other mechanisms, then run queries over it using R2 SQL or your favourite engines like Snowflake. Pipelines can ingest, transform, and load streaming data into Apache Iceberg or Parquet in R2. Both got new features on Monday: You can now ingest logs directly into Pipelines from your Cloudflare Workers by using Pipelines as a Logpush destination. Logs can be noisy, so using Pipelines, you can trim down to the fields you need before the data is stored in R2 in Parquet files or Apache Iceberg tables. R2 SQL now supports functions for easily querying JSON data, alongside improving readability of EXPLAIN queries by allowing them to be formatted as JSON. Lastly, unpartitioned Iceberg tables can now be queried directly - this should only be used for smaller tables, as partitions will significantly help performance at larger scale.
2
1
4
322
Hey @CloudflareDev users! What are you using for your SaaS/App telemetry and analytics? I see countless posts between here, Reddit, etc with you awesome folks basically using "the package" of Workers, KV, D1, R2, etc - but what are you doing with all the telemetry you generate?
4
8
1,907
fun fact, this is actually the first thing I got to build and ship at Cloudflare. It's something special to work for a place that's setup the infra and tooling to let anyone roll up their sleeves and get stuff that needs to get done, done. So easy, a PM could do it™
Last week we shipped the R2 Data Catalog UI so now users have a single spot to create catalogs, see metrics, schema evolution diffs, metadata, and manage compaction/snapshot expiration settings from the Dash. Made possible with a new API route that gives you the metadata from your Iceberg table snapshots without having to go through a client like Spark. Makes it much easier for your agents to get all the good info about your data without having to go through extra steps. GET "api.cloudflare.com/client/v4…{account_id}/r2-catalog/{bucket_name}/namespaces/{namespace}/tables/{table_name}" ... Next stop: How about a way to query your data directly from the Dash?
14
1,392
Last week we shipped the R2 Data Catalog UI so now users have a single spot to create catalogs, see metrics, schema evolution diffs, metadata, and manage compaction/snapshot expiration settings from the Dash. Made possible with a new API route that gives you the metadata from your Iceberg table snapshots without having to go through a client like Spark. Makes it much easier for your agents to get all the good info about your data without having to go through extra steps. GET "api.cloudflare.com/client/v4…{account_id}/r2-catalog/{bucket_name}/namespaces/{namespace}/tables/{table_name}" ... Next stop: How about a way to query your data directly from the Dash?
2
3
38
8,996
I used to (and maybe still do in some cases) feel the same way until I started using and building serverless systems that aren't economically and operationally beholden to the "traditional" cloud middle person.
serverless was a mistake
2
142
Are you?
1
1
74
opencode stats

ALT Puppet Awkward GIF

33
Marc ⛅️ retweeted
hey all! i’m a pm intern @Cloudflare this summer working on alerts & notifications, including webhooks & emails. got any feature suggestions or feedback? let me know!
9
7
43
11,537
Anyone else find it amusing that AI "skills" are basically just Cliff notes?
52
"Hey Marc, we saw a user try an R2 SQL function we don't have exposed yet, we're going to expose it to everyone in the next release" - I love this small but mighty team so much. Needless to say, a really distinct R2 SQL changelog is going to union us all together next week.
8
240
Happy Friday! JOINs, subqueries, and multi-table queries have *ahem* joined @CloudflareDev R2 SQL! I can't overstate how impressive the work @jeromeschneider and Sergio Estevez did to make JOINs work on Cloudflare infrastructure. Check it 👉 developers.cloudflare.com/ch… And let us know what you think! We already have improvements in the works to make these run even faster 🚀
3
8
59
4,410
Big R2 SQL changelog joining us very soon 😎
1
1
22
1,528
“Let a thousand flowers bloom” was my high school band’s name. It’s also what I tell Claude before every coding session.
agent harnesses are like web frameworks let a thousand flowers bloom, no one-size-fits-all, right tool for the job. there are and will be many great ones what agent harnesses have in common — they all benefit from using the Agents SDK to run in Durable Objects, the runtime designed for agents npm i agents agents.cloudflare.com
185
Marc ⛅️ retweeted
agent harnesses are like web frameworks let a thousand flowers bloom, no one-size-fits-all, right tool for the job. there are and will be many great ones what agent harnesses have in common — they all benefit from using the Agents SDK to run in Durable Objects, the runtime designed for agents npm i agents agents.cloudflare.com
Introducing Flue — The First Agent Harness Framework Flue is a TypeScript framework for building the next generation of agents, designed around a built-in agent harness. Flue is like Claude Code, but 100% headless and programmable. There's no baked in assumption like requiring a human operator to function. No TUI. No GUI. Just TypeScript. But using Flue feels like using Claude Code. The agents you build act autonomously to solve problems and complete tasks. They require very little code to run. Most of the "logic" lives in Markdown: skills and context and AGENTS.md. Flue is like Astro or Next.js for agents (not surprising, given my background 🙃). It's not another AI SDK. It's a proper runtime-agnostic framework. Write once, build, and deploy your agents anywhere (Node.js, Cloudflare, GitHub Actions, GitLab CI/CD, etc). We originally built Flue to power AI workflows inside of the Astro GitHub repo. But then @_bgiori got his hands on it, and we realized that every agent needs a framework like Flue, not just us. Check it out! It's early, but I'm curious to hear what people think. Are agents ready for their library -> framework moment?
5
18
281
54,874
Hold on…let me cook
Music-2.6 from @MiniMax_AI is free on Cloudflare this week! Generate full-length songs or instrumentals from a text prompt, with optional auto-generated lyrics. Fast global inference on Cloudflare’s network. Perfect for building music apps on Cloudflare Workers. Try it free before the promo ends: developers.cloudflare.com/ai…
5
203