Software Engineer @VictoriaMetrics, building VictoriaLogs

Joined August 2014
267 Photos and videos
Pinned Tweet
6 Nov 2025
I have made thousands of static visualizations for Go, it's time to try something new: youtube.com/watch?v=fwHok9Zh…
12
22
252
21,535
Phuong Le retweeted
I'm starting something new in my blog: one guest every month, walking through the internals of Open Source projects from their own point of view. They are going to take the first post of every month (The first Monday of the month). I'm extremely happy to have already 3 outstanding engineers in the queue. I would recommend you to go and check what they are doing (probably follow them). First up for July is @func25 from VictoriaMetrics. If you've ever fallen down a rabbit hole reading how the sync package works there's a good chance you were reading Phuong. His series on the sync package internals are very good. He's also the author of The Anatomy of Go, a deep dive into how the language actually works under the hood, from data structures and assembly to memory, the GC, and concurrency. Exactly the kind of guest this series is about. And the lineup keeps going: 🔹 @alextrending, will be the August post, he is Principal Engineer at Memed, author of System Programming Essentials with Go, and a relentless community builder. Lately he's been on stages everywhere with talks like "Green Tea GC: The Insight Behind Go's New Garbage Collector," "Tug-of-Code: The battle for efficient iteration in Go," and "Why Go Hides Spin Locks?" His blog (alexrios.me) is a goldmine on Go internals, systems programming, and tech leadership. 🔹 @mdelapenya will be the author of September guest post. He is an open source engineer at Docker and core maintainer of testcontainers-go, the library that's quietly become a the facto standard on how to write intengration tests in Go. Also, you can check out his blog, where is already talking about some internals of testcontainers. The goal is simple: make the scary parts of Open Source internals approachable, for interns, for seniors, for anyone who's ever typed a function call and wondered what really happens next.
8
63
2,541
The VictoriaMetrics team is growing, and we're excited to welcome Jesús. Can't wait to see the deep technical and systems insights you'll share along the way.
Today is my first day at @VictoriaMetrics ! 🎉 Back to working on Open Source, on exactly the kind of deep systems project I've wanted for ages, performance, and understanding hardware, OS fundamentals. Lots to learn, and that's the best part. I'll be writing about what I learn as I go.
1
2
48
4,321
Chapter 8 of The Anatomy of Go is now available. It discusses channels, timers, concurrency, goroutine creation and lifecycle, runtime startup, preemption, scheduling, I/O handling, the system monitor... and a bunch of other things. The book is now complete at 873 pages, and early access will be closing soon after the editing phase, (which may add a few more pages). As usual, here is an excerpt:
7
47
450
20,455
Phuong Le retweeted
If your are interested in why VictoriaLogs is so efficient and fast, then take a look into these slides about VictoriaLogs architecture internals - docs.google.com/presentation…
3
33
205
15,107
Enums are good. I wish Go supported "real" enums. But union types are not, at least IMO. At the very least, I do not want union types in a TypeScript-ish form: func F(x int | string | []byte | io.Reader) { ... } This function accepts many unrelated shapes of input. Inside the function, you must figure out which one you got. It's never good to be too convenient. It loosens the language in a way that makes APIs harder to reason about. If Go somehow were to add union types, a more Go-ish version would be: type ParseResult[T] union { Ok(T) Err(error) } type Shape union { Circle(radius float64) Rect(width, height float64) } That is closer to an enum with payloads, or a sum type or tagged union or Rust-style enums. It keeps the set of cases closed and explicit. Go does not need "any value can be A or B or C" union types. Go would benefit from "this domain value has one of these known variants" sum types. You cannot pass values like in TypeScript: f(123) f("hello") You need to pass explicit variants instead, assuming Input is a tagged union: f(Input.Number(123)) f(Input.Text("hello")) More verbose but safer and stricter. See more at github.com/golang/go/issues/… github.com/golang/go/issues/…
hot take: Go would be the greatest programming language on earth if it had enums and union types. they simplified their way into having to do really weird stuff just to represent reality. you should still probably use it though.
8
5
111
17,453
Phuong Le retweeted
"We weren't gambling on a new technology — we were promoting one we'd already been operating in production against our own workload, and that consistently outperformed Loki in our side-by-side testing on the kinds of queries we and our users actually run."
The dedicated logger service inside every Zerops project is now using VictoriaLogs. - 100x more log retention - 3x faster queries - VictoriaLogs UI embedded into Zerops - Already rolled out to every existing project Full story below.
1
10
1,337
Phuong Le retweeted
Something is seriously wrong with how people contribute to open source these days. People submit AI-generated code they clearly don't understand. When reviewers comment, they feed it back to the AI and submit again. The loop never ends. Low-effort PR requires extremely high effort to review. That math is never fair. You think this is how you build a reputation in open source? No, no, and no. You're just burning out maintainers. --- Use AI to understand the codebase instead. Ask it to explain things. Build your own judgment. It's ok to use AI as a tool for coding, but make sure you filter out any AI slop before submitting. You gain the skills. You gain the reputation. The project actually benefits. Is it that hard?
3
2
17
1,316
Code is becoming cheaper than ever, but maintainability is becoming more expensive than ever.
6
8
119
5,052
Go is not bad at concurrency. A more factual take is Go is excellent at making concurrency cheap, explicit, and approachable (and approachable, and approachable), especially for common backend patterns. But Go gives you relatively low-level primitives, a lot of correctness around cancellation, task lifetimes, cleanup, error propagation, and backpressure is left to us, the programmer. So the fair comparison is not "Go vs JVM, who wins?" It is: Go optimizes for simple pragmatic concurrency, the modern JVM ecosystem has stronger tools for structured and resource-safe concurrency. Which one is better depends on how complex the concurrency problem is. This is just a "depends on the problem" discussion.
Generally developers think of Go as being great for concurrency. Its not. JVM approaches are vastly superior. And even some of the best in the whole industry when you include virtual threads, structured concurrency & Effects.
11
43
543
76,712
Phuong Le retweeted
Who's the biggest Victoria-stack bull in my feed that wants to say why they love it? I'm yolo porting from Prom/Loki etc in a project because I saw some nice numbers in the past and want to try it out but want to hear some people that are excited by it geuinely
6
1
18
7,134
Go is simple, so I ended up writing an 865-page book about how it works internally, just to see how it maintains that simplicity 😇
48
162
2,199
91,452
Phuong Le retweeted
New article: The Go System Monitor How sysmon, a thread with no P, keeps the runtime fair. It preempts goroutines, retakes Ps from syscalls, forces GC every 2 minutes, and ensure network poller keeps ticking. The Go's watchdog internals-for-interns.com/po… #golang #go #runtime
7
53
8,277
Chapter 8 of The Anatomy of Go is ready for review. It goes into concurrency, the Go scheduler, threads, the system monitor thread, goroutine preemption, what happens at startup, how channels work, how the select statement works, and how timers work behind the scenes, a lot and a lot, with visual explanations. This is the final chapter, it is near the end of the journey.
7
27
447
16,120
I saw the "procedural vs OOP" debate today, and it's an overrated discussion, especially for beginners The real issue is whether your code stays understandable after the project stops being small. If you need 10 files, 5 abstractions, and a pile of patterns just to move data from a request to a database, that is not good design. That is "decoration". - A lot of people learn OOP as if it is the default serious way to program. It is surely not. It's just one way to organize code. Sometimes it helps, often it creates useless structure, especially when people copy Java-style habits into places where they do not belong. - Same, "just write functions" style can also become sloppy if there is no separation between business logic, input handling, storage, and side effects. So the answer is not "everything should be OOP" and not "everything should be procedural." The answer is that structure should serve clarity. If you are building backend code, especially in a language that rewards simplicity like Go, the best way is - plain code first, abstraction later. - Keep handlers thin. - Keep logic in clear units. - Keep database code separate. - Add methods and interfaces only when they solve an actual problem like testing, replacement, or repeated behavior. - Do not build a system around imaginary future needs. - Most bad code comes from trying to look scalable before earning it. So we should stop worrying about paradigm branding and start worrying about readability, boundaries, and change cost. "OOP vs procedural" is usually a fake hard question. The useful question is this: when this code gets bigger, will it still make sense without a lecture? If yes then you are probably doing it right.
3
4
55
4,484
Phuong Le retweeted
Last year I got introduced to VictoriaMetrics through an article "Graceful Shutdown in Go". It was very well written by @func25. High quality articles instil developer confidence in a company's product. Its the same case with @ScyllaDB
VictoriaMetrics, VictoriaLogs, VictoriaTraces Not because I'm a dev and recommend them all the time, but because they are the simplest solutions on the market right now while still having top-tier performance with the least learning curve and minimal configuration. VictoriaMetrics has proven its position, but VictoriaLogs is a newer solution entering the market. It is being adopted from homelabs to large-scale companies to significantly drop costs (I can't reveal their names yet). Try it to believe it. If you have any problems, DM me directly or create an issue at github.com/VictoriaMetrics/V…
4
51
5,269
Phuong Le retweeted
Mar 28
I can second this statement, we're using the stack in petabyte scale and improving our throughput while reducing infra cost and alert fatigue :D currently we're POC-ing VictoriaTraces because our Tempo stack is quite slow
VictoriaMetrics, VictoriaLogs, VictoriaTraces Not because I'm a dev and recommend them all the time, but because they are the simplest solutions on the market right now while still having top-tier performance with the least learning curve and minimal configuration. VictoriaMetrics has proven its position, but VictoriaLogs is a newer solution entering the market. It is being adopted from homelabs to large-scale companies to significantly drop costs (I can't reveal their names yet). Try it to believe it. If you have any problems, DM me directly or create an issue at github.com/VictoriaMetrics/V…
2
1
13
2,443
VictoriaMetrics, VictoriaLogs, VictoriaTraces Not because I'm a dev and recommend them all the time, but because they are the simplest solutions on the market right now while still having top-tier performance with the least learning curve and minimal configuration. VictoriaMetrics has proven its position, but VictoriaLogs is a newer solution entering the market. It is being adopted from homelabs to large-scale companies to significantly drop costs (I can't reveal their names yet). Try it to believe it. If you have any problems, DM me directly or create an issue at github.com/VictoriaMetrics/V…
We tried 4 observability platforms in 2 years. Datadog: best UX, eye-watering cost. New Relic: cheaper, clunky dashboards. Grafana stack: free, but you own the ops. VictoriaMetrics: fast, lean, no hand-holding. Ended up on Grafana VictoriaMetrics. Half the cost, full control.
8
9
193
31,771
A lot of Go devs get taught a fake rule: never use panic. The advice is simple but it is also lazy. Panic is just a sharp tool that needs to be used correctly. The real advice is do not use panic for normal failure, use it when the program has reached a state that should not exist and cannot be handled in a sane way. If your code can recover, return an error. If your code is - broken, - your assumptions are false, - startup cannot continue, Basically, this can't be happen, if this happen, my data is corrupted, my application is not in correct state, etc. Then failing hard is cleaner than pretending everything is fine. Bad error handling is often worse than no error handling. People wrap impossible states in polite-looking error returns and push the mess upward, so now every caller has to act like a broken invariant is just another routine case. That makes code noisier and less honest. So a well-placed panic can say something important: this is not business logic, this is a bug, a bad configuration, or a state the program was never designed to survive. We do use panic in our codebase to catch corrupted state early: github.com/VictoriaMetrics/V…
11
17
167
15,146
People do not hate ORM because putting data into structs is bad. They hate ORM because it sells convenience first and cost later. It is fast, clean, and simple at the start. But it hides how the database really works over time. When too much is hidden, devs stop thinking carefully about what they are doing. Then problems start to show up, e.g., - unexpected queries, - strange defaults, - bad updates, - code where no one really knows what is happening until it breaks in production. If you do not understand SQL, an ORM will not fix that. It only delays the problem. If you do understand SQL, using an ORM can feel unnecessary and slower than writing queries directly. That is why many devs prefer tools that stay close to SQL (github.com/sqlc-dev/sqlc). It is easier to understand performance, correctness, and side effects when you see the real query.
11
20
288
20,161
VictoriaMetrics becomes a common choice in AI agent systems: - OpenAI Codex uses VictoriaMetrics, VictoriaLogs, and VictoriaTraces internally for observability: openai.com/index/harness-eng… - now we’re seeing other multi-agent systems adopt them to monitor agent behavior itself: github.com/steveyegge/gastow… > Gas Town emits all agent operations as structured logs and metrics to any OTLP-compatible backend (VictoriaMetrics/VictoriaLogs by default) Simple, lightweight, no config, high performance, plug-and-play. Too good to be true
5
13
146
10,538