Today, I tried a more complex test case that required multi-step reasoning. The newly released #Gemini 2.0 Flash Thinking model (01-21 version) was disappointing and significantly inferior to the o1-mini. The critical issue was that it couldn't even generate XML/HTML tags correctly - <foo> became ```foo>. Alas. Meanwhile, Claude 3.5 sonnet, through prompt engineering, can achieve results close to o1-mini's performance. After the holiday, I plan to try Deepseek R1 with a few H100/H200 GPUs. #LLM

Recently, I accidentally noticed that Google Gemini's large language model API quietly changed its JSON format for returning citation meta error messages, so I had to modify my own API client code accordingly. Such backward-incompatible API changes are pretty annoying. Of course, Google's citation error checking for references and copyrighted text is exceptionally frustrating in itself. I haven't encountered such issues with other large language models. #Gemini #LLM #API

The cost-effectiveness of the Gemini 1.5 Flash model API is quite good, especially in batch processing mode (although its batch API is difficult to use). However, Google's cheapest Gemini 1.5 Flash 8b model is too weak—so weak that it sometimes doesn't follow instructions, causing many of our test cases to fail, and it's not easy to work around these issues. In my daily AI application development, I really enjoy using the 1.5 Flash model for testing and debugging—it's fast and affordable. The experimental Gemini 2.0 Flash model shows significantly better quality than 1.5 Flash. I hope the pricing won't increase after it becomes generally available (GA). #LLM

I recently noticed that the concurrency of large model batch processing tasks in each Google Cloud region has dropped from 4 to just 1. This is true for all Asia-Pacific and North American regions at least. The change happened just a few days ago. Perhaps they're preparing computing resources for the GA release of the Gemini 2.0 Flash new model? The official documentation states that the regional batch processing concurrency is 3. I'm looking forward to Gemini 2.0 GA. Currently, the API rate limits for the experimental model are too strict, and the computing power is too low - the 1500 RPD (requests per day) isn't enough to run our test suite. Running the test suite once requires at least tens of thousands of LLM requests. #Gemini #LLM

Our OpenResty XRay dynamic tracing product's Java analyzers finally support Azul's Zulu JDK! This includes Java-land CPU/off-CPU/diskIO flame graphs, latency analysis and smart packet capture, dynamic probes for Java method entry points, and more. We have a cryptocurrency exchange client who needs this explicitly. OpenJDK, Oracle JDK, and Zulu JDK—all three JVM variants are now fully supported. Depending on our enterprise customers' actual needs, we'll consider supporting other variants like IBM JDK and Microsoft's JDK later. #DynamicTracing #Java #JVM #ZuluJDK #Azul #eBPF

Our OpenResty XRay, a dynamic-tracing product, transforms black-box production systems into white-box ones. Many issues can be observed and captured before they escalate into harmful problems. This continuous insight and control over production systems is what's most crucial. Offline simulated traffic and testing methods often have limited coverage. No matter how much effort is invested in offline simulation, gaps will always exist. Additionally, many of our customers prefer using OpenResty XRay for white-box analysis even in offline environments; regardless of the environment, white-box is much more convenient than black-box. You can't keep adding monitoring metrics and debug logs every time, can you? ;) Our ultimate goal is to help customers quickly understand their legacy systems (or "sh*t mountain systems") and improve them efficiently at low cost rather than just being a tool for solving production issues. "Turning sh*t into gold" - that's what makes this truly interesting and valuable.

Many users ask me about the implementation principles and technology behind @OpenResty XRay, as it looks magical. • For those interested, you can refer to my previous article about dynamic tracing: blog.openresty.com/en/dynami… • Here's another series of articles I wrote introducing the ylang language: blog.openresty.com/en/ylang-… • And here's an article I wrote about the y language: blog.openresty.com/en/cpp-tr… Our recent advanced dynamic tracing capabilities for Java are implemented using our C -compatible y language. Interestingly, our team has been using OpenResty XRay to analyze OpenResty XRay itself. However, this capability is disabled for our customers to prevent misuse. #DynamicTracing #DomainSpecificLanguages #eBPF

OpenResty XRay now supports setting dynamic probes at any #Java function entry point! This feature has wide applications - for example, we've implemented real-time latency statistics for requests within Tomcat servers based on this feature, and integrated it with our online intelligent packet capture analyzer. It doesn't depend on any configuration of the target process or log files. It supports both interpreted and JIT-compiled Java methods. It's still based on 100% non-intrusive dynamic tracing technology, with negligible performance overhead. It doesn't rely on bytecode injection or machine code injection, doesn't depend on JVM Agent mechanism, and is completely read-only for the JVM target process - the target process doesn't even need to restart. It supports both the latest Linux kernels and old Red Hat 7's 3.10 kernel. Since this feature is very new, it currently only supports OpenJDK, but will soon support Oracle JDK. In the near future, it will definitely support more variants like Zulu JDK, IBM JDK, Azul JDK, and others. Support for dynamic probes at Java function exit points will be added later. #JVM #java #DynamicTracing #eBPF

Asahi Fedora Linux runs very well on my M2 Ultra Mac Studio. All devices including USB-C, HDMI, 10G network card, etc., work without any issues. It's very stable. In comparison, Linux support for Qualcomm Snapdragon X Elite devices is far behind. Thanks to the great work from the Asahi team. Previously, running Linux VMs through qemu HVF on macOS had too many problems and poor performance. Going forward, our team will rely on Asahi Linux for development work targeting the Aarch64/ARM64 architecture.

Following OpenJDK, our OpenResty XRay can now deeply analyze #OracleJDK! We're still using our 100% non-intrusive dynamic tracing technology. The analysis covers everything from the C VM level to the #Java source code level without any blind spots. Next up will be other JDK variants like Zulu JDK and IBM JDK 😀 #JVM #java #DynamicTracing #eBPF

Some people speculate that our OpenResty XRay uses the ELF file's build ID to look up existing DWARF information in a database, which made me laugh so hard. We're actually targeting binaries that customers have modified and compiled without the -g flag or binary software packages downloaded from third parties without debug symbols. This is the most challenging and general problem to solve. For any existing binary programs and their debug symbols found online, our crawler system has already indexed them in our Pkg DB, which has accumulated hundreds of terabytes of data and is still growing daily. Why would we need such advanced machine learning technology for those cases?

Someone on the Internet asked: "What if you randomly grab a programmer from the street, and they can solve problems using your OpenResty XRay tools?" They seem to be concerned that only experts could use our products. My response is that their worry is unnecessary, and our technical experts will directly help clients solve problems - that's our guarantee. It doesn't matter if clients themselves don't understand technology or know how to use OpenResty XRay preciselyproducts. This is exactly why some non-technical business owners like our services. As the saying goes, "You make a wish, we make it happen!"

The open-source vector database Qdrant actually has a nasty bug that occasionally causes unexpected data loss. The official team finally confirmed it was a bug in the database index implementation. Alas. github.com/qdrant/qdrant/iss… There are so many bugs in the software world... a database that loses data is the scariest thing. Fortunately, it got fixed in the latest version.

One of my dreams for many years has been to turn all tech stacks and applications in all programming languages from black boxes into white boxes, eliminating mysterious issues in production environments. This is precisely the mission of our OpenResty XRay. Previously, we encountered a case where a hardware bug was discovered using OpenResty XRay on a customer's production Xeon CPU. Intel engineers initially didn't acknowledge it was a hardware issue and insisted on patching the Linux kernel to confirm the problem. Dramatically, OpenResty XRay then discovered bugs in their kernel patch. Finally, it was confirmed to be a CPU hardware issue. We still need to thank the Intel engineers for helping confirm the problem. There are so many bugs in the world, so OpenResty XRay aims to be the best bug hunter. Going further, hopefully, one day, it can even automatically apply binary patches to VMs or machine code in running processes to fix bugs or security vulnerabilities. This belongs to live patching. Everything can be live-patched from kernel to system software and up to business code without even restarting processes. We have already made some exciting experiments in this direction, and it's very promising. Advanced technology looks like magic 😁

Years ago, while working at Alibaba, I wrote online services using Erlang/OTP. However, for the same business complexity, the performance of #Erlang services was far behind my OpenResty [Grin]. Nevertheless, Erlang is still a comfortable, functional programming language to use, unlike Haskell, which requires an entirely different programming mindset. Now OpenResty XRay already supports dynamic tracing for Erlang, and in the future, we might add support for #Haskell applications in OpenResty XRay as well.

All the new shiny Erlang dynamic tracing tools are written in our Ylang universal debug language as always: blog.openresty.com/en/ylang-…

Some users asked whether our @OpenResty XRay dynamic tracing technology relies on JVM frame pointer preservation. Of course, we don't have such a requirement—we've always pursued a strict zero-modification, zero-cooperation approach. Preserving frame pointers has a significant performance impact. I know some third-party open-source solutions and even commercial solutions require frame pointer preservation, which is not suitable for production, obviously. #DynamicTracing #JVM