This post hits home hard.
A perfect stranger told me that I have lost my mind yesterday. I am grateful.
(receipts:
x.com/polyMatto/status/20550…
x.com/polyMatto/status/20549…
)
I think it all boils down to a discussion around trust and quality.
How can you trust any folder with source code inside?
I thought about this hard. Certainly I am missing some pieces and I am glad that people like you and the gentleman
@YoavCodes call my BS directly or indirectly so that I can learn and grow.
I am going to ask something very stupid.
Isn't a compiler a form of automation that speeds up the dev process? (we used to write machine code manually, I wasn't born yet but I am sure some code scribes were horrified initially).
What about linters, syntax highlighting, language servers? Lower impact tools but certanly they do something in an automated way to speed up the dev process and raise quality.
I see them as tools, black boxes that have one job and have to do it well. But if you have two compilers, how do you discern a GOOD compiler from a BAD compiler?
Trust and quality need to be measured... but HOW?
In a world that is basically embracing the whole "Move fast and break things" mentality what do you do?
My reaction to AI has been:
- Never gonna work, ridiculous
- Good luck fixing AI generated code that blows up on your face at 3am LOL
- Fine this could be useful but overall it's garbage
- Garbage in, toxic waste out
- OK, this really helped me. Small task but I did it 10x faster... shit, am I missing something? I need to investigate
- I can now do things I could have never done before.
- Power is nothing without control.
- Am I a better or worse engineer today? I used to drop in a codebase like a navy seal and do things that looking back I can't believe I did before AI, but now... I can build things in languages I am only acquainted to and I can learn to be proficient in them much faster than I used to (Google Stack Overflow weeks reading/writing code).
- I feel like I am taking crazy pills <- Now I am here.
So yes... I worry too but I want to be optimistic about it.
Stuff WILL blow up on our face.
AI will make it happen faster.
To me that also means that we can learn a -hard- lesson earlier and improve from there.
AI can find exploits that are a decade old. Scary, right? But isn't it ultimately a good thing? We can fix them.
To me a 0Day is less damaging than a latent/undiscovered -2423523421day.
Like this horror story from last March...
x.com/TrungTPhan/status/2032…
AI allows you to rewrite million-loc codebases in a week... is it good or bad? I don't know but I trust the team that built the thing depending on my personal exposure to that project.
I trust the judgement of the engineers that will deploy systems I use and where I input my data.
But is this a new thing or we have always done it?
Closed source software.
If we think rationally, we shouldn't touch it.
How can you trust a black box? Even the very operating systems we use.
You have to have "faith" that they will not do anything dodgy to you... that bugs will be fixed timely, etc.
Do you care which tools they use internally? Should you?
I care about the results but how do I measure them?
I haven't been hacked = good.
Pictures of my feet are online = bad? (hold up, lemme check, nah, still ok)
Now to the practical. What do we do? If I don't trust something, do I switch or I build my own.
Is it different today? Not really.
@YoavCodes used bun in his ElectroBUN project.
And it's basically morphing into ElectroZIG...
Switching runtime is faster like it's faster to rewrite the entire runtime (different scales but same powerup).
Catastrophe looming if you use the RustBun? Depends.
I wouldn't use it for prod stuff like I wouldn't use any unstable library but let's not forget that "unstable/alpha/beta etc" are labels applied by the team that builds the thing, so you are ultimately trusting PEOPLE.
I don't know personally the people that built tools I use but I trust them, you are the perfect example
@mitchellh.
Do I care about how much AI you use in your workflow?
Out of curiosity, yes, but in qualitative terms... not really, I still trust the man trusting his tools. (don't gender-grammar-shame me please. Focus, please.)
Now on automating our way into a disaster.
Yeah that's scary. But maybe the disaster sometimes is inevitable to prevent the next one? There's always gonna be another disaster... but can you recover from it without permanent loss?
We need to build in an antifragile way.
MTBF vs MTTR... we need a mix of both maybe? Even if I'd lean towards MTTR. Easier said than done tho.
Tests bug reports are a proxy for quality I believe.
Do they paint the whole picture? No. That's a bug!
We accept that we don't know what else is broken in our code. We "hope" to find out sooner rather than later, hopefully in an inner dev loop.
If I were
@AnthropicAI I would have kept ZigBun and RustBun, maintained together for a while.
Used them in a round-robin way internally with non-critical tools, maybe even built new tooling / throaway apps that mimick production use cases meant to stress test the areas where the well known class of bugs they are aware of hit the most and of course I would have fuzzyied he hell out of it because we don't know what we don't know.
Too expensive? Really? You wrote the whole thing in a week! You don't have a little cluster that can run 24/7 on this? GTFO.
So no, it's not fine to ship bugs. You wash your dirty clothes at home, you stamp a beta/radioactive/do not touch label on it for a while anyway and you prove that YOU as a team can be trusted.
But again. I know nothing. If the creator of the project TRUSTS his tools (even if he is 99% biased since his employer is the company that builds the tool).
I need to make a call... The Kubernetes cluster in my brain is killing services due to trust dropping below a totally arbitrary made-up threshold. I need to recover quickly, I switch runtime.
Am I better off? Probably...
I am choosing WHO to trust. And of course I might be completely wrong and have bad judgement myself.
But, we'll figure out. We'll learn one way or the other.