Some long but hopefully helpful thoughts about self learning these things:
The list is actually in reverse chronological order imo if you are starting from 0.
"familiarity with the AI research landscape" <> "high taste".
I think it starts with reading. Read papers, read discussions on here, ask (dumb) questions. Have a feel for what people are doing, how they think about problems, how they think about the solutions to these problems. Also, get out of tutorial hell/from-scratch hell asap.
Step 2: Congrats, you should now have a somewhat "unique" but still possibly wrong perspective towards AI research. Evals are a good next step.
Start thinking about what you think are gaps in existing literature. Are people evaluating a certain capability wrongly? Are people not evaluating a certain capability at all? Why? Notice that everyone is using the same set of evals because the interesting ones are all proprietary. What do you think is missing?
Step 3: Hopefully, you've come up with an eval that exposes unique and useful failure modes. Now you've got to come up with the solution. Try prompting, ICL, dspy? Only when all of these fail, then try finetuning.
A common misconception is that writing the finetuning code is difficult. I disagree. For most usecases, just use Axolotl, LlamaFactory etc. The highest barrier to entry is the data.
What approach works best for your particular task? Big model distillation? Or are you working with synthetic tasks? What are the latest papers doing? For example, specialised experts seem to be new thing from the Chinese labs. If you are doing RL, what should your reward be like? What are some of the interesting techniques you've come across? Can they be applied to your problem?
Iteration speed is important for learning here. Keep trying out things and see what works and what doesn't. Iteration is so important: read new paper -> does it apply to my task -> implement -> try -> repeat
Miscellaneous:
> Building Agents.
Since most people reading this probably come from a more traditional SWE background, this is the simplest step. It is largely normal SWE, but with an understanding of the current 'meta' and implementing these techniques. As an example: how many tools? Grep or semantic search?
> OSS contribution
Many more accomplished people have spoken about this so i'll just re-emphasize, don't optimize for commits for the sake of commits. Contribute because you actually use a certain project and encountered a certain bug or want a certain feature yourself.
Also, do not 100% vibe code a PR. Your reputation is more important than the contributor status.
> Write and speak very well.
Honestly, im not the right person to ask lmao. All im gonna say is that if you spend too much time on this site, your ngmi in this regard :)
And lastly, "that dawg in them". I interpret this as agency. Everyone's building RL environments nowadays and its actually a very good analogy for learning. You want to build the best possible environment for yourself to improve as much as possible. This means never stop training (learning), keep doing things (rolling out) and verifying (asking questions/doing experiments). You are the agent in this analogy.
Genuine question: where does one even go to learn a lot of this stuff? I doubt it is in school. Do people find resources online and self study? Is this just a sign of the sector maturing and people are expected to learn these skills at other jobs?
I’ve learned a decent amount of this, but my statistics PhD and a genuine interest carried a lot of that, and it’s entirely theoretical as I have had no industry experience