In AI, every direction has its time in the spotlight
That can happen due to
> natural progress: other capabilities enable it, or new models make old ideas viable
> external shifts: a new product, paper, project, talk
My experience in the last few years:
β’ pre-training: πππ
β’ model arch: ππππ
β’ multimodal: πππ
β’ small models: πππππ
β’ post-training: πππ
β’ RLHF: πππ
β’ synthetic data: ππππ
β’ inference-time scaling: ππππ
β’ "real" RL: πππππ
β’ code LLMs: ππππππ
β’ tool use and agents: πππππ
β’ alignment, safety, interpretability: ππππ
β’ self-improvement: πππ
β’ open-endedness: ππ
β’ continual learning: ππ
(there is only one of these you should be working on now btw)
In order to maximize impact (your own definition), you should:
> focus on goals and not methods
> balance staying the course with staying ahead of the trend, and
> work on fundamental problems that will continually be relevant