LLM Inference workloads are becoming monolithic, heavy, & hard to scale. That's where platform engineers can embrace
@_llm_d_ , a new open-source effort thatโs starting to tackle a problem weโre seeing more and more in prod ML stacks!!! and it is now a CNCF Sandbox project ๐