When is Serverless Inference Cheaper than Your Self Hosted GPU? I Benchmarked gpt-oss-120b on Both
If you run LLM inference in production, you eventually will ask yourself, should you rent a GPU and...
Yash Sharma7 min read
Author profile
Yash Sharma
Engineering DigitalOcean | Maintainer Meshery (CNCF project) | Open Source
Articles by @yashsharam_f
Browse the latest writing surfaced through DevArt.
Yash Sharma7 min read
#ai#kubernetes
We Got 2x LLM Inference Speed With Three Kubernetes Settings
Serving LLMs is not easy, especially when it comes to scalability, we have to optimise the infra to...
Yash Sharma6 min readAdvertisement