https://gen-ai.fyi/tags/ttft/ Recent content in Ttft on Gen-AI Hugo -- gohugo.io en Sun, 30 Aug 2026 00:00:00 +0000 https://gen-ai.fyi/one-layer-down/streaming-vllm-ttft-tpot-colab/ Sun, 30 Aug 2026 00:00:00 +0000 https://gen-ai.fyi/one-layer-down/streaming-vllm-ttft-tpot-colab/ In the first One Layer Down run, I loaded TinyLlama with vLLM on a Colab T4 and generated 100 tokens. That was useful because it proved the basic path: GPU, CUDA, PyTorch, vLLM, model weights, tokenizer, and one successful generation. But it was not the latency measurement I actually needed. When an application calls a model, it can wait for the full answer and show it all at once, or it can stream partial output as the model generates.