Vishnu's engineering notebook

Infrastructure. Local AI. Learning by doing.

I’m Vishnu, a platform engineer in Singapore. These are my notes from building and breaking things: Kubernetes clusters, GPU nodes, and models running on my own hardware. Most posts include the exact commands I ran. More about me →

Read the latest Search /
Singapore · UTC+8
Jump to

$ ls -t ./notes

Latest notes.

All notes
Pixel art of a floating control plane linked to two server racks, the right rack with four glowing GPU cards

vCluster on Bare Metal: A GPU Tenant from One YAML File in Six Minutes

A diagram-first walkthrough of vCluster Platform private nodes and auto nodes on simulated bare metal: one request, Metal3 powers on two servers, writes Ubuntu, and a tenant cluster schedules pods onto a GPU node.

Pixel art of a control tower sending a request beam to a row of dark servers, one of them powering on and lighting up cyan

vCluster Private Nodes and Auto Nodes: How a Tenant Gets Its Own Servers

How vCluster Platform turns one tenant request into real machines: node providers, node types, NodeClaims, and the Metal3 and Ironic flow that writes an OS to a bare-metal server and joins it.

Pixel art of one large server rack holding several small glowing Kubernetes control towers, each wired to its own group of worker blocks

vCluster: Virtualise Clusters, Not Machines

vCluster gives every team or customer its own Kubernetes API, running as a pod in a shared cluster. Here is how the control plane, the syncer and the three node models fit together.

Pixel art of a local SSD transferring model tiles through a short connection into a GPU server inside a rack

Faster vLLM Cold Starts on Kubernetes: Cache the Model, Not the Download

A simple, diagram-first guide to making a new vLLM Pod ready in seconds instead of minutes: where cold-start time goes, how to cache weights on a volume, how to pre-pull the image, and which storage to pick.

Pixel art of three llama portraits rendered at progressively coarser precision beside a compact GPU

LLM Quantization, Simply: FP16, 8-bit, 4-bit, and What Q4_K_M Means

A simple, diagram-first guide to LLM quantization: what rounding a weight actually does, how much memory each bit width saves, how to read names like Q4_K_M, AWQ, and FP8, and a toy script that shows where quality falls off.

Pixel art of a token bucket metering small requests through a shield-shaped gateway while blocking an oversized request

Rate Limiting LLM APIs: Count Tokens, Not Requests

A diagram-first guide to token budgets, fair admission, and load shedding for shared LLM APIs, with a runnable Python simulation.

Pixel art comparison showing a glowing branching Radix tree with structured bracket gates on the left and a GPU processor core packing memory pages into parallel token streams on the right

SGLang vs. vLLM: RadixAttention, Structured Outputs, and When to Pick Which

A simple, diagram-first comparison of SGLang and vLLM: how RadixAttention works, how jump-forward decoding speeds up JSON outputs, how their schedulers compare, and which engine fits your workload.

Pixel art laptop projecting a holographic GPU cluster of amber wireframe cards, with a router tower sending request cubes to them and the brightest path going to the card that holds matching cache cubes

No GPU? Simulate One: vLLM, llm-d, and 16 Fake H100s on a Laptop

A hands-on lab for learning GPU scheduling and LLM routing without a GPU: kind, fake-gpu-operator, llm-d-inference-sim, and the llm-d router, with real screenshots and benchmark numbers.

Pixel art router tower sending glowing requests to GPU racks, with the brightest path going to the rack that already holds matching cache cubes

llm-d, Simply: A Smarter Router for vLLM on Kubernetes

A practical, diagram-first guide to llm-d: why round-robin wastes GPUs, how cache-aware routing works, which well-lit path to start with, and the commands to deploy it.

Pixel art workflow continuing through a durable history server from a crashed worker to a healthy replacement

Temporal Hands-On: Durable Execution on Kubernetes

A hands-on Temporal demo in Python: build a small order workflow, deploy it on Kubernetes, then delete the worker pods mid-execution and watch durable execution resume it.