Imran Nazir the-cross-art

Imran Nazir

AI Applied Engineer · LLM Serving · GPU Inference Infrastructure

LLM serving at scale — throughput/latency trade-offs across vLLM, SGLang, and Triton Inference Server
GPU inference internals — batching strategies, KV-cache management, capacity-aware scheduling
ML systems architecture — designing platforms that stay reliable under production load
Kubernetes-native orchestration for asynchronous, high-throughput inference workloads

	vLLM · vllm-metal _{Multimodal serving · MLX/Metal · paged attention} LFM2-VL image input · vision-tower adapter · first-token parity View work →
	Hugging Face · transformers _{Attention correctness · SDPA · seq2seq training} Decoder causality fix · root-cause trace · regression tests View work →
	OpenNMT · CTranslate2 _{Inference runtime · model conversion · C++} UMT5 support · per-layer attention bias · validated on a 2.9B checkpoint View work →

Selected upstream impact — PR evidence

Project	Contribution	Product / user impact
vllm-metal	#1182 — LFM2-VL image input on the paged path	`LFM2.5-VL-*` served text-only and silently dropped images. Now served end-to-end on Apple Silicon with EXACT first-token parity against native mlx-vlm, reproduced on a second independent setup
transformers	#49135 — UMT5 decoder self-attention was not causal under SDPA	The decoder could attend to future positions during teacher forcing, silently affecting every UMT5 model trained through the default `Seq2SeqTrainer` path. Position-0 logit drift fell from ~1e-1 to ~2e-6
CTranslate2	#2103 — UMT5 inference support via per-layer relative attention bias	Made UMT5 servable for the first time, closing a 2+ year feature request, with existing T5/mT5 conversions left byte-identical

Root cause over symptom _{Trace a numerical mismatch to the line that causes it, even across repos.}	Evidence before claims _{Quantify the before/after and pin the first bad commit.}
Narrow, reversible diffs _{Keep existing models bit-identical; no config flags or format changes.}	Tests that would have caught it _{Regression tests that fail on the old behavior, not just pass on the new.}