TorchEmbed is now integrated into DeepSpeed and ships as a dependency. The fused Triton RoPE kernel replaces DeepSpeed’s transformer engine’s rotary embedding path, delivering 4–7× speedup over pure PyTorch with no code changes required from users. Read more →

Telugu is spoken by roughly 80 million people, but remains severely under-resourced in the ASR landscape. whisper-large-v3-te is our fine-tuned version of OpenAI’s Whisper Large v3, trained specifically for Telugu speech recognition. Why Telugu? Standard Whisper Large v3 performs reasonably on high-resource languages but degrades significantly on Telugu. The fine-tune targets conversational and broadcast Telugu,… Read more →

Single-judge LLM evaluation is noisy. Ask GPT-4 to rate two outputs and the ordering can flip on a re-run. JuryEval addresses this by running multiple LLM judges in parallel, then applying statistical significance tests to determine whether score differences are meaningful or just variance. The problem with single-judge evaluation LLM judges are inconsistent across temperatures,… Read more →

We built SLM-10M as a research experiment: can a sub-10M parameter transformer-based language model compete meaningfully on standard benchmarks? The answer turned out to be yes — SLM-10M currently tops the Open SLM Leaderboard in the under-10M tier. Architecture SLM-10M uses a compact transformer architecture with aggressive parameter sharing and sparse attention. We applied post-training… Read more →