AI & ML Tech Glossary

LLM rate limiting restricts requests or token usage per user/tenant to protect latency, control cost, and reduce abuse. Because LLM cost scales with tokens and...

Long-context fine-tuning adapts a language model to handle very long prompts by training on extended-context examples. It improves attention to earlier information, reduces “lost in...

LLM observability instruments and monitors LLM applications with traces, logs, and metrics that capture prompts, retrieval context, tool calls, tokens, latency, cost, and quality signals....

LLM guardrails are runtime controls—input/output checks, tool permissions, policy enforcement, and monitoring—that constrain a language model to be safer, more reliable, and compliant in real...

An LLM router is a middleware layer that dynamically chooses the best language model or provider for each request based on cost, latency, quality, and...

Low-Rank Adaptation (LoRA) fine-tunes large models efficiently by learning small low-rank weight updates while freezing the base model, reducing GPU memory, training time, and storage...