AI & ML Tech Glossary

Inference budgeting sets and enforces limits on tokens, latency, compute, and cost during model inference. It uses controls like token caps, routing to smaller models,...

Instruction tuning is a supervised fine-tuning approach that trains a pretrained language model on instruction-response pairs so it learns general instruction-following behavior across many tasks,...

Inference serving is the production layer that hosts models and handles prediction requests with constraints on latency, throughput, cost, and reliability. For LLMs, it includes...

Inference optimization reduces the latency and cost of serving trained AI models by improving compute, memory, and batching efficiency, using methods like quantization, optimized attention...

Inference-time compute is the compute required to run a trained model in production. For LLMs, it covers prompt processing and token-by-token decoding and determines latency,...