Job Information
- Date Opened: 09/08/2026
- Job Type: Full time
- Industry: IT Services
- Location: Hyderabad OR Pune
- City: Hyderabad
- State/Province: Telangana
- Country: India
- Zip/Postal Code: 500081
- Work Mode: Hybrid
- Notice Period: 0-30 days
About the Role
MLOps / Serving Engineer
We are seeking an experienced MLOps / Serving Engineer who can design and operate the production serving infrastructure for fine-tuned LLMs on AWS — optimised inference engines, shadow-mode and staged rollout pipelines, monitoring dashboards, and the path from experimental model to full production traffic.
Experience: 5+ years
Key Responsibilities
- Deploy fine-tuned LLMs using vLLM, TensorRT-LLM, or Triton with continuous batching on AWS GPU instances
- Build shadow-mode deployment: run fine-tuned model alongside production, log comparison data without impacting live traffic
- Execute staged rollout: canary (5%) → gradual ramp (25% → 50% → 100%) with automated rollback on quality degradation
- Optimize inference for input-heavy workloads (~17K token inputs, ~130 token outputs): prefill throughput, KV-cache, INT8 quantization
- Build monitoring dashboards: latency, throughput, accuracy metrics, cost per request
- Design auto-scaling; implement high-availability (2× instances); automated rollback triggers on end-to-end quality metrics
Requirements
- 5+ years MLOps or ML infrastructure engineering
- Hands-on with vLLM, TensorRT-LLM, or Triton Inference Server
- Deep familiarity with g5, p4de, p5 instance families, EC2 auto-scaling
- Worked on deployment patterns like shadow-mode, canary, A/B traffic routing, automated rollback
- Experience with continuous batching, INT8 quantization, KV-cache management
- Expertise in Docker, Kubernetes (EKS) for ML workloads
- Worked with CloudWatch, Prometheus, Grafana
Benefits
- Comprehensive Medical Coverage: Health insurance of INR 5.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind
- Robust Protection Plans: Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones
- Retirement Benefits: PF and Gratuity provided as per standard government regulations
- Flexible Work Options: Enjoy hybrid work arrangements & flexible working hours
- Generous Leave Policy: 21 days of annual leave, in addition to 10 company-declared holidays
- Employee Well-being Spaces: Access to a dedicated break-out area with round-the-clock refreshments for relaxation and rejuvenation
About DATAECONOMY
DATAECONOMY is a data and AI consultancy that provides enterprise-grade solutions for data modernization, analytics, and artificial intelligence. The company serves financial services, banking, insurance, and other enterprises with services spanning data engineering, data governance, AI/ML, cloud enablement, and digital transformation.
Interested in this role?
Apply now to join DATAECONOMY.