automationroles.aiJoin
DATAECONOMY

MLOps / Serving Engineer

Hyderabad, Telangana, India3 months ago

Job Information

  • Date Opened: 09/08/2026
  • Job Type: Full time
  • Industry: IT Services
  • Location: Hyderabad OR Pune
  • City: Hyderabad
  • State/Province: Telangana
  • Country: India
  • Zip/Postal Code: 500081
  • Work Mode: Hybrid
  • Notice Period: 0-30 days

About the Role

MLOps / Serving Engineer

We are seeking an experienced MLOps / Serving Engineer who can design and operate the production serving infrastructure for fine-tuned LLMs on AWS — optimised inference engines, shadow-mode and staged rollout pipelines, monitoring dashboards, and the path from experimental model to full production traffic.

Experience: 5+ years

Key Responsibilities

  • Deploy fine-tuned LLMs using vLLM, TensorRT-LLM, or Triton with continuous batching on AWS GPU instances
  • Build shadow-mode deployment: run fine-tuned model alongside production, log comparison data without impacting live traffic
  • Execute staged rollout: canary (5%) → gradual ramp (25% → 50% → 100%) with automated rollback on quality degradation
  • Optimize inference for input-heavy workloads (~17K token inputs, ~130 token outputs): prefill throughput, KV-cache, INT8 quantization
  • Build monitoring dashboards: latency, throughput, accuracy metrics, cost per request
  • Design auto-scaling; implement high-availability (2× instances); automated rollback triggers on end-to-end quality metrics

Requirements

  • 5+ years MLOps or ML infrastructure engineering
  • Hands-on with vLLM, TensorRT-LLM, or Triton Inference Server
  • Deep familiarity with g5, p4de, p5 instance families, EC2 auto-scaling
  • Worked on deployment patterns like shadow-mode, canary, A/B traffic routing, automated rollback
  • Experience with continuous batching, INT8 quantization, KV-cache management
  • Expertise in Docker, Kubernetes (EKS) for ML workloads
  • Worked with CloudWatch, Prometheus, Grafana

Benefits

  • Comprehensive Medical Coverage: Health insurance of INR 5.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind
  • Robust Protection Plans: Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones
  • Retirement Benefits: PF and Gratuity provided as per standard government regulations
  • Flexible Work Options: Enjoy hybrid work arrangements & flexible working hours
  • Generous Leave Policy: 21 days of annual leave, in addition to 10 company-declared holidays
  • Employee Well-being Spaces: Access to a dedicated break-out area with round-the-clock refreshments for relaxation and rejuvenation

About DATAECONOMY

DATAECONOMY is a data and AI consultancy that provides enterprise-grade solutions for data modernization, analytics, and artificial intelligence. The company serves financial services, banking, insurance, and other enterprises with services spanning data engineering, data governance, AI/ML, cloud enablement, and digital transformation.

Industry
IT Consulting and Outsourcing
Head office
Dublin, Ohio, United States
Company size
588
Founded
2018
Data EngineeringData AnalyticsData GovernanceArtificial Intelligence and Machine LearningCloud Platform ServicesData Platform ModernizationBusiness IntelligenceMLOpsGenerative AIDigital Transformation
See all 12 jobs at DATAECONOMY→

Interested in this role?

Apply now to join DATAECONOMY.

Similar roles

Let employers come to you

Upload your CV and we build your profile from it. Employers contact you. Free for job seekers.

Create your profile
Free email alerts

Be the first to know.

New AI and automation jobs, employers and talent, straight to you as they go live. Stay in the loop.

Get the inside track →

MLOps / Serving Engineer