TMS LLC
Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years
ContractLos Angeles, California, United Statestoday
AI Engineer / Generative AI Engineer
Location: Los Angeles, CA (Hybrid)
Duration: 3+ Months (Extension Possible)
Experience: 15+ Years
Job Summary
We are seeking a hands-on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock AgentCore, and LLM evaluation to build, deploy, monitor, and continuously improve production-grade AI agents.
Key Responsibilities
- Develop AI agents using Python and Claude Agent SDK
- Deploy agents using AWS Bedrock AgentCore Runtime
- Design golden datasets and implement LLM-as-a-Judge evaluations
- Integrate automated evaluations into CI/CD pipelines as quality gates
- Implement Langfuse/OpenTelemetry for AI observability and tracing
- Evaluate agent accuracy, task completion, tool execution, and performance
- Support RAG pipelines and AWS data processing workflows
Required Skills
- Python and Agentic AI development
- AWS Bedrock AgentCore Evaluations / Bedrock evaluation
- Golden-dataset design and curation
- LLM-as-a-Judge design and calibration against human labels
- Automated evaluation and CI gate integration
- Langfuse and/or OpenTelemetry
Good to Have
- RAGAS or equivalent evaluation frameworks
- Trajectory evaluation and agentic red-teaming
- Bedrock Guardrails, Strands Agents SDK, and GitHub Actions
Qualifications
- 6–8+ years of software engineering, cloud engineering, or AI/ML experience
- Hands-on experience with production AI agents and continuous evaluation
- Strong problem-solving and troubleshooting skills
Additional Information
All your information will be kept confidential according to EEO guidelines.
About TMS LLC
tms is a global company that provides technology, marketing, and sourcing solutions for major consumer and corporate brands. The company operates as a creative agency, strategic consultancy, and technology provider, managing iconic long-term marketing platforms including McDonald's Happy Meal and MONOPOLY programs.
Industry
Marketing; Business Consulting and Services
Head office
Chicago, Illinois
Company size
1,200-2,300
Founded
1986
Brand activation and retail experiencePromotional and gamified experiencesSourcing and supply chain managementProduct licensing and merchandise developmentMarketing technology and strategy
Interested in this role?
Apply now to join TMS LLC.