What the work involves
Most production recommenders follow the three-stage design Google teaches in its recommendation course. Candidate generation starts from the full catalog and returns a small subset fast. At YouTube, that step takes billions of videos down to hundreds or thousands. Several generators often run side by side: one pulls related items from a matrix factorization model, another pulls trending items, a third uses the social graph. A single scoring model then ranks the merged pool, so scores from different generators never have to be compared directly.
The ranking model can afford richer features because it sees far fewer items. Google's YouTube example predicts the chance a user watches a video from user features such as watch history, language and country, plus video features such as title, tags and an embedding. The final re-ranking step drops items the user said they disliked, gives a boost to fresh content, and keeps the list varied. YouTube's published system uses two separate deep networks: one generates candidates and one ranks them.
Objectives, testing and the limits of personalization
Picking the objective is the hardest decision on the project. Google's course warns that a model trained to maximize clicks drifts toward click-bait, and one trained on watch time drifts toward very long videos. Position bias also distorts the training data, because items lower on the screen get fewer clicks whatever their quality. A common fix is to score every candidate as if it sat in the top slot.
Netflix describes improving its recommender by pairing offline experiments on historical engagement data with A/B tests aimed at member retention and medium-term engagement. Offline metrics screen out weak ideas. The live test decides what ships. Personalization also gets deliberately limited. Leaked Facebook documents reported by The Batch showed posts the company judged harmful having their ranking scores cut by 50 to 90 percent. Meta's video team has also built a framework that shows users more varied content that still fits their interests.
How it relates to search and ads
Recommendation, search and ads ranking share the same building blocks: retrieval, a learned ranker and online testing. The boundaries between them are blurry. Netflix wrote that search on its service turns into a recommendations problem too. LinkedIn's job recommendation system sends a query to its distributed search index and then scores the jobs that come back with a final-pass ranking model. Meta runs ads ranking models to match ads to users, alongside its feed and video ranking teams.
For your career, this means ranking skills carry over. An engineer who has shipped a feed ranker can usually make a credible case for a search relevance or ads ranking role, and the reverse is also true.
Who hires and what hiring teams look for
The published work points to consumer platforms with large catalogs and heavy traffic. Examples include streaming (Netflix, YouTube), social feeds and video (Meta), and professional networks and job marketplaces (LinkedIn). Retailers and marketplaces have the same problem whenever the catalog is too big to browse.
Hiring teams generally want proof that you have taken a ranking change all the way to production. That means you framed the objective, built features that match between training and serving, trained a retrieval or ranking model, and read an A/B test honestly, including tests that failed. A solid ML coding background matters, and so does the judgment to say when engagement is the wrong target. No certification or license governs this work.
Pay
USD a year, before tax. These are US figures. BLS has no recommendation systems or AI series, so the BLS rows use software developers (BLS OEWS May 2025, moved to 2026 with the Employment Cost Index), the nearest series, which covers the whole occupation. The job ad rows are midpoints of pay ranges in 139 live machine learning engineer ads from 56 employers on AutomationRoles.ai on 5 November 2026. They lean toward large AI and tech employers and leave out stock and bonus. None of these figures is specific to recommendation work. Refer to the Machine Learning Engineers salary guide for figures from live adverts.
What you need
- Retrieval and candidate generationKnow embedding-based retrieval, matrix factorization and how to merge candidates from several generators under tight latency limits.
- Ranking modelsTrain scoring models on user and item features, and handle position bias in click data.
- Feature pipelinesBuild features from watch, click and purchase logs that match exactly between training and live serving.
- ExperimentationDesign A/B tests, choose metrics tied to long-term outcomes such as retention, and interpret results that are noisy or contradict each other.
- Objective judgmentSpot when a target like clicks or watch time rewards the wrong behavior, and use re-ranking for diversity, freshness and safety.
- No credential requiredNo license or regulator covers this job. Shipped ranking work counts for more than any certification.
How to get there
- Phase 1Frame the objectiveAgree on what counts as a good recommendation, and pick online metrics that won't reward click-bait or overly long content.
- Phase 2Build feature and log pipelinesTurn interaction logs into training data and serving features for both users and items.
- Phase 3Ship candidate generationStand up one or more fast generators that cut the catalog to hundreds or thousands of items.
- Phase 4Train the ranking modelScore the merged candidate pool with a single, richer model, and correct for position bias.
- Phase 5Add re-ranking rulesRemove disliked items, boost fresh content, enforce diversity, and demote content that breaks policy.
- OngoingTest online and iterateScreen ideas offline on historical data, then run A/B tests to decide what ships.
Open now on AutomationRoles.ai
Questions people ask
Is a recommendation systems engineer the same as a machine learning engineer?
It is a machine learning engineering specialty focused on retrieval and ranking. The day-to-day work leans heavily on feature pipelines, low-latency serving and online experiments. Many companies post these roles under the machine learning engineer title.
What are the three stages of a recommender?
Candidate generation narrows a large catalog to a small set, scoring ranks that set with a more precise model, and re-ranking applies final constraints such as diversity, freshness and removing disliked items. Google teaches this design in its public recommendation systems course.
How much does a recommendation systems engineer earn?
No source gives a pay figure for this specialty alone. The nearest BLS series, software developers, has a 2026-adjusted US median of $140,200. Machine learning engineer ads on AutomationRoles.ai had a median base midpoint of $227,400, leaning toward large tech employers and leaving out stock and bonus.
Why do recommenders need A/B tests if offline metrics exist?
Offline metrics computed on historical data don't always predict how real users will behave. Netflix describes using offline experiments to filter ideas and A/B tests on retention and engagement to decide what actually ships.
Can I move from search or ads into recommendations?
Usually, yes. All three use retrieval followed by learned ranking and online testing. LinkedIn's job recommendations run through a search index, and Netflix treats search as a recommendations problem.
Sources
- Google for Developers: Recommendation systems overview
- Google for Developers: Scoring
- ACM TMIS: The Netflix Recommender System
- ACM RecSys: Deep Neural Networks for YouTube Recommendations
- RecSys 2017: Personalized Job Recommendation System at LinkedIn
- IEEE Spectrum: AI Expert Engineers Meta's Video Recommendations
- DeepLearning.AI The Batch: How Facebook Fills the Feed
- U.S. Bureau of Labor Statistics: Software Developers, Occupational Outlook Handbook
