AI glossary
Learning to Rank
Learning to rank is a machine learning approach for training a model to order a list of items, such as search results or product recommendations, by relevance to a query, rather than to classify each item independently or predict a single score for it in isolation.
How it differs from classification
A standard classifier or regressor is trained to produce the best individual prediction for each input on its own. A learning-to-rank model is trained so that its outputs put the right items in the right relative order. This is a different objective even if the underlying model architecture is similar. When you use supervised learning for classification, the model learns to label each instance correctly. In ranking, the model learns to arrange instances in a specific sequence. The goal is not just to identify what an item is, but to determine where it belongs in a list relative to other items.
Pointwise, pairwise and listwise approaches
Ranking methods generally fall into three categories. Each handles the concept of “order” differently.
Pointwise ranking
Pointwise approaches treat ranking as predicting a relevance score for each item independently. The model assigns a score to every document or product. After scoring, the system simply sorts the items by that score. This method is straightforward because it reduces ranking to a standard regression or classification problem. However, it ignores the context of other items in the list. A high score does not guarantee the item is ranked above all others if the scoring function does not account for relative importance.
Pairwise ranking
Pairwise approaches train the model on pairs of items. The model learns to correctly predict which of the two items is more relevant. This method more directly optimizes for correct ordering than a pointwise score alone. By focusing on relative comparisons, the model learns to distinguish between better and worse items. This approach often yields better results than pointwise methods because it aligns the training objective more closely with the final goal: getting the order right.
Listwise ranking
Listwise approaches train directly on entire ranked lists at once. The model optimizes a metric that considers the whole list’s order rather than individual items or pairs. This is the most comprehensive approach because it accounts for the position of every item in the final output. Listwise methods often achieve the highest performance but can be more complex to implement. They are particularly useful when the position of an item matters significantly, such as in search results where the first few results receive the most attention.
Real examples: RankNet and LambdaMART
Two prominent examples illustrate how these approaches work in practice.
RankNet, introduced by Chris Burges and colleagues at Microsoft Research in a 2005 paper, is a pairwise learning-to-rank method. It trains a neural network using pairs of documents and gradient descent. RankNet was used in early versions of Microsoft’s Bing search ranking. It demonstrated how pairwise optimization could improve search relevance compared to traditional pointwise methods.
LambdaMART, also developed at Microsoft Research, combines the pairwise gradient idea from RankNet with gradient-boosted decision trees. It was the winning approach in the 2010 Yahoo! Learning to Rank Challenge. LambdaMART became a standard in the industry because it balanced accuracy with computational efficiency, combining tree-based models with the pairwise optimization logic that makes ranking effective.
Where it’s used
Learning to rank is used by web search engines to order search results by relevance. It is also used by recommendation systems to order suggested products or content. Additionally, ad systems use it to order which ads to show to users. In each case, the goal is to maximize user engagement or satisfaction by presenting the most relevant items first.
The technology relies on artificial neural networks and other machine learning techniques to process vast amounts of data. It helps systems understand complex relationships between queries and items. This allows for more personalized and accurate results across various platforms.
Evaluating a ranking
Common evaluation metrics for a ranking model include NDCG (Normalized Discounted Cumulative Gain). This metric rewards placing highly relevant items near the top of the list. It accounts for the position of each item and discounts lower positions. Another key metric is Mean Reciprocal Rank. This focuses on how high up the first relevant result appears. It is particularly useful when only the top result matters.
Choosing the right metric depends on the use case. If the first result is critical, Mean Reciprocal Rank is appropriate. If the entire list’s quality matters, NDCG is better. These metrics help developers tune their models and ensure they are optimizing for the correct objective.
FAQ
Is learning to rank the same as classification?
No. Classification assigns labels to individual items. Learning to rank arranges items in a specific order based on relevance.
What is the difference between pairwise and listwise ranking?
Pairwise ranking compares pairs of items to learn relative order. Listwise ranking considers the entire list at once to optimize the overall ranking structure.
Which algorithm is best for learning to rank?
There is no single best algorithm. RankNet and LambdaMART are popular choices. The best choice depends on your data, computational resources, and evaluation metrics.