Triplet Preference Data Resolves Correlation Blind Spot in Random Utility Models
A study presented at ICLR proves that pairwise comparison data cannot identify correlations among item utilities in random utility models, whereas ordinal rankings of three alternatives (or a mix of best-of-two and best-of-three data) can, with efficient estimators. The result shifts data-collection practice for preference learning and has direct implications for recommender systems and LLM alignment.
A proof presented at the International Conference on Learning Representations shows that pairwise comparisons, the dominant data format for estimating random utility models (RUMs) since Thurstone's 1927 law of comparative judgment, cannot reveal correlations among the utilities of different alternatives. The same paper shows that ordinal rankings of three alternatives, or a mix of best-of-two and best-of-three judgments, do identify such correlations, and that the number of required experiments need not grow exponentially in the number of items. This moves the practical bottleneck from model estimation to data-collection design.1
Context
RUMs treat choice as a comparison of latent utility values, with randomness accounting for person-to-person and occasion-to-occasion variation. They are standard tools for counterfactual policy questions, such as predicting transport mode and route choices under a corridor closure, and are central to preference-based training of large language models. The commonly used pairwise-comparison protocol is cognitively easy for respondents, but it assumes that the utility random variables are independent, which the paper argues is both false and consequential for platforms such as Netflix, Amazon, or Google.1
Assessment
The central claimed result is a mathematical impossibility: correlations among utilities cannot be recovered from two-way comparisons alone. The constructive counterpart—that triplewise rankings or a combination of pairwise and triplewise data suffice—comes with an algorithmic result: efficient estimators exist, and the data requirement does not explode with catalog size. If correct, the result gives a concrete, low-cost change to data collection: ask users to rank three items instead of posing a binary choice. The MIT group further claims that the same framework underpins LLM alignment, because ranking-based reward modeling relies on human preferences over candidate outputs.1
Caveats
The evidence here is a press release, not the peer-reviewed paper; formal identification conditions, sample-complexity rates, and estimator construction are not reported in the document. The external endorsement from Emma Frejinger is qualitative. My confidence would rise with access to the ICLR paper's proofs, simulations, and any negative results on partial triplewise designs. The claim about LLM alignment is an assertion about the field's trajectory rather than a demonstrated experimental outcome in this article.1
In his 1927 paper, “A law of comparative judgment,” the American psychologist L. L. Thurstone proposed that when people select one option among multiple alternatives, they are picking the one that has the highest value to them, even though they cannot assign a particular number to that choice.
Thurstone was a pioneer of “psychometrics” — a field built upon the premise that mental processes, which we cannot see, can nevertheless be measured and quantified. His 1927 paper laid the groundwork for what are now called random utility models, which provide a mathematical framework for describing human preferences — information that can be relied upon, in turn, to make predictions about various hypothetical situations.
Random utility models (RUMs) are so named because they assess the “utility,” or benefit, that can be obtained from a given choice — such as deciding which book to read first among the stack of novels you brought back from the library. “These models are inherently random,” explains Gabriele Farina, an assistant professor in MIT’s Department of Electrical Engineering and Computer Science (EECS) and principal investigator at the Laboratory for Information and Decision Systems (LIDS), “because people are different. Everyone has their own preferences, and even those preferences can vary from time to time.” For example, someone who normally picks coffee over tea in the morning, and prefers tea after dinner, may, upon occasion, mix up that order entirely.
RUMs, to be sure, are frequently used within government and industry in situations of far greater consequence than the selection of a hot (or iced) beverage. The models routinely facilitate predictions regarding what people will elect to do in so-called counterfactual (“what-if”) scenarios such as: How will they get to work or school if a major thoroughfare is shut down for construction? What routes and modes of transport will they take? Or, if a city suddenly receives a windfall of $20 million, how should those funds be disbursed to maximize the common good?
Given that RUMs have been with us for almost 100 years, growing in sophistication over time, one might imagine that, at this stage, there would be little room for improvement. That, however, is not the case.
A paper presented in April at the International Conference on Learning Representations in Rio de Janeiro, Brazil, uncovered basic facts that show there is much more to be gleaned from these models than had traditionally been supposed. The paper was authored by Yeshwanth Cherapanamjeri, a former MIT postdoc now based at Nanyang Technological University in Singapore; Farina, also core faculty in MIT’s Operations Research Center (ORC); Constantinos Daskalakis, the Avanessians Professor of Computer Science at MIT and a member of MIT's Computer Science and Artificial Intelligence Laboratory; and Sobhan Mohammadpour, an MIT PhD student in computer science based at LIDS and EECS.
The group’s findings stem, in part, from a deficiency in the way RUMs are commonly estimated in practice, which has persisted since the days of Thurstone. The data upon which the models are estimated have been largely drawn from so-called pairwise-comparisons: In a choice between items A and B — whether it pertains to movies on Netflix, competing products on Amazon.com, news stories posted on Google, and so forth — which one would you pick? One reason this approach has been so pervasive, explains Daskalakis, is that “assigning a precise numerical score, such as 4.37, to the benefit you get from a single item is very hard. Whereas comparing two things, and deciding which one you like better, is cognitively much easier to do.” But therein lies the rub, he adds. “With this way of assessing people’s preferences, looking at just two things at a time, it is impossible to find correlations between the numerous choices.”
The standard way of applying RUMs assumes that the utilities derived from A and B are independent, but they may, in fact, be linked, and that would be important to know. If someone campaigning for elective office finds out that a potential voter favors gun control, for instance, there is a reasonable chance that same person also favors government-sponsored child care. Similarly, a fan of independent movies might also be partial to foreign films, but less enthusiastic about Hollywood action blockbusters. “If a digital platform has a blind eye to the existence of such correlations, it will not be able to estimate preferences very accurately,” Daskalakis notes. “And if Netflix regularly shows you an assortment of movies you don’t care about, you might sign off and cancel your subscription.”
The MIT team proved that it is impossible to get information about correlations from two-way comparisons alone. Correlations can be discerned, however, when large numbers of people rate three alternatives in their order of preference. The same information can also be obtained from a combination of best-of-three and best-of-two choices. In practice, Mohammadpour explains, “you would get a bunch of people to rank three items. You could then utilize the method we developed for merging those individual results into one big model that can provide us with the big picture.”
Their research effort, according to Farina, is focused on the computational side of RUMs, devising algorithms that can extract preference information and figuring out how much data is needed to do so or, equivalently, how many experiments need to be run. The good news, he says, is that efficient algorithms are, indeed, possible for this purpose. The requisite number of experiments does not grow exponentially with the number of items in the catalog or database that’s under review.
“This paper provides a crucial breakthrough,” comments Emma Frejinger, a computer scientist at the University of Montreal. “It mathematically proves why traditional data collection fails and demonstrates that simply asking users for their best-of-three [choices] unlocks the ability to accurately train these powerful models. This finding provides a highly practical roadmap for collecting better data to drive more accurate optimizations.”
“Building utility models is going to remain a very active area,” Daskalakis insists. “Just as RUMs have been critical to the internet economy since the late 1990s, they are, and will remain to be, critical to the alignment of AI models going forward.” More importantly, he adds, “RUMs play a central role in the commercial viability and usefulness of large language models [LLMs].” During the training period, people are typically asked to rank the various candidate outputs of these LLMs, from which the models can gain a better sense as to the kind of text — in terms of tone, style, and content — that is preferred.
Given that we’re constantly “besieged with a vast sea of options in so many different domains,” Daskalakis says, “you cannot possibly ask people to communicate all their personal preferences for all possible scenarios. So what you can do instead is build a model that predicts what people think about the different possible outcomes. And you have to keep improving and updating your model in an iterative process until, hopefully, you can make good predictions.”
The source text above remains the property of its publisher and is reproduced for commentary.
