Bruno Nicenboim's Blog https://bruno.nicenboim.me/posts/ Posts about computational cognitive science, Bayesian methods, EEG, and AI. quarto-1.8.27 Wed, 06 May 2026 22:00:00 GMT Of birds and planes Bruno Nicenboim https://bruno.nicenboim.me/posts/posts/2026-05-07-birds-and-planes/ I sometimes struggle to explain to people outside the field why psycholinguists like me use large language models to investigate how people understand language. This post breaks it down in a very simplified way: one important thing that happens when you read or listen, how language models perform something at least superficially similar, and why that comparison actually teaches us something.

Your brain is always one step ahead

Try finishing this:

She put on her coat, picked up her keys, and walked out the ___

You almost certainly thought about door.

Now this one:

The scientist published her results in a peer-reviewed ___

You probably thought about journal. Or maybe paper.

He had a coffee and a ___

Croissant? Slice of cake? Brownie? This one is harder. Several continuations fit equally well.

On some occasions, words are much easier to predict than on others. It seems that predicting the next word is something your brain does automatically, all the time, without you noticing.

How do we actually know the brain predicts?

How do we know this also happens passively when you read or listen? You can’t feel your brain making predictions.

Scientists have found ways to figure this out.

Image by Lucs-kho, public domain, via Wikimedia Commons.

One way is tracking people’s eyes while they read. Your eyes don’t slide smoothly across a page. They jump from word to word. It looks like the figure on the right: circles where your eyes stop (called fixations), and lines where they jump (called saccades). And it turns out that the eyes spend less time on predictable words, sometimes skipping them entirely, while lingering on surprising ones. The brain already has a good guess for predictable words, so it doesn’t need to look as carefully.

There’s also a more direct way to look into the brain. Your brain runs on electricity: billions of cells communicating through small bursts of electrical activity, these are your neurons. All those tiny signals add up, and some of that activity actually reaches the surface of your head. By placing small sensors on someone’s scalp (a technique called EEG, for electroencephalography), you can pick up those signals while the person reads. It looks like a cap covered in wires, and what it records is a kind of summary of what millions of neurons are doing at each moment. When a word shows up that the reader didn’t expect, the electrical response shifts. The more surprising the word, the bigger the shift.

Photo by Chris Hope, CC BY 2.0, via Wikimedia Commons.

Context is everything

Watch what happens when you add more context to the last example, “He had a coffee and a ___“. Now we have, for example:

He was celebrating his birthday. He had a coffee and a ___

The continuation slice of cake is much more obvious.

Guesses become easier the further into a sentence you get. Your brain pulls in everything available: the words already said, what makes sense in the real world, what people typically say in that kind of situation. More context generally means fewer plausible options, and easier guesses.

The same thing happens when you listen. You start processing what someone is saying before they’ve finished the sentence. In conversation, people often begin composing their reply while the other person is still speaking, because they can already tell where it’s going.

What large language models are doing

Large language models are programs trained by reading enormous amounts of text, including books, news articles, websites, and conversations.

The training works like this: the model sees a sequence of words and tries to guess what comes next. When it’s wrong, it adjusts a little. After billions of rounds, it develops something like an intuition for how language flows, which words tend to follow which, in what contexts. This next-word prediction task is actually the foundation of chatbots like ChatGPT. Before a chatbot can hold a conversation or answer your questions, it first has to learn the patterns of language by practicing exactly this: predicting the next word, over and over, on enormous amounts of text.

What makes a language model bigger or smaller comes down to parameters: numbers inside the model that get adjusted during training. Think of them as tiny knobs. Each knob controls a small part of how the model responds to a word or a pattern. More knobs means finer distinctions and subtler patterns. Fewer knobs means a rougher, blurrier picture of the language.

Of birds and planes

Are human minds and language models the same thing? Definitely not. They are as different as birds and airplanes. One is alive and evolved over millions of years; the other is engineered metal. They also have very different motivations for flying. But they both fly, and they both succeed because they exploit the same physics: aerodynamics, the way air flows around a wing.

Image generated with ChatGPT (OpenAI).

Studying airplanes can teach us a lot about the medium birds fly through. Not about feathers or muscles, but about what makes flight possible in the first place.

Something similar is going on with language models and human minds. They’re built completely differently, but they both navigate the same thing: language. By studying what a model finds predictable or surprising, or how it encodes words, we learn about regularities in the language: what tends to appear where, what’s common, what’s unusual. Those are the same regularities that shape how our brains process words.

How far this analogy stretches is genuinely controversial. Some researchers argue that language models are so different from brains that comparing them is misleading. Others think the similarities go deeper than we’d expect. Nobody has settled this, and figuring it out is an intriguing open question in the field right now.1

How to cite this post

BibTeX:

@misc{nicenboim2026ofbirdsandplanes,
  author = {Nicenboim, Bruno},
  title = {Of birds and planes},
  year = {2026},
  month = {May},
  url = {https://bruno.nicenboim.me/posts/posts/2026-05-07-of-birds-and-planes/},
  doi = {10.5281/zenodo.20084629}
}

APA:

Nicenboim, B. (2026, May 07). Of birds and planes. https://doi.org/10.5281/zenodo.20084629

Footnotes

  1. For a longer treatment of this analogy, see Grace Lindsay’s Planes don’t flap their wings: does AI work like a brain? on Aeon.↩︎

Reuse

MIT
]]> SciCom LLM psycholinguistics computational https://bruno.nicenboim.me/posts/posts/2026-05-07-birds-and-planes/ Wed, 06 May 2026 22:00:00 GMT Bayesian ordinal models: A very short practical guide Bruno Nicenboim https://bruno.nicenboim.me/posts/posts/2026-01-09-ordinal-models/ I’ll first load some R packages that will be useful throughout this post.

library(extraDistr)
library(tidytable)
library(ggplot2)
library(brms)
library(bayesplot)
library(patchwork)
library(ggridges)

theme_set(theme_minimal(base_size = 14))

Introduction

This post explains some nuances of Bayesian ordinal regression with cumulative links and focuses on how to set sensible priors.1 These models are appropriate for modeling data where the dependent variables are ordered responses without meaningful metric distances, such as Likert scales. The presentation builds on Bürkner and Vuorre (2019), but restricts attention to cumulative models.

Modeling an intervention effect

In an ordinal regression with a cumulative link, our goal is to understand how a latent variable shifts based on predictors or interventions. Suppose we want to assess how reading a book about Bayesian modeling3 affects responses to our understanding question.

Simulating data

Let’s generate synthetic data to illustrate this. We’ll simulate a group of students before and after reading our book.

This type of model assumes that, underlyingly, there is a continuous latent “rating”, and we specify a regression on this latent variable that determines how the response categories are affected by our manipulation (note that there is no intercept here; the thresholds will later act as something close to intercepts for each category):

with

where is a treatment indicator (in more realistic settings, one may prefer richer contrast coding; see Chapter 6 of Nicenboim, Schad, and Vasishth (2025)).

In principle, one could allow the predictor to have category-specific effects by estimating regressions for each threshold. However, Bürkner and Vuorre (2019) argue that while this parameterization is unproblematic for some ordinal models (such as sequential or adjacent-category models), it can lead to difficulties in model fitting for cumulative models.

To move from the continuous latent variable to a discrete response, we define the observed outcome as follows:

with boundary conditions and .

We keep the thresholds () defined above as the default cutoff points (brms refers to these as intercepts, though note the difference in sign convention below).

Given a cumulative link, the probability of observing category is

Here, denotes the cumulative distribution function of the error term of . We assume a logistic distribution for computational convenience, which yields a logistic cumulative link (assuming a normal distribution would lead to a probit link, with otherwise very similar expressions).

Thus, under a logistic cumulative link, the probability of observing category is

where denotes the CDF of the logistic distribution (available in R via plogis()). Notice that the regression term, , is subtracted from each threshold.

To simulate data from this model, we treat the resulting category probabilities as defining a categorical distribution and sample responses accordingly:

One important insight is that affects all categories equally: when , increases, increasing the probability of higher response categories at the expense of lower response categories; when , decreases, increasing the probability of lower categories at the expense of higher response categories.

In the following simulation, we use the last thres values we defined (-0.5, 0.5, 3, 4.5) as the cutoff points of the model (; called intercepts in brms), and we assume that reading increases (and also on average) by . We generate data from 1000 participants before and after reading our book:

set.seed(123)

N <- 1000
tau <- thres
beta <- 1.5
eta_noread <- 0 * beta
eta_read  <- 1 * beta

df_sim <- data.frame(
  read = rep(c(0, 1), each = N),
  category = c(
    rcat(N, diff(plogis(c(-Inf, tau - eta_noread, Inf))), labels = category),
    rcat(N, diff(plogis(c(-Inf, tau - eta_read, Inf))), labels = category)
  )
) |>
  mutate(response = as.numeric(category))

# Display summary statistics
table(df_sim$read, df_sim$category)
   
    Nothing A little More or less Something Everything
  0     382      238          329        40         11
  1     114      154          549       138         45

Let’s visualize the distribution of responses in both groups:

Code 
ggplot(df_sim, aes(x = response, fill = factor(read, labels = c("Before reading", "After reading")), 
                   group = factor(read))) +
  geom_bar(position = "dodge", alpha = 0.7, color = "black") +
  scale_fill_manual(values = c("Before reading" = "coral", "After reading" = "forestgreen")) +
  labs(title = "Distribution of Responses",
       x = "Understanding Level",
       y = "Count",
       fill = "Group") +
  theme_minimal(base_size = 12)

Now that we have (simulated) data, let’s fit it with brms.

Understanding priors for ordinal models

When fitting ordinal regression models with brms, we need to specify priors for (at least):

  1. “Intercepts” which in fact are thresholds or cutoff points
  2. Regression coefficients

In brms, the thresholds are automatically constrained to be ordered ().

Why tight priors concentrate probability on extremes

Let’s start by examining an intercept-only model with tight priors on the thresholds:

priors_tight <- c(
  prior(normal(0, 0.5), class = "Intercept")
)

With normal(0, 0.5) “Intercept” priors, all thresholds are pulled toward zero with relatively little spread. This concentrates the thresholds in a narrow region. This means that when thresholds cluster tightly around zero, most of the latent distribution’s mass falls outside the middle thresholds: either far below or far above . This assigns most probability to extreme categories (the first and last).

fit_prior_tight <- brm(
  response ~ 1,
  data = df_sim,
  family = cumulative(),
  prior = priors_tight,
  sample_prior = "only",
  cores = 4,
  seed = 123,
  refresh = 0,
  control = list(adapt_delta = 0.9) # avoid divergent transitions
)
Code 
ppd_ridgeplot <- function(fit, title = "Prior Predictive Distribution", 
                          subtitle = NULL, ndraws = 500) {
  
  yrep <- posterior_predict(fit, ndraws = ndraws)
  
  proportions_per_draw <- lapply(1:nrow(yrep), function(i) {
    props <- table(factor(yrep[i, ], levels = 1:5)) / ncol(yrep)
    data.frame(
      draw = i,
      response = factor(1:5),
      proportion = as.numeric(props)
    )
  })
  ppd_data <- do.call(rbind, proportions_per_draw)
  ggplot(ppd_data, aes(x = proportion, y = response)) +
    geom_density_ridges(fill = "steelblue", alpha = 0.7, scale = 0.9, stat = "binline") +
    scale_x_continuous(breaks = c(0,.2,.4,.6,.8,1)) +
    geom_vline(xintercept = 0.2, linetype = "dashed") +
    labs(title = title,
         subtitle = subtitle,
         x = "Proportion",
         y = "Response Category") +
    theme_minimal(base_size = 12) +
    coord_flip()
}
ppd_ridgeplot(fit_prior_tight, 
              title = "Prior Predictive Distribution: Tight Priors (SD = 0.5)",
              subtitle = "Most probability mass concentrated in extreme categories")

With tight priors, the model tends to predict extreme responses, as the thresholds cluster near zero and don’t span the full range of the latent distribution.

Intercept-only model: Medium priors

Now let’s use more dispersed priors:

priors_medium <- c(
  prior(normal(0, 3), class = "Intercept")
)

With normal(0, 3) “Intercept” priors, the thresholds can spread out more widely across the latent scale. This means the middle thresholds are more likely to be well-separated, capturing more of the distribution’s central mass and leading to more balanced predictions across all categories.

fit_prior_medium <- brm(
  response ~ 1,
  data = df_sim,
  family = cumulative(),
  prior = priors_medium,
  sample_prior = "only",
  cores = 4,
  seed = 123,
  refresh = 0
)
ppd_ridgeplot(fit_prior_medium, 
              title = "Prior Predictive Distribution: Medium Priors (SD = 3)",
       subtitle = "More balanced distribution across categories")

Intercept-only model: Diffuse priors

Let’s examine very diffuse priors:

priors_diffuse <- c(
  prior(normal(0, 10), class = "Intercept")
)

fit_prior_diffuse <- brm(
  response ~ 1,
  data = df_sim,
  family = cumulative(),
  prior = priors_diffuse,
  sample_prior = "only",
  cores = 4,
  seed = 123,
  refresh = 0,
  control = list(adapt_delta = 0.9)
)
ppd_ridgeplot(fit_prior_diffuse, 
              title = "Prior Predictive Distribution: Diffuse Priors (SD = 10)",
       subtitle = "Very flat distribution - least informative")

Very diffuse priors allow thresholds to be extremely spread out, often resulting in a very sparse distribution across categories.

Intercept-only model: Nudging priors

We might have substantive information suggesting most people start as beginners. We can encode this by centering the “Intercept” priors at a positive value:

priors_mid_inf <- c(
  prior(normal(1, 3), class = "Intercept")
)

fit_mid_inf <- brm( 
  response ~ 1,
  data = df_sim,
  family = cumulative(),
  prior = priors_mid_inf,
  sample_prior = "only",
  cores = 4,
  seed = 123,
  refresh = 0
)
ppd_ridgeplot(fit_mid_inf, 
              title = "Prior Predictive Distribution: Nudged Priors (Mean = 1, SD = 3)",
       subtitle = "Prior belief that lower understanding is more common")

By centering the”Intercept” priors at 1 instead of 0, we shift thresholds rightward, making lower categories more probable a priori. On the other hand, if we had substantial information that suggests the opposite (e.g. that most people start advanced), we would center the “Intercept” priors at a negative value, shifting the thresholds leftward, making higher categories more probable a priori.

Prior predictive distribution for the slope

Let’s also examine what our prior beliefs about the treatment effect imply:

priors_full <- c(
  prior(normal(0, 3), class = "Intercept"),
  prior(normal(0, 1.5), class = "b")
)

fit_slope_prior <- brm(
  response ~ read,
  data = df_sim,
  family = cumulative(),
  prior = priors_full,
  sample_prior = "only",
  cores = 4,
  seed = 123,
  refresh = 0,
  control = list(adapt_delta = 0.9)
)
ppd_ridgeplot(fit_slope_prior,
              title = "Prior Predictive Distribution: Full Model with Slope Prior",
       subtitle = "normal(0, 1.5) prior on treatment effect")

The prior normal(0, 1.5) on the slope is weakly informative: it allows for moderate effects in either direction while gently regularizing against implausibly large effects.

Fitting to the simulated data

Now let’s fit the model to our simulated data:

fit_full <- brm(
  response ~ read,
  data = df_sim,
  family = cumulative(),
  prior = priors_full,
  cores = 4,
  seed = 123,
  warmup = 1000,
  iter = 2000,
  refresh = 0
)
fit_full
 Family: cumulative 
  Links: mu = logit 
Formula: response ~ read 
   Data: df_sim (Number of observations: 2000) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Regression Coefficients:
             Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept[1]    -0.50      0.06    -0.62    -0.38 1.00     4501     3153
Intercept[2]     0.50      0.06     0.37     0.62 1.00     4802     3359
Intercept[3]     2.98      0.09     2.79     3.16 1.00     3939     3068
Intercept[4]     4.55      0.15     4.26     4.85 1.00     3991     2905
read             1.50      0.09     1.32     1.66 1.00     3854     2924

Further Distributional Parameters:
     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
disc     1.00      0.00     1.00     1.00   NA       NA       NA

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

If this were real data, based on the read coefficient, we would conclude that reading our book affects understanding. The positive coefficient read ≈ 1.5 indicates that reading makes higher categories (more understanding) more probable.

Interpreting predicted probabilities

To understand what the model estimates mean in practical terms, let’s examine the predicted probabilities:

Code 
ce <- conditional_effects(fit_full, effects = "read", categorical = TRUE)

plot(ce, plot = FALSE)[[1]] +
  scale_fill_brewer(palette = "RdYlGn", direction = 1) +
  labs(title = "Predicted Response Probabilities by Reading Status",
       subtitle = "Model predictions with 95% credible intervals",
       x = "Read Book",
       y = "Predicted Probability",
       fill = "response") +
  theme_minimal(base_size = 12) +
  theme(legend.position = "right")

The conditional effects plot shows how reading the book shifts the entire distribution toward higher understanding categories. Before reading (read = 0), lower categories have higher probabilities; after reading (read = 1), the distribution shifts substantially toward “something” and “everything.”

We can also examine specific fitted values:

Code 
new_data <- data.frame(read = c(0, 1))
fitted_probs <- fitted(fit_full, newdata = new_data)

fitted_df <- data.frame(
  read = rep(c("Before", "After"), each = 5),
  category = rep(1:5, times = 2),
  probability = c(fitted_probs[1, "Estimate", ], fitted_probs[2, "Estimate",]),
  lower = c(fitted_probs[1, "Q2.5",], fitted_probs[2 , "Q2.5",]),
  upper = c(fitted_probs[1, "Q97.5",], fitted_probs[2 , "Q97.5",])
)

ggplot(fitted_df, aes(x = factor(category), y = probability, fill = read)) +
  geom_col(position = "dodge", alpha = 0.7, color = "black") +
  geom_errorbar(aes(ymin = lower, ymax = upper), 
                position = position_dodge(0.9), width = 0.2) +
  scale_fill_manual(values = c("Before" = "coral", "After" = "forestgreen")) +
  labs(title = "Model-Predicted Probabilities with 95% Credible Intervals",
       x = "Understanding Category",
       y = "Predicted Probability",
       fill = "Reading Status") +
  theme_minimal(base_size = 12)

The fitted probabilities show concrete estimates: for instance, before reading, the probability of responding “nothing” is about 0.38, whereas after reading it drops to about 0.12. Conversely, the probability of responding “everything” increases from about 0.01 to 0.05.

Posterior predictive check

Finally, let’s verify that the model fits the observed data well:

pp_check(fit_full, type = "bars", ndraws = 100) +
  labs(title = "Posterior Predictive Check: Full Model",
       subtitle = "Model predictions compared to observed data") +
  theme_minimal(base_size = 12)

The posterior predictive distribution closely matches the observed data patterns, indicating good model fit.

Recovery of the parameters

Let’s examine whether the model successfully recovered the true parameters we used to generate the data. We simulated with thresholds and a treatment effect of .

true_params <- c(thres, beta)
names(true_params) <- c(paste0("b_Intercept[", 1:4, "]"), "b_read")

posterior_draws <- as.array(fit_full,
                            variable = names(true_params))

mcmc_recover_hist(posterior_draws, true = true_params) +
  labs(title = "Parameter Recovery",
       subtitle = "Posterior distributions (histograms) vs. true values (vertical lines)") +
  theme_minimal(base_size = 12)

The histograms show the posterior distributions for each parameter, with vertical lines indicating the true values used in simulation. We can see that:

  1. Thresholds: All four threshold parameters are well-recovered, with posterior distributions centered near the true values
  2. Treatment effect: The read coefficient is also well-recovered, with the posterior centered around 1.5

Conclusion

This short tutorial has demonstrated how cumulative link models provide a principled framework for ordinal regression.

By respecting the ordinal nature of the data, rather than treating categories as metric, cumulative link models yield more appropriate inferences for ordered categorical outcomes. The example above is intentionally simple; in cognitive science, data are typically clustered (e.g., by participants or items), which naturally calls for hierarchical models. For an application to ordinal ratings in this setting, see Taylor et al. (2022). For a more general introduction to hierarchical modeling with brms, see chapter 5 of Nicenboim, Schad, and Vasishth (2025).

How to cite this post

BibTeX:

@misc{nicenboim2026bayesianordinalmodelsaveryshortpracticalguide,
  author = {Nicenboim, Bruno},
  title = {Bayesian ordinal models: A very short practical guide},
  year = {2026},
  month = {January},
  url = {https://bruno.nicenboim.me/posts/posts/2026-01-09-bayesian-ordinal-models-a-very-short-practical-guide/},
  doi = {10.5281/zenodo.18225472}
}

APA:

Nicenboim, B. (2026, January 09). Bayesian ordinal models: A very short practical guide. https://doi.org/10.5281/zenodo.18225472

Session info

R version 4.5.2 (2025-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Amsterdam
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ggridges_0.5.7      patchwork_1.3.2     bayesplot_1.15.0   
[4] brms_2.23.0         Rcpp_1.1.1          ggplot2_4.0.2      
[7] tidytable_0.11.2    extraDistr_1.10.0.1

loaded via a namespace (and not attached):
 [1] tensorA_0.36.2.1        bridgesampling_1.2-1    tidyr_1.3.2            
 [4] generics_0.1.4          stringi_1.8.7           lattice_0.22-7         
 [7] digest_0.6.39           magrittr_2.0.4          evaluate_1.0.5         
[10] grid_4.5.2              RColorBrewer_1.1-3      mvtnorm_1.3-3          
[13] fastmap_1.2.0           plyr_1.8.9              jsonlite_2.0.0         
[16] Matrix_1.7-4            pkgbuild_1.4.8          backports_1.5.0        
[19] gridExtra_2.3           Brobdingnag_1.2-9       purrr_1.2.1            
[22] QuickJSR_1.8.1          scales_1.4.0            codetools_0.2-20       
[25] abind_1.4-8             cli_3.6.5               rlang_1.1.7            
[28] withr_3.0.2             yaml_2.3.12             otel_0.2.0             
[31] StanHeaders_2.36.0.9000 inline_0.3.21           rstan_2.36.0.9000      
[34] tools_4.5.2             parallel_4.5.2          reshape2_1.4.5         
[37] rstantools_2.6.0        checkmate_2.3.3         coda_0.19-4.1          
[40] dplyr_1.1.4             vctrs_0.7.1             posterior_1.6.1.9000   
[43] R6_2.6.1                stats4_4.5.2            matrixStats_1.5.0      
[46] lifecycle_1.0.5         stringr_1.6.0           htmlwidgets_1.6.4      
[49] pkgconfig_2.0.3         RcppParallel_5.1.11-1   pillar_1.11.1          
[52] gtable_0.3.6            loo_2.9.0.9000          data.table_1.18.2.1    
[55] glue_1.8.0              xfun_0.55               tibble_3.3.1           
[58] tidyselect_1.2.1        knitr_1.51              farver_2.1.2           
[61] htmltools_0.5.9         nlme_3.1-168            labeling_0.4.3         
[64] rmarkdown_2.30          compiler_4.5.2          S7_0.2.1               
[67] distributional_0.6.0

References

Bürkner, Paul-Christian, and Matti Vuorre. 2019. “Ordinal Regression Models in Psychology: A Tutorial.” Advances in Methods and Practices in Psychological Science 2 (1): 77–101. https://doi.org/10.1177/2515245918823199.
Nicenboim, Bruno, Daniel J. Schad, and Shravan Vasishth. 2025. Introduction to Bayesian Data Analysis for Cognitive Science. 1st ed. Chapman; Hall/CRC. https://doi.org/10.1201/9780429342646.
Taylor, Jack E., Guillaume A. Rousselet, Christoph Scheepers, and Sara C. Sereno. 2022. “Rating Norms Should Be Calculated from Cumulative Link Mixed Effects Models.” Behavior Research Methods 55 (5): 2175–96. https://doi.org/10.3758/s13428-022-01814-7.

Footnotes

  1. Thanks to Lenneke Lichtenberg for a helpful discussion about this kind of model.↩︎

  2. The logistic distribution is not that different from the normal distribution: it is a symmetrical bell-shaped distribution. The reason for using this distribution rather than the normal distribution is that the modeling will need to rely on the cumulative density function (CDF), and the logistic tends to be more numerically stable↩︎

  3. Shameless self-promotion of Nicenboim, Schad, and Vasishth (2025).↩︎

Reuse

MIT
]]> Stan Bayesian Tutorial R brms https://bruno.nicenboim.me/posts/posts/2026-01-09-ordinal-models/ Thu, 08 Jan 2026 23:00:00 GMT A simple way to model rankings with Stan Bruno Nicenboim https://bruno.nicenboim.me/posts/posts/2026-01-06-a-simple-way-to-model-rankings-with-stan/
NoteUpdate Notice

This is an updated version of a post originally published on March 21, 2021.

I’ve updated this post in January 2026 to work with current versions of Stan and R packages. The main changes include:

  • Updated Stan syntax to current standards

  • Using cmdstanr instead of rstan and some other R packages.

The core content and ideas remain the same.

The initial problem

I wrote what I thought was the generative process for some modeling work, and it looked too common to not have a name, so I started asking around on what it was a very popular website back in 2021, Twitter.

One useful clue was about the exploded logit distribution.1

In this post, I’ll show how this model can be fit in the probabilistic programming language Stan, and how it can be used to describe the underlying order of ranking data.

I’m going to load some R packages that will be useful throughout this post.

library(tidytable) # Nicer alternative to dplyr and purrr
library(ggplot2) # Nice plots
library(extraDistr) # More distributions
library(rcorpora) # Get random words
library(cmdstanr) # Lightweight Stan interface
library(bayesplot) # Nice Bayesian plots
set.seed(42)  # Keep everything R the same (not for Stan though)

Ranking data

Ranking data appear when we care about the underlying order that certain elements have. We might want to know which are the best horses after looking at several races (Gakis et al. 2018), which is the best candidate for a job after a series of interviewers talked to several candidates. More in line with cognitive science, we might want to know which are the best possible completions for a sentence or the best exemplars of a category.

One way to get a ranking of exemplars of a category, for example, is to present them to participants and ask them to order all (or a subset) of them (see Barsalou 1985).

A ranking simulation using pizza toppings

toppings <- corpora("foods/pizzaToppings")$pizzaToppings
N_toppings <- length(toppings)
toppings
 [1] "anchovies"        "artichoke"       
 [3] "bacon"            "breakfast bacon" 
 [5] "Canadian bacon"   "cheese"          
 [7] "chicken"          "chili peppers"   
 [9] "feta"             "garlic"          
[11] "green peppers"    "grilled onions"  
[13] "ground beef"      "ham"             
[15] "hot sauce"        "meatballs"       
[17] "mushrooms"        "olives"          
[19] "onions"           "pepperoni"       
[21] "pineapple"        "sausage"         
[23] "spinach"          "sun-dried tomato"
[25] "tomatoes"

Let’s say that we want to know the underlying order of pizza toppings. For the modeling, I’m going to assume that the toppings are ordered according to an underlying value, which also represents how likely it is for each topping to be the exemplar of their category.

To get a known ground truth for the ranking, I’m going to simulate an order of pizza toppings. I assign probabilities that sum up to one to the toppings by drawing a random sample from a Dirichlet distribution. The Dirichlet distribution is the generalization of the Beta distribution. It has a concentration parameter, usually , which is a vector as long as the probabilities we are sampling (25 here). When the vector is full of ones, the distribution is uniform: All probabilities are equally likely, so on average each one is ( here). By setting all the concentration parameters below one (namely ), I’m enforcing sparsity in the random values that I’m generating, that is, many probability values close to zero.

These is the true order that I’m assuming here:

# all elements of the vector are 0.2
alpha <- rep(.2, N_toppings)
# Generate one draw from a Dirichlet distribution
P_toppings <- c(rdirichlet(1, alpha)) %>%
  # Add names
  setNames(toppings) %>%
  # Sort from the best exemplar
  sort(decreasing = TRUE)
P_toppings %>%
  round(3)
 breakfast bacon          chicken             feta 
           0.294            0.241            0.087 
       anchovies sun-dried tomato           olives 
           0.087            0.077            0.057 
       pepperoni        artichoke           cheese 
           0.056            0.049            0.010 
  Canadian bacon            bacon              ham 
           0.008            0.008            0.006 
       meatballs    chili peppers           garlic 
           0.004            0.004            0.004 
     ground beef         tomatoes        hot sauce 
           0.003            0.003            0.002 
          onions          sausage        pineapple 
           0.000            0.000            0.000 
         spinach        mushrooms   grilled onions 
           0.000            0.000            0.000 
   green peppers 
           0.000
  • Given these values, if I were to ask a participant “What’s the most appropriate topping for a pizza?” I would assume that 29.37 percent of the time, I would get breakfast bacon.

Essentially, we expect something like this to be happening:

With representing the different probabilities for each topping. The probability mass function of the categorical distribution is absurdly simple: It’s just the probability of the outcome.

where breakfast bacon, chicken, feta, anchovies, sun-dried tomato, olives, pepperoni, artichoke, cheese, Canadian bacon, bacon, ham, meatballs, chili peppers, garlic, ground beef, tomatoes, hot sauce, onions, sausage, pineapple, spinach, mushrooms, grilled onions, green peppers.

We can simulate this with 100 participants as follows:

response <- rcat(100, P_toppings, names(P_toppings))

And this should match approximately P_toppings.

table(response)/100
response
 breakfast bacon          chicken             feta 
            0.26             0.19             0.16 
       anchovies sun-dried tomato           olives 
            0.07             0.15             0.06 
       pepperoni        artichoke           cheese 
            0.01             0.08             0.00 
  Canadian bacon            bacon              ham 
            0.02             0.00             0.00 
       meatballs    chili peppers           garlic 
            0.00             0.00             0.00 
     ground beef         tomatoes        hot sauce 
            0.00             0.00             0.00 
          onions          sausage        pineapple 
            0.00             0.00             0.00 
         spinach        mushrooms   grilled onions 
            0.00             0.00             0.00 
   green peppers 
            0.00

It seems that by only asking participants to give the best topping we could already deduce the underlying order…

True, but one motivation for considering ranking data is the amount of information that we gather with a list due to their combinatorial nature. If we ask participants to rank items, an answer consists in making a single selection out of possibilities. Ordering 7 pizza toppings, for example, constitutes making a single selection out of 5040 possibilities!

If we don’t relay on lists and there is sparcity, it requires a large number of participants until we get answers of low probability. (For example, we’ll need a very large number of participants until we hear something else but hammer as an exemplar of tools).

  • Now, what happens if we ask about the second most appropriate topping for a pizza?

Now we need to exclude the first topping that was given, and draw another sample from a categorical distribution. (We don’t allow the participant to repeat toppings, that is, to say that the best topping is pineapple and the second best is also pineapple). This means that now the probability of the topping already given is zero, and that we need to normalize our original probability values by dividing them by the new total probability (which will be lower than 1).

Here, the probability of getting the element (where ) is

where represents the probabilities of all the outcomes except of , which was the first one.

  • We can go on with the third best topping, where we need to normalize the remaining probabilities by dividing by the new sum of probabilities (e.g., we remove elements and ).

  • We can do this until we get to the last element, which will be drawn with probability 1.

And this is the exploded logit distribution.

This process can be simulated in R as follows:

rexploded <-  function(n, ranked = 3, prob, labels = NULL){
  # run n times
  lapply(1:n, function(nn){
    res <- rep(NA, ranked)
    if(!is.null(labels)){
      res <- factor(res, labels)
    } else {
      # if there are no labels, just 1,2,3,...
      labels <- seq_along(prob)
    }
    for(i in 1:ranked){
      # normalize the probability so that it sums to 1
      prob <- prob/sum(prob)
      res[i] <- rcat(1, prob = prob, labels = labels)
      # remove the choice from the set:
      prob[res[i]] <- 0
    }
    res
  })
}

If we would like to simulate 50 subjects creating a ranking of the best 7 toppings, we would do the following:

res <- rexploded(n = 50,
                 ranked = 7,
                 prob = P_toppings,
                 labels = names(P_toppings))
# subject 1:
res[[1]]
[1] sun-dried tomato artichoke        olives          
[4] breakfast bacon  chicken          pepperoni       
[7] anchovies       
25 Levels: breakfast bacon chicken feta ... green peppers

We have simulated ranking data of pizza toppings, can we recover the original probability values and “discover” the underlying order?

Fitting the exploded logistic distribution in Stan

To fit the model in Stan, I’m going to create a custom probability mass function that takes an array of integers, x, which represents a set of rankings, and a vector of probability values, theta, that sums up to one.

The logic of this function is that the probability mass function of a ranking can be written as a product of normalized categorical distributions (where the first one is just divided by 1).

For Stan, we need the log-PDF. In log-space, products become sums, and divisions differences, and the log of will be zero:

The following Stan code has a custom function that follows this logic but iterating over the rankings. In each iteration, it aggregates in the variable out the addends of the log probability mass function, and turns the probability of selecting again the already ranked element to zero. I save this code as "exploded.stan".

functions {
  real exploded_lpmf(array[] int x, vector Theta){
    real out = 0;
    vector[num_elements(Theta)] thetar = Theta;
    for(pos in x){
      out += log(thetar[pos]) - log(sum(thetar));
      thetar[pos] = 0;
    }
    return(out);
  }
}
data{
  int N_ranking; // total times the choices were ranked
  int N_ranked; // total choices ranked
  int N_options; // total options
  array[N_ranking, N_ranked] int res;
}
parameters {
  simplex[N_options] Theta;
}
model {
  target += dirichlet_lpdf(Theta | rep_vector(1, N_options));
  for(r in 1:N_ranking){
    target += exploded_lpmf(res[r] | Theta);
  }
}

The whole model includes the usual data declaration, the parameter Theta declared as a simplex (i.e., it sums to one), and a uniform Dirichlet prior for Theta. (I’m assuming that I don’t know how sparse the probabilities are).

Let’s see if I can recover the parameter values.

# Make the list of lists into a matrix
res_matrix <- t(sapply(res, as.numeric))
ldata <- list(
  res = res_matrix, 
  N_ranked = length(res[[1]]), 
  N_options = length(P_toppings), 
  N_ranking = length(res)
) 

m_expl <- cmdstan_model("exploded.stan")

f_exploded <- m_expl$sample(
  data = ldata,
  seed = 123,
  parallel_chains = 4,
  refresh = 0
)
f_exploded
 variable    mean  median   sd  mad      q5     q95 rhat
 lp__     -723.36 -723.05 3.64 3.64 -729.48 -718.06 1.00
 Theta[1]    0.29    0.29 0.04 0.04    0.23    0.35 1.00
 Theta[2]    0.27    0.27 0.03 0.03    0.21    0.33 1.00
 Theta[3]    0.08    0.08 0.01 0.01    0.06    0.11 1.00
 Theta[4]    0.09    0.08 0.01 0.01    0.06    0.11 1.00
 Theta[5]    0.07    0.07 0.01 0.01    0.05    0.09 1.00
 Theta[6]    0.05    0.04 0.01 0.01    0.03    0.06 1.00
 Theta[7]    0.04    0.04 0.01 0.01    0.03    0.06 1.00
 Theta[8]    0.05    0.05 0.01 0.01    0.03    0.06 1.00
 Theta[9]    0.01    0.01 0.00 0.00    0.00    0.02 1.00
 ess_bulk ess_tail
     1785     2746
     5569     2874
     6497     3121
     4994     3018
     6242     2855
     5037     2946
     5286     2654
     5002     3016
     4965     2937
     5453     3001

 # showing 10 of 26 rows (change via 'max_rows' argument or 'cmdstanr_max_rows' option)

I plot the posterior distributions of the probability values and the true probability values below.

mcmc_recover_hist(f_exploded$draws("Theta"),
                  P_toppings,
                  facet_args =
                    list(scales = "fixed", ncol = 3)) +
  theme(legend.position = "bottom")

It looks reasonable. However, if we really want to be sure that this is working, we should probably use simulation based calibration (Talts et al. 2018).

What is this good for?

This super simple example shows how to get an underlying ranking based on a set of responses from a number of subjects. It’s straightforward to adapt this model to data from participants ranking elements from different sets of the same size (e.g., 7 out of 25 toppings, 7 out of 25 tools). It’s a little less straightforward if the sets are of different sizes, e.g., rank 7 toppings out 25, but 7 tools out 50. This is just because Stan doesn’t allow ragged arrays. See this Stan Discourse thread for some tips on implementing the latter model.

Could this be used as a cognitive model of people’s rankings?

Maybe. And I enter here in the realm of half baked research, ideal for a blog post.

Lee, Steyvers, and Miller (2014) show the implementation of a cognitive model for rank order data from the latent knowledge of participants, which is based on Thurstonian models (Thurstone 1927, 1931) fitted with Bayesian methods in JAGS (Johnson and Kuhn 2013).

The exploded logit model seems to be closely related to the Thurstonian model. The Thurstonian model assumes that each participant assigns an underlying score to each item of a set, which is drawn from a true score with normally distributed error. The score determines the order that the participant gives. We can think about the exploded logit similarly. While I modeled the underlying ranking based on probability values, one could assume that each participant had their own score for each item (or pizza topping) , which is built as a common score together with some individual deviation :

If we assume that has a Gumbel distribution, then the probability of being ranked first out of N options is determined by a softmax function:

where is the vector of scores for all elements of the set.

And the probability of ordering second is:

and so forth.

These last equations are essentially the same categorical distributions that I used before, but the softmax function converts the unbounded scores into probabilities first. However, with the exploded logit, the error term goes away leading to a more tractable model. This is not the case for the Thurstonian model. The Thurstonian model is more complex, but at the same time we gain more flexibility. With the error term, the Thurstonian model can incorporate the reliability of the participants’ judgments and even correlations, which, as far as I know, can’t be included in the exploded logit model.

How to cite this post

BibTeX:

@misc{nicenboim2026asimplewaytomodelrankingswithstan,
  author = {Nicenboim, Bruno},
  title = {A simple way to model rankings with Stan},
  year = {2026},
  month = {January},
  url = {https://bruno.nicenboim.me/posts/posts/2026-01-07-a-simple-way-to-model-rankings-with-stan/},
  doi = {10.5281/zenodo.18171805}
}

APA:

Nicenboim, B. (2026, January 07). A simple way to model rankings with Stan. https://doi.org/10.5281/zenodo.18171805

Session info

sessionInfo()
R version 4.5.2 (2025-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Amsterdam
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
[1] bayesplot_1.15.0    cmdstanr_0.9.0     
[3] rcorpora_2.0.1      extraDistr_1.10.0.1
[5] ggplot2_4.0.2       tidytable_0.11.2   

loaded via a namespace (and not attached):
 [1] gtable_0.3.6         jsonlite_2.0.0      
 [3] dplyr_1.1.4          compiler_4.5.2      
 [5] tidyselect_1.2.1     Rcpp_1.1.1          
 [7] stringr_1.6.0        scales_1.4.0        
 [9] yaml_2.3.12          fastmap_1.2.0       
[11] plyr_1.8.9           R6_2.6.1            
[13] labeling_0.4.3       generics_0.1.4      
[15] distributional_0.6.0 knitr_1.51          
[17] backports_1.5.0      htmlwidgets_1.6.4   
[19] checkmate_2.3.3      tibble_3.3.1        
[21] pillar_1.11.1        RColorBrewer_1.1-3  
[23] posterior_1.6.1.9000 rlang_1.1.7         
[25] stringi_1.8.7        xfun_0.55           
[27] S7_0.2.1             otel_0.2.0          
[29] cli_3.6.5            withr_3.0.2         
[31] magrittr_2.0.4       ps_1.9.1            
[33] processx_3.8.6       digest_0.6.39       
[35] grid_4.5.2           lifecycle_1.0.5     
[37] vctrs_0.7.1          tensorA_0.36.2.1    
[39] evaluate_1.0.5       glue_1.8.0          
[41] data.table_1.18.2.1  farver_2.1.2        
[43] abind_1.4-8          reshape2_1.4.5      
[45] rmarkdown_2.30       matrixStats_1.5.0   
[47] tools_4.5.2          pkgconfig_2.0.3     
[49] htmltools_0.5.9

References

Barsalou, Lawrence W. 1985. “Ideals, Central Tendency, and Frequency of Instantiation as Determinants of Graded Structure in Categories.” Journal of Experimental Psychology: Learning, Memory, and Cognition 11 (4): 629.
Beggs, S, S Cardell, and J Hausman. 1981. “Assessing the Potential Demand for Electric Cars.” Journal of Econometrics 17 (1): 1–19. https://doi.org/https://doi.org/10.1016/0304-4076(81)90056-7.
Gakis, Konstantinos, Panos Pardalos, Chang-Hwan Choi, Jae-Hyeon Park, and Jiwun Yoon. 2018. “Simulation of a Probabilistic Model for Multi-Contestant Races.” Athens Journal of Sports 5 (2): 95–114.
Johnson, Timothy R., and Kristine M. Kuhn. 2013. “Bayesian Thurstonian Models for Ranking Data Using JAGS.” Behavior Research Methods 45 (3): 857–72. https://doi.org/10.3758/s13428-012-0300-3.
Lee, Michael D., Mark Steyvers, and Brent Miller. 2014. “A Cognitive Model for Aggregating People’s Rankings.” PLOS ONE 9 (5): e96431. https://doi.org/10.1371/journal.pone.0096431.
Luce, R. Duncan. 1959. Individual Choice Behavior : A Theoretical Analysis. Book. Wiley N.Y.
Plackett, R. L. 1975. “The Analysis of Permutations.” Journal of the Royal Statistical Society. Series C (Applied Statistics) 24 (2): 193–202. http://www.jstor.org/stable/2346567.
Talts, Sean, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. 2018. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration,” April. http://arxiv.org/abs/1804.06788.
Thurstone, Louis L. 1927. “A Law of Comparative Judgement.” Psychological Reviews 34: 273–86.
———. 1931. “Rank Order as a Psycho-Physical Method.” Journal of Experimental Psychology 14 (3): 187.

Footnotes

  1. This model is also called the rank ordered logit model (Beggs, Cardell, and Hausman 1981) or Plackett–Luce model due to Plackett (1975) and Luce (1959), but I liked the explosion part more.↩︎

Reuse

MIT
]]> Stan Bayesian R Citable https://bruno.nicenboim.me/posts/posts/2026-01-06-a-simple-way-to-model-rankings-with-stan/ Tue, 06 Jan 2026 23:00:00 GMT Welcome to my new Quarto site Bruno Nicenboim https://bruno.nicenboim.me/posts/posts/2026-01-06-welcome-to-my-new-quarto-site/

A fresh start

After years of using Hugo, I’ve finally made the switch to Quarto.

The main reason? Hugo wasn’t allowing me to update my website anymore!

The old posts from my Hugo site are gone (for now), but I’m looking forward to building up new content here. Stay tuned!

]]> Meta Quarto https://bruno.nicenboim.me/posts/posts/2026-01-06-welcome-to-my-new-quarto-site/ Mon, 05 Jan 2026 23:00:00 GMT