Bayesian Cognitive Assessment Scoring for Digital Therapeutics

Honest scoring for ordinal data

We started with the scoring problem. Clinical assessments produce ordered categories, so we modeled them as what they are — ordinal data with latent cognitive constructs underneath. This means estimating where the boundaries fall between response categories and what underlying cognitive ability each pattern of responses implies. The approach respects the structure of the data rather than forcing it into a shape that's convenient but wrong.

Tracking outcomes over time

For tracking outcomes over time, we built models that account for the natural nesting in clinical trial data — repeated measurements within patients, variation across assessors, and the messiness of real treatment schedules where people miss visits and drop out. Prior checks against clinical knowledge kept the models grounded before they ever saw data.

Making intractable models tractable

The computational challenge was the most interesting piece. Some cognitive process models that Akili needed to evaluate don't have tractable likelihood functions — you can simulate data from them, but you can't efficiently calculate the probability of observed data. We used neural networks trained to approximate those likelihoods, which made inference on these models fast enough to be practical across clinical trial scales.