Tag: pre-registration
All the writing and notes filed under "pre-registration" — 1 entry.
-
An NLP Audit of a Cultural Narrative: Pre-Registration, Cluster Permutation, and Placebo Calibration on Drake's Lyrics
A worked example of interrupted-time-series NLP on a small, clustered text corpus when the cultural narrative around it already carries strong priors. Pipeline: pre-registered surface metrics with HAC SEs and Bonferroni, sentence-embedding centroid permutation, TF-IDF discriminative interpretability, album-level cluster permutation, and a placebo time-cut calibration. Substantive result on Drake's full discography (2009–2026): the four pre-registered surface metrics are null; the song-level embedding signal (p ≈ 0.0002) does not survive album-level cluster permutation (p = 0.11 unconditional, 0.35 within rap); against 23 placebo cut dates the May 2024 result sits in the bottom quartile of L2 distances; the same pipeline on a peer artist (J. Cole) at the same cutoff also returns null (p = 0.76). Methodological takeaway: with low post-event sample sizes, song-level permutation can manufacture orders of magnitude of apparent evidence; cluster-level inference, placebo time-cut calibration, and a peer-artist control are what determine whether the signal is real.