~/projects/general-conference/canonization
Which talks last
Most talks are given once and never quoted again. A few become the talks everyone quotes. This study asks whether that fate is visible at delivery: from the text and what is known on the day, can a model say which talks later speakers will carry forward? The answer is a measured yes, with the fame of the speaker doing much of the work and the talk itself doing the rest.
What stands out
- Text and metadata reach an AUC of 0.766; text alone 0.748. The gap is the speaker's calling and the era; the rest is in the words. Base rate: 37% of talks to 2006 are quoted by a later talk within 20 years.
- First Presidency raises the chance most, Sunday School lowers it most. Among the words, prophet joseph, hymns, song, joseph, night raise it; organization, his children, the land, school lower it.
- The model is better in some eras than others. 1900s: AUC 0.767; 1920s: AUC 0.773; 1940s: AUC 0.725; 1960s: AUC 0.7; 1980s: AUC 0.756; 2000s: AUC 0.712.
- Of talks since 2007, the model rates Thomas S. Monson's “Come, All Ye Sons of God” (2013) most likely to last at 95%, and it has already been quoted.
- The sleepers are the interesting failures. Christian D. Fjeldsted's “” (1905) was given a 3% chance and has been quoted by 7 later talks.
Loading the study…
Model card
- Label
- A talk is "quoted" if any later conference talk shares a passage of fifteen words or more with it within 20 years, from the reuse study (scripture, rosters and set phrases excluded). Only talks up to 2006 have a full window; 7,572 talks of 300 words or more from 1900.
- Features
- Masked TF-IDF over words and two-word phrases (30,000 features), the stylometric vector of the style study (scaled by 0.3), and metadata known on the day: the calling held when speaking, priesthood or women's session, log of word count, year, and citations per thousand words. The text-only model drops the metadata.
- Model and validation
- Logistic regression with balanced class weights, five folds grouped by speaker so a famous speaker's other talks cannot leak. AUC, average precision, ten-bin calibration and AUC by twenty-year era are all out of fold. The full model fitted on every labelled talk gives the weights and scores the recent talks.
- Known failure modes
- The label is quotation in conference only, not reprinting in manuals or curricula, which are not in the data. Famous talks are quoted because of who gave them, and the calling feature captures part of that; the text-only model is the fairer measure of the talk. Recent talks have had less time to be quoted and their scores are predictions, not outcomes.
Methods and data for the corpus as a whole.