~/projects/general-conference/year-predictor
Guess the year
Can the words of a talk tell you when it was given? A ridge regression over every word and two-word phrase learns to date a talk, and its mistakes are the interesting part: the talks that read decades ahead of or behind their time, the vocabulary that marks each decade, and the speakers who sound like another era.
What stands out
- Language alone dates a talk to within about 11 years. On talks by speakers the model never saw, the typical miss is 10.78 years (median 8.55), 57% land within a decade, and the model explains 91% of the variance in year.
- Knowing the speaker barely helps. With random folds, where a speaker's other talks are in the training set, the typical miss is 9.71 years against 10.78 without them. The dating comes from the era's language, not from recognising a voice.
- Masking names, years and citations costs almost nothing. Without the masks the typical miss is 10.7 years, with them 10.78. The model is not reading dates off the page.
- The 1840s are hardest and the 1940s easiest. Typical error is 22.7 years for talks from 1840 to 1859 and 7.1 for 1940 to 1959; the ends of the range are pulled toward the middle, as a ridge model does.
- Some talks read as another decade. Brigham Young's “An Incident of Nauvoo” (1871) reads like 1938, 57.2 years newer than its decade; Joseph Fielding Smith's “The Kingdom of God is Eternal” (1968) reads like 1901, 62.9 years older.
- The vocabulary of the present is possessive and personal. The words that most mark a recent talk include god’s, lord’s, the lord’s, jesus christ, christ amen, all of; the words that mark an early one include re, this people, the people, brother, we shall, brethren and.
Loading the study…
Model card
- Data
- 10,420 talks of 300 words or more with a known year, from the General Conference corpus. Text is the prose paragraphs only; headings, notes and bylines are excluded.
- Masking
- Before tokenising, titles followed by names (President X, Elder Y), names of the form First M. Last, four-digit years, magazine and report titles, and parenthetical bibliographic citations are removed. The same masks run in the browser demo.
- Features
- TF-IDF over words and two-word phrases: lowercase, letters and apostrophes only, present in at least five talks, the 30,000 most frequent kept, sublinear term frequency, L2-normalised.
- Model
- Ridge regression on the year (alpha 3). Predictions shown are out-of-fold: five folds grouped by speaker for the headline numbers, five random folds for comparison. The full model fitted on everything supplies the coefficients and the browser demo.
- Validation
- Typical absolute error 10.78 years (median 8.55), R² 0.91, 57% within ten years, on talks by held-out speakers. Residuals are reported net of each decade's mean so the shrinkage toward the middle of the range does not dominate the lists.
- Known failure modes
- The oldest and newest talks are always guessed toward the middle. Citation style ("see", "italics added") and house vocabulary of an administration can date a talk as surely as doctrine. Short passages in the demo are unreliable; the model was trained on whole talks.
- Not for
- Deciding what any speaker believed or intended. A talk that reads older than its date is using older language, nothing more.
Methods and data for the corpus as a whole.