~/projects/byu-speeches/year-predictor

Guess the year

Can the words of a speech tell you when it was given? A ridge regression over every word and two-word phrase learns to date a speech, and its mistakes are the interesting part: the speechs that read decades ahead of or behind their time, the vocabulary that marks each decade, and the speakers who sound like another era.

Supervised1,971 speechs30,000 featuresTry it in the browser ~/byu-speeches

What stands out

Language alone dates a speech to within about 8 years. On speechs by speakers the model never saw, the typical miss is 8.15 years (median 7.48), 66% land within a decade, and the model explains 59% of the variance in year.
Knowing the speaker barely helps. With random folds, where a speaker's other speechs are in the training set, the typical miss is 7.9 years against 8.15 without them. The dating comes from the era's language, not from recognising a voice.
Masking names, years and citations costs almost nothing. Without the masks the typical miss is 7.94 years, with them 8.15. The model is not reading dates off the page.
The 1960s are hardest and the 2000s easiest. Typical error is 13.7 years for speechs from 1960 to 1979 and 6 for 2000 to 2019; the ends of the range are pulled toward the middle, as a ridge model does.
Some speechs read as another decade. Neal A. Maxwell's “In This Time of Complexity and Challenge” (1980) reads like 2004, 14.2 years newer than its decade; Thomas S. Monson's “Principles from Prophets” (2009) reads like 1990, 16.5 years older.
The vocabulary of the present is possessive and personal. The words that most mark a recent speech include byu, focus, at byu, invite, atonement, god’s; the words that mark an early one include men, ought, teachers, i’d, tonight, spirit of.

Loading the study…

Model card

Data: 1,971 speechs of 300 words or more with a known year, from the BYU Speeches corpus. Text is the prose paragraphs only; headings, notes and bylines are excluded.
Masking: Before tokenising, titles followed by names (President X, Elder Y), names of the form First M. Last, four-digit years, magazine and report titles, and parenthetical bibliographic citations are removed. The same masks run in the browser demo.
Features: TF-IDF over words and two-word phrases: lowercase, letters and apostrophes only, present in at least five speechs, the 30,000 most frequent kept, sublinear term frequency, L2-normalised.
Model: Ridge regression on the year (alpha 3). Predictions shown are out-of-fold: five folds grouped by speaker for the headline numbers, five random folds for comparison. The full model fitted on everything supplies the coefficients and the browser demo.
Validation: Typical absolute error 8.15 years (median 7.48), R² 0.592, 66% within ten years, on speechs by held-out speakers. Residuals are reported net of each decade's mean so the shrinkage toward the middle of the range does not dominate the lists.
Known failure modes: The oldest and newest speechs are always guessed toward the middle. Citation style ("see", "italics added") and house vocabulary of an administration can date a speech as surely as doctrine. Short passages in the demo are unreliable; the model was trained on whole speechs.
Not for: Deciding what any speaker believed or intended. A speech that reads older than its date is using older language, nothing more.

Methods and data for the corpus as a whole.