~/projects/general-conference/authorship
Whose talk is this
Given a talk with the name removed, can a model say who gave it? Among the {s.speakers} speakers with {s.minTalks} or more talks it can, most of the time, and where it fails it fails in pairs: the speakers it confuses are the speakers who sound alike.
What stands out
- The right speaker 74% of the time, in the top five 91%. Chance is 0.9%. Words and phrases plus a stylometric vector, five-fold cross-validation.
- Style alone gets 70%. With no content words at all, function words, sentence shape and punctuation identify the speaker almost as well as the full vocabulary does.
- Wilford G. Edling is the most recognisable (100% of 21 talks); A. Theodore Tuttle the least (15% of 34).
- Brigham Young and Heber C. Kimball are mistaken for each other most, 16 times, 3% of their talks between them; then Heber J. Grant and Joseph F. Smith.
- The calling is harder than the speaker. With every speaker held out, the text names the calling 42% of the time; Quorum of the Twelve is the most recognisable calling (49%).
Loading the study…
Model card
- Data
- 7,532 talks of 300 words or more by the 117 speakers with 20 or more.
- Features and model
- Masked TF-IDF over words and two-word phrases (names, years, magazines and citations removed; 40,000 features) concatenated with the standardised stylometric vector of the style study, scaled down by 0.3; a multinomial logistic regression. The style-only model uses the stylometric vector alone.
- Validation
- Five stratified folds by talk for speaker attribution (the question is whether a new talk by a known speaker can be attributed). For calling, five folds grouped by speaker so the model cannot recognise the person.
- Known failure modes
- Speakers of one era share vocabulary and transcription conventions, so part of what the model learns is period, not person; the style-only score benefits from the same. Speakers with few talks are harder, and a long-serving speaker's early and late talks can read as two people.
Methods and data for the corpus as a whole.