~/projects/byu-speeches/functions

What a paragraph does

A speech is a sequence of moves: it explains, it urges, it tells a story, it bears witness, it reports, it quotes. This study tags every paragraph with the move it makes, from a classifier trained on 464 paragraphs labelled by hand, and then asks how the mix has changed by era and how it differs by calling and by gender.

Supervised112,535 paragraphs464 hand labels61% cross-validated ~/byu-speeches

What stands out

The corpus is 29% doctrine, 19% exhortation and 24% story, with 9% quotation, 2% testimony and 8% policy.
Story went down by 6 points from the 1970s to the 2020s, exhortation up 4 and doctrine up 2.
Presiding Bishopric speakers tell the most stories (28% of their paragraphs); BYU faculty speakers explain the most doctrine (32%).
Gender is known only where the byline carries a title.
The classifier is 61% accurate on seven classes, strongest on story and quotation, weakest on testimony and other; the shares above carry that error. The shape study uses the same tags in sequence.

Loading the study…

Model card

Labels: 464 prose paragraphs of 40 to 180 words drawn at random, stratified by corpus and era, labelled by hand with one of seven functions: doctrine (explaining a teaching), exhortation (urging the listener), story (narrative, historical or personal), testimony (bearing witness), policy (programmes, reports, procedure), quotation (a paragraph that is itself a quoted passage) and other (greetings, transitions, prayers, logistics).
Model: TF-IDF over words and two-word phrases (6,000 features) plus eleven form features (share of quoted words, whether it opens with a quotation mark, past-tense density, first- and second-person rates, an imperative opening, questions, a testimony formula, amen, digits, length), in a logistic regression with balanced classes. Five-fold stratified cross-validation gives the accuracy; the final model is fitted on all 464 and applied to every paragraph of twelve words or more.
Aggregates: Shares are over paragraphs, not words. Calling is the calling held when speaking; gender is known only from the byline's title since 1971.
Known failure modes: 61% on seven classes means roughly one paragraph in three is mislabelled, mostly between doctrine and exhortation and between story and other. Rare classes (testimony, policy) are the least reliable. A paragraph that both tells a story and draws its lesson gets one tag.

Methods and data for the corpus as a whole.