About 1,250 word families cover 95% of what anyone says on the show.
I scraped the subtitle transcripts for all 38 episodes, 224,815 words, and
measured how much English a viewer needs to follow them. Published estimates
put 95% coverage of everyday spoken English at about 3,000 word families. This
show needs well under half that.
How many English word families does a viewer need to follow Love Island USA?
Data
Subtitle transcripts of all 38 episodes of season 4 (2022): 224,815 tokens, 7,856 word types
Method
Tokens lemmatized with WordNet as a stand-in for word families, ranked by frequency, coverage read off the cumulative share; textstat readability, a Zipf fit and MTLD per episode
Benchmark
About 3,000 word families for 95% of everyday spoken English (Adolphs and Schmitt 2003; Nation 2006), and about the same for TV dialogue (Webb and Rodgers 2009)
A line from episode 1. Under each word is its word-family rank in the
season's corpus ("is" counts as "be"). A viewer who knows the top 1,250 families
understands the first five words; the two in italics fall outside. Ranks computed
with the lemmatizer from scripts/analyze_vocab_level.py.
How much of the show would you follow?
Know the 1,249 most common word families and you follow 95.0% of the speech. About one word in 20 would be new to you.
Every point is the season's own coverage curve: the 224,815 tokens lemmatized and ranked
with the method in scripts/analyze_vocab_level.py, which reproduces the
250, 592, 1,249 and 2,554 in V4/tables/vocabulary_coverage.csv exactly.
Word families are approximated by WordNet lemmas.
61%
of all speech is the 100 most common words. The top 10 alone are 22.7%.
2,554
word families reach 98% coverage, still below the 3,000 that everyday English needs for 95%.
2.8
Flesch-Kincaid grade, with a Flesch Reading Ease of 90.2. 97.9% of
words are on the Dale-Chall list of 3,000 familiar words.
R² = 0.98
for the Zipf fit of log frequency on log rank, slope −1.45.
The question
How many words before you can follow it?
Intermediate learners are routinely told to watch native-speaker television,
and most of it is too hard for them. A corpus turns "is this show hard?" into a
number: rank every word family by frequency, add up their share of the speech,
and read off how many families it takes to reach 95% coverage.
The comparison figures come from the published literature: about 3,000 word
families for 95% of everyday spoken English (Adolphs and Schmitt 2003; Nation
2006), and about the same for TV dialogue in general (Webb and Rodgers 2009). This
show reaches 95% at 1,249 families, 90% at 592 and 98% at 2,554.
Reading level
Grade 2.8 by Flesch-Kincaid, about grade 7 by the stricter formulas
The formulas disagree by four grades. Flesch-Kincaid, which leans on sentence
length, scores the short spoken sentences as very easy, and every episode falls
between 2.3 and 3.7. SMOG and Dale-Chall weight long and unfamiliar words more,
and read higher. Only 2.1% of the words spoken fall outside the Dale-Chall list
of 3,000 familiar words.
Flesch-Kincaid2.81
textstat consensus3.0
SMOG6.91
Dale-Chall (approx.)7.28
US school grade by formula, bars on a 0 to 8 scale. From V4/tables/reading_level.csv (textstat).
The distribution
Thirty words are 40% of everything said
Word frequency follows Zipf's law closely: regressing log frequency on log rank
gives R² = 0.981 and a slope of −1.45, as in most natural
language. What stands out is how much the head carries. The top 10 words are
22.7% of all speech, the top 30 are 39.8%, the top 100 are 61.0%
and the top 1,000 are 91.1%. "Like" alone is 2.8%.
Rank, word and share of all 224,815 tokens, from V4/tables/top100_words.csv. Raw word forms, not families.
Every word type in the season, rank against count, both on log scales; the dashed line is the least-squares fit (slope −1.453, R² = 0.981, matching V4/tables/statistical_tests.csv). Counts recomputed from the corpus with the tokenizer in scripts/; the top 100 match top100_words.csv exactly.
Across the season
No episode-to-episode trend in variety or tone
Lexical diversity averages an MTLD of 59.96 per episode with no
trend across the season (Spearman ρ = −0.03, p = 0.87). Sentiment polarity from
TextBlob is positive in every episode, also with no trend (ρ = −0.15, p = 0.37).
Episode 38, the last, is the longest at 11,366 words.
Per-episode values from V4/tables/lexical_diversity.csv and sentiment_scores.csv; the dashed line is the season mean. Hover or use the arrow keys for a single episode.
Texture
Verbs and filler, few adjectives
Adjectives are only 8.4% of words, against 24.5% verbs and 13.6%
pronouns. The filler words are frequent enough to chart on their own: "like"
appears 28 times per 1,000 words, "know" 13, "yeah" 12 and "just" 12. This is
unscripted speech, and unscripted speakers lean on a small stock of words.
like28.2
know13.0
yeah12.5
just11.6
really6.5
right3.6
okay3.2
definitely2.5
Uses per 1,000 words, the top 8 of 24 tracked fillers, from V4/tables/filler_word_counts.csv. Every use is counted, including "like" as a verb, so these overstate filler use.
Limitations
One season, read from subtitles
This is one season of one show, and generalizing to reality TV as a genre would
take more seasons to compare against.
Subtitles drop what makes speech hard to follow: overlapping talk, accents and
speed of delivery. A show can be lexically easy and still hard to parse at full
speed, so the coverage number is a lower bound on difficulty.
Word families are approximated by WordNet lemmatization, which merges inflections
("making" to "make") but not derived forms. Readability formulas were built for
written prose and disagree on this dialogue, from grade 3 to about 7.
Method
Where the numbers come from
The repository ships the cleaned corpus, one .txt per episode plus a
combined CSV, so every number can be checked. The paper is written in Markdown,
converted to LaTeX by a script in the repo, and compiled to PDF.
Corpus
Subtitle transcripts of all 38 episodes of Love Island USA season 4 (2022), scraped with Selenium; 224,815 tokens, 7,856 word types
Coverage
Tokens lemmatized with WordNet to approximate word families, ranked by frequency, cumulative share read off at 80, 90, 95 and 98%
Diversity
TTR, root TTR and MTLD per episode; MTLD because it holds steady across episode lengths
NLTK part-of-speech tags, TextBlob sentiment, Pearson and Spearman tests for trends across the season
Built with
Python, pandas, NLTK, TextBlob, textstat, SciPy, matplotlib, seaborn; the charts on this page are hand-written SVG and HTML drawn from the CSVs in V4/tables/
