What Makes a Best-Selling Novel?
Skip to content
A Machine Learning Approach
In 2013, Ashok et al. answered this question basing on the writing style, with 61–84% accuracy. This post, on the other hand, examines plot themes in best sellers. Note that my model can hardly predict the commercial success of a novel from its plot. That would be quite a surprising feat, making reviewers obsolete. My goal was more modest: finding statistically profitable topics to write about.
Using PetScan and Wikipedia’s page export, I downloaded 25,359 Wikipedia articles belonging to Category:Novels by year. From each article, I extracted the section named Plot , Plot summary , Synopsis , etc. if present and, stripped of MediaWiki markup, saved it into an SQLite database along with the title of the novel, its year of publication, and a Boolean that indicates if it ever topped the New York Times Fiction Best Seller list:
SELECT title, year, was_bestseller, length(plot) FROM Novels<br>ORDER BY random() LIMIT 5;<br>Sharpe's Havoc | 2003 | 0 | 2759<br>The Rescue (Sparks novel) | 2000 | 1 |<br>Slayers | 1989 | 0 |<br>The Warden | 1855 | 0 | 2793<br>The Fourth Protocol | 1984 | 1 | 5666
SELECT count(*) FROM Novels<br>WHERE plot IS NOT NULL;<br>17744
SELECT count(*) FROM Novels<br>WHERE plot IS NOT NULL AND was_bestseller;<br>398
SELECT min(year) FROM Novels -- The year of publication.<br>WHERE was_bestseller; -- The NYT list starts in 1942.<br>1941<br>To obtain easy to interpret results, I have built a logistic regression model on top of the TF–IDF transformation of articles processed by the Porter stemmer. The parameters have default values. In particular, the logistic regression uses L2 regularization so all lowercase words that are not stopwords appear in the model.
import nltk<br>from nltk.corpus import stopwords<br>from nltk.stem import porter<br>from sklearn import cross_validation<br>from sklearn import linear_model<br>from sklearn import pipeline<br>from sklearn.feature_extraction import text
def Tokenize(<br>text,<br>stemmer=porter.PorterStemmer(),<br>uppercase=set(string.uppercase),<br>stop_set=set(stopwords.words('english')),<br>punctuation_re = re.compile(<br>ur'[’“”…–—!"#$%&\'()*+,\-./:;?@\[\\\]^_`{|}~]',<br>re.UNICODE)):<br>text = punctuation_re.sub(' ', text)<br>tokens = nltk.word_tokenize(text)<br>return [stemmer.stem(x) for x in tokens<br>if x.lower() not in stop_set and x[0] not in uppercase]
X = []<br>y = []<br>connection = sqlite3.connect('novels.sqlite')<br>for row in connection.cursor().execute(<br>"""SELECT plot, was_bestseller FROM Novels<br>WHERE year >= 1941 AND plot IS NOT NULL"""):<br>X.append(row[0])<br>y.append(row[1])<br>connection.close()<br>X_train, X_test, y_train, y_test = (<br>cross_validation.train_test_split(X, y, test_size=0.3))<br>model = pipeline.Pipeline(<br>[('tfidf', text.TfidfVectorizer(<br>lowercase=False, tokenizer=Tokenize)),<br>('logistic', linear_model.LogisticRegression())])<br>model.fit(X_train, y_train)<br>The model can return the probability of being a best seller for any novel b with a plot summary:
logit(b) = −4.6 + 2.5 tfidf(lawyer, b) + 2.4 tfidf(kill, b) + ⋯ − 1.5 tfidf(planet, b)
Pr(was_bestseller(b)|plot(b)) = elogit(b) / (1 + elogit(b))
To put these coefficients in context, tfidf(lawyer, The Firm) ≈ 0.06. As it happens, the model returns logit(b) > 0, that is Pr(was_bestseller(b)|plot(b)) > 1/2 for no novel b from the train or test set. The highest probability, 0.39, is predicted for Cross Fire, indeed a best seller in December 2010. Only if I disable the normalization in TF–IDF or weaken the regularization in the logistic regression, I can overfit the model to the train set while for the test set both its precision and recall would be at most 20%. But, like I wrote in the introduction, this is not the point of this exercise. Let us look at the words with high absolute value of coefficients.
Apparently, it pays off to write legal thrillers: lawyer +2.5, case +2.4, law +1.5, client +1.3, jury +1.3, trial +1.3, attorney +1.0, suspect +1.0, judge +0.9, convict +0.8;
kill +2.4, murder +1.8, terrorist +1.2, shoot +1.1, body +1.1, die +1.0, serial +0.9, attack +0.9, assassin +0.8, kidnap +0.8, killer +0.8.
Political thrillers are not bad either: agent +1.4, politics +1.4, president +1.3, defector +1.2.
Business may be involved: firm +1.3, company +1.3, career +1.1, million +1.0, success +1.0, business +0.9, money +0.9.
Finally, the characters should have families: husband +1.4, family +1.3, house +1.2, couple +1.2, daughter +1.2, baby +1.1, wife +1.0, father +1.0, child +0.9, birth +0.8, pregnant +0.8, and use a car +1.5 and a phone +0.8.
The genres to avoid for prospective best-selling authors?
Sci-fi: planet −1.5, human −1.0, space −0.7, star −0.4, robot −0.3, orbit −0.3.
Children’s literature: boy −1.3, school −1.0, young −0.8, girl −0.8, youth −0.4, teacher −0.4, aunt −0.4, grow −0.4.
Geography and travels: village −1.0, city −1.0, ship −0.8, way −0.7, go −0.7, land −0.6, adventure −0.6, colony −0.5, native −0.5, follow −0.5, mountain −0.5, crew −0.5, forest −0.5, travel −0.5, inhabit...