The Best Christian Book of All Time: By the Numbers

If we’re going to take the sports angle to this tournament all the way, then at some point we need a “Moneyball” type to come in with formulas and spreadsheets to turn the whole thing on its head. Turns out that time is now, and that guy is me. I like my sports with a heavy dose of “advanced metrics” so I figured I’d try to bring a little of that flavor to our book tournament.

Let’s start by taking a closer look at what we mean by “best” book. In this format, what we really mean is “most head-to-head wins in a single elimination tournament.” Confessions  was named the best book by that standard, but it was really only picked over 6 of the 63 other books. We can’t really say for sure that one or more of those 57 other books wouldn’t have beat  Confessions if they had gone head to head. We assume this sort of transitivity (if A beats B and B beats C, A will beat C) when staging single elimination tournaments, but we know from experience that it doesn’t always hold. The most thorough measure would be to play out all 4,032 head-to-head matchups and see which book accumulates the most wins, but that’s a bit impractical. On the other hand, a biased sample of only 63 / 4,032 matchups may not be representative.

So then, what else can we learn from the tournament data we do have? For starters, we can look at the  total number of votes accumulated during the tournament. Ranking by that metric, our top 5 are:

Title	Votes
Mere Christianity	402
The Cost of Discipleship	347
Confessions	342
The Lord of the Rings	309
The City of God	236

All of a sudden, our victor is in third place — and it’s not even close.  Mere Christianity got 15% more votes than the next closest book, and 18% more than  Confessions. In fact, it is so far ahead that even if we don’t count the votes  Mere Christianity received in the last round (because those were “losing” votes), it would still wind up ahead by 17 points. This result is consistent with Mike’s observations that  Mere Christianity was the most nominated book and seems to be the most popular anecdotally.

Now, you might be thinking that counting total votes is biased towards books that made it late into the tournament, and you’d be right. We can adjust for that by looking at  average votes per matchup. Using that metric, we get the following top 5:

Title	Average Votes
The Cost of Discipleship	69.4
Mere Christianity	67
The Lord of the Rings	61.8
The City of God	59
Les Misérables	58.67

On this list,  Confessions finishes 6th with 57 votes on average. The leaders here were likely helped by big first round victories; 4 of the 5 were the top vote-getters in the first round. This reminds us that votes are context dependent; they reflect more than just preference for a given book. In a given head-to-head contest, a voter may have been voting  against  a book rather than for one. We might then try to adjust for the strength of the opponent, to account for the different paths that each book took through the tournament. But here we run up against the limitations of the available data. We don’t have a good independent measure of which books are better. And we can’t use the tournament results themselves, because half of the books were eliminated in the first round; that rules out the possibility of adjusting the result of a given matchup by considering each book’s performance in all other matchups except that one.

However, we could try a crude adjustment by looking at genre. It turns out that the four genre regions differed significantly in vote counts. Theology & Apologetics, the region  Mere Christianity came from, tallied 1,453 votes, while  Memoirs, Devotionals & Spirituality only brought in 1,173. (Fiction & Poetry had 1,363 and  Christian Life & Discipleship had 1,236.) If we scale the votes so that each genre got the same total,  Confessions becomes the top vote-getter and moves into second place in the average vote ranking.

Title	Total Adjusted Votes	Average Adjusted Votes
The Cost of Discipleship	328	65.6
Confessions	342	57
Mere Christianity	321	53.5
The Lord of the Rings	263	52.6
Les Misérables	150	50

But is that a fair adjustment? Did the Theology & Apologetics region get more votes because its books are more popular on average? Does that mean we should discount its votes, or does that mean it was actually a tougher bracket to get through? Do fewer votes mean tougher matchups, leading to more people abstaining because they considered the opponents equal? Or were there more abstentions because one of the books was unfamiliar and voters felt uncomfortable voting against a book they didn’t know?

At this point, perhaps you are wondering  how we come to any kind of conclusion if we can make the numbers say whatever we want. I would propose an alternate perspective. What attracts me to the advanced metrics school, and to statistics in general, is that expressing things numerically or mathematically really forces you to think clearly about what question you are asking and what the answer really means. In this little exercise, we’ve asked a few different questions; not surprisingly, they have different answers. But those answers all come from the data; I can’t make The Purpose Driven Life the answer to any of those questions if I just tweak the numbers the right way.

So, whether you agree or disagree with the results of this tournament, whether you liked my alternative lists or not, I hope this helps you to reflect more precisely on whatever questions you are asking.