The Story of The Book of Why

Because The Book of Why has two authors, it has two origin stories.

I don’t know exactly when the idea of writing a popular book came to Judea, but he had the very good sense to realize that he needed a co-author. He is actually an excellent writer, with more than 400 academic articles to his credit, and he has a wonderful sense of humor. But like a good sailor, he realized that on such a daunting journey he would need someone else to share the helm. So he was looking for a co-author for at least a year or two before he found me.

In 2013, I had done some writing for the American Statistical Association in connection with the International Year of Statistics, and someone at the ASA had the bright idea of writing a little article about me in their newsletter, Amstat News. I barely even noticed it, and yet that article changed my life. Judea read it and said, “Hm. A former mathematician who writes about math and science for the public. I wonder if he might be the right person to work on my book?” Then he e-mailed me to ask if I would be interested.

Now for my side of the story, we have to go back 17 years to when I was first starting out as a free-lancer. The very first assignment I ever received was to write an article about Microsoft’s then-new research laboratory for Science magazine. When I went to Redmond, they were all agog over Bayesian networks. They used them in spam filters and in Microsoft Office Assistant — that annoying little paperclip in the corner of the screen in Windows 95 and 97, which all the users hated. Perhaps not the greatest of success stories, but anyway, Bayesian networks were the hottest thing in artificial intelligence and I heard over and over about their inventor, Judea Pearl.

So after I got done with my piece about Microsoft Research, I thought I would write an article about Bayesian networks and interview the guru himself. But when I started reading about him, I found out a stunning fact: Judea Pearl didn’t work on Bayesian networks any more! He was working on this new thing, causality, and as I read his thoughts my mind was blown in several different ways at once. First, this was amazing, fundamental philosophical stuff, going back to people like Hume and Galileo and Aristotle. But what did it mean? What was it good for? I couldn’t really tell, and I was pretty sure that as a just-starting-out freelance writer I couldn’t go to some editor and say, “Hey, I want to write an article on causality. You know, like, how things relate to other things. Yeah.” So I gave up on the story. I never even contacted Judea. For seventeen years, it was for me The Story That Got Away.

So when that e-mail arrived in my inbox in April 2014, I knew exactly who Judea Pearl was. For me, it was as if Zeus had reached down from his mountaintop, tapped me on the shoulder, and said, “Would you like to ride in my chariot? Perhaps throw a thunderbolt or two?” It was the easiest “Yes” in my writing career.