On Co-Authorship

One of my readers (Dan) asked if I could write a little bit about the process of co-writing a book. I’m happy to oblige!

First, how did Judea and I get in touch in the first place? A lot of people have asked this. The long version of the story is on my main web page, and it’s worth reading, but I’ll just give you the short version here. I had done some work for the American Statistical Association and someone thought, “Hmm, this is interesting, a guy who makes a living writing about math and statistics.” So they interviewed me for a short profile in their magazine for members. Judea had already been hunting for someone to collaborate with him on a popular book, and he noticed this article. He e-mailed me to ask if I might be interested in this. I already knew very well who he was (see the long version for more details), so I instantly said, “Yes.” No question that his e-mail changed my life, or at least my career.

The next step is getting an agent, writing a book proposal, and finding a publisher who is interested in your proposal. Again this part is a long story, with a lot of ups and downs. We both learned a lot through this process. Judea, who has spent his whole life in academia, was very much a naive innocent about the publishing business, and he was amazed by many things in the process. “You will auction the rights to the book?” he asked our agent. “Like selling fish?” Yes, like selling fish. Or he would ask me, “What do we need an editor for?”

I’ve been in the publishing business for a longer time, written two popular books, but there were still some things that were new to me. This was my first time working with an agent, and I finally got to see how “the other half” (i.e., authors who actually make some money writing books) lives. Suffice to say, our agent was worth every penny that he’s earned. The Merriam-Webster definition of an agent should read as follows. “agent — 1. A person who makes problems go away.” That describes John Brockman to a tee.

Once you’ve got a contract, now the process of writing the book begins. I’m sure this is the main thing that Dan was asking about.

I’ll talk about co-writing a book like this one, where the two people bring very different skill sets to the book: one is the “expert,” we’ll say, and one is the “writer,” whose job it is to put the expert’s story into words. It’s a common arrangement, and there are three essential ingredients to make it work.

1. Both people must have complete respect for each other. Very often this is not the case; the expert looks at the writer as someone who is just hired to do a job. Maybe the expert hands the writer a bunch of papers and scribblings and says, “Here, turn this into a book.” I would hate to work with someone who had that attitude. I talked with a friend who co-wrote a book with someone who treated her that way, and she says that on the day the book came out, she and her friends got a pinata that looked like her co-author and started whaling away at it. “Very cathartic,” she said.

Suffice to say that Judea was as far from that as possible. No prima donna, he understood what I was there for and how I could help him. He insisted that I should be open and honest in saying what I liked or didn’t like. He wanted the book to have music and poetry to it — which was such a breath of fresh air compared to other scientists I’ve worked with, who only want the facts and don’t understand music or poetry.

2. As a writer, you need to know your co-author. I tried to learn Judea’s sense of humor (which is really sly) and what he was passionate about. I wanted to do my best to incorporate his voice in the book. As far as I’m concerned, more Judea is always better. I took notes on every phone conversation and would try to incorporate his phrases into the text, even if he didn’t ask me to.

I think that one big change that happened as we worked on this book was that I did not do as much research into other approaches to causation as I had planned to. Judea has some opponents, and for the most part I never interviewed them. If I were writing the book on my own, I would of course have interviewed all sides — that’s what a journalist does. But I gradually realized that my job here was to present Judea’s view of causality. Here’s an analogy: If you pick up a book by David Hume, you don’t expect him or want him to present John Stuart Mill’s point of view, or Immanuel Kant’s. You want Hume. I realized that readers of this book would want Judea, not his opponents. This brings me to …

3. As a writer, you have to absolutely believe the expert’s message. This is tricky, because it’s not your own message, but you’ve got to believe it as much as if it were. I could never have done this project if I weren’t convinced, as a mathematician, that Judea’s approach is absolutely sound and better than his opponents. (Okay, I have to be careful here. In a mathematical sense, his approach is equivalent. But it’s better in practice because the language of causal diagrams makes the assumptions transparent and explicit. The other methods sweep the assumptions under the rug — you’re making them, but you don’t realize it.)

Not only that, I absolutely agree that causality has been absurdly ignored by “mainstream” science. I’ve personally taught a Statistics 101-type course, and taught students the “correlation is not causation” mantra, and I know for a fact that I never prepared students to think about what causation is or, more importantly, how it behaves.

Now, how did the writing actually work? Well, I’m not going to do the Lennon/McCartney thing and say, “This song was a Lennon song. That song was a McCartney song.” Every word in the book belongs jointly to both of us. However, here’s what I can say. Somewhat to my surprise, Judea allowed me to write every first draft of every chapter. In fact, that’s the way he wanted it. This means that the basic architecture of the book, and the basic architecture of every chapter, is more or less the way that I laid it out. This was an awesome responsibility! Of course, before getting started on a chapter I consulted his books, his articles, and talked with him personally. But he trusted me enough to let me shape the book. See point 1 above. I think that this is one reason the book reads way better than a traditional “as told to” book.

Within each chapter, we bounced the drafts back and forth many times. I have no idea how many thousands of e-mails we’ve written and how many hundreds of phone calls we’ve had. The “first draft” that we sent to our editor was really more like the tenth draft. In this way, every chapter became very deeply imbued with his knowledge, his outlook, and his voice. That is why it reads so seamlessly. For me as a writer, it was an amazing luxury to have Judea as a safety net. I could write stuff in the first draft that I wasn’t sure about, confident that he would correct me and steer me in the right direction.

When we first talked with our agent, he warned us about “collaborator’s disease.” I’m happy to say that this disease never hit us. Not that there weren’t some frustrating moments. But they were mostly the result of our interaction with others. The whole process of being edited was so foreign to him, because as an academic you are pretty much in control of your own words. There are editors, who accept or reject your article, but they have only a small say over what actually goes into your article. In academia, a much more important role is that of the referee. But here was another surprise for Judea — when you are writing a book for the public, there are no referees! In a way the editor is like a referee, but his role is completely different: instead of judging whether you’ve written is accurate, he is judging whether it’s written in a way that the ordinary reader can understand.

So working with an editor was challenging for Judea, but we got through it and I’m convinced that our editor did improve the book. Here my role was sometimes that of a sounding board or a counselor. Judea is sometimes combative — the result of a lifetime of battling with referees — and he had to see that the editor is there to help us tell the story better, not to get in our way.

The last part of writing a book is seemingly the easiest part. After you turn the final draft in, there are still about six months to go. It seems as if you’re just waiting, but actually there is a lot to do. This is when the publisher’s marketing and publicity effort starts ramping up, and you have to talk with the publicists about what the book is about, who should get free copies to review, etc. They asked us to compose a list of talking points. After I was done, I almost wished that we had written the talking points before writing the book! It would have been so much easier.

Another thing that seems routine, but really isn’t, is copy-editing and proofreading. It’s especially challenging in a book like this, which has tons of diagrams and even an equation or two. I don’t want to say anything bad about our publisher because in the end they did a beautiful job, but it was amazing to see how mistakes are just like weeds. You pull one up, and suddenly you notice another one. It’s exhausting.

And finally, one day, your book arrives in the mail. You look at it in wonder, just like a newborn baby, and you say, “But it’s so small!” Is this really what I’ve worked on for three years? But behind every word and every picture there is a story, so that really there is a whole journey stored up in that one little book. And hopefully, people read it and take a little bit of the journey with you.

I was going to stop there, but I realized that there was one more thing I wanted to mention. We had two illustrators working with us on this book: Maayan Harel and Dakota Harr. This was also Judea’s idea: he thought that illustrations would liven up the book and make it easier for readers to get through. I think it’s quite non-traditional for a “serious” non-fiction book like ours to have artist-drawn illustrations. I’ll be curious to see whether any of our reviewers will comment on this, and whether they will think they added to it or were just an embellishment.

Speaking personally, I really enjoyed Maayan’s and Dakota’s work, and it was exciting to see how they turned our rough ideas into actual drawings. I’ll show you one of my favorites, the frontispiece for Chapter 8, drawn by Maayan Harel.

This is the frontispiece for a chapter on counterfactuals. We wrote about ways that data and causal models can be used to answer questions like, “What would have happened if I had not taken that aspirin for my headache?” or “Would my grandfather have lived longer if he had not smoked?”

Implicit in the notion of counterfactual is that you have an action that you took, and a second action that you did not take, whose consequences you want to understand — and you are allowed the wisdom of hindsight. Suddenly I realized that this is exactly the situation that Robert Frost is talking about in “The Road Not Taken,” one of the most famous poems in the English language. For the first time I noticed how Frost spells out exactly what the issue is: “I could not travel both, and be one traveler.” But that is what we are doing with causality, and causal models! We are allowing the traveler to experience both roads, at least in his mind.

I asked Maayan if she could send me an illustration of “The Road Not Taken,” and she sent me this. I was absolutely swept away. It does what an illustration should do: it puts us right there, with the fork in the road before our eyes, and it evokes the wistfulness that the traveler feels. This is a feeling in the gut that every one of us has experienced, at one time or another. Her illustration tells us better than any words I could have written that counterfactuals are not just a word on the page, they are part of the meaning of being human. Counterfactuals are our lives. So yes, we’re talking about something pretty important in this chapter.

That is one of the joys of writing a book — being part of a team, and occasionally having a moment when everything falls into place better than you could have imagined.