For presidential historian Jon Meacham, a subtle but significant shift occurred between the publication of an essay in 2017 and the book that it inspired soon after.

The essay in Time magazine was titled “American Hate, a History”  and he wrote it in the aftermath of the deadly gathering of white supremacists in Charlottesville, Virginia. Meacham’s 2018 book — and subsequent HBO documentary arriving Tuesday, Oct. 27 — moved from those shadows into light with the title “The Soul of America.”

“As the weeks after Charlottesville unfolded, it stayed with me,” Meacham says from his home in Nashville, the day after he attended the final debate between President Donald Trump and Democratic presidential nominee Joe Biden in that city.

“I was trying to figure out where these themes (of hate) recurred,” he says. “And at what point did we see them ebb, and at what point do we see them flow.”

A month later, the perspective he’d had in his Time essay shifted focus, Meacham says.

“I started thinking, It’s not hate versus hope so much as it’s fear versus hope,” he says. “Landing on the idea of fear was critical to the book, and thus to the documentary. Because you can be fearful without hating, right?

“It widened the aperture, this idea of where were we broadly fearful that then would manifest itself in acts of hatred. It was an important shift, I think, because it is more representative: You can be fearful without being hateful, actively hateful.”

“The Soul of America,” both book and film, explores that idea across the sweep of American history. In a time when many feel the United States has never faced such dangerous, divisive days, and democracy itself hangs by a thread, Meacham looks to the past for lessons drawn from similarly perilous points in the past.

“The push and pull of the constitutional experiment has been between hope and fear, selfishness and sacrifice, personal ambition and the common good,” he says.

“What I was pushing against was the tendency of the present to feel that our problems are insuperable and unique,” Meacham says. “And on the other side, pushing against people who tend to be more reactive than redemptive.”

Making the movie

In the HBO documentary, Meacham serves as our guide to the case he makes in his book. For him, it was an easy yes when director KD Davison and Kunhardt Films made their pitch to adapt “The Soul of America.”

“I believe the argument in this book as firmly and as deeply as I believe anything,” says Meacham, a former editor of Newsweek magazine and the Pulitzer Prize-winning author of the biography “American Lion: Andrew Jackson in the White House.” “So any set of eyeballs, any set of ears that I can get to engage the argument I’m delighted about.”

Davison was in the middle of making a documentary on mass incarceration and its roots in slavery and Jim Crow-era racism when she read Meacham’s book in a three-day burst and saw in it the possibilities for the film.

“I just kind of devoured it,” she says. “Really, it’s the most fun as a filmmaker, I think, when a thing begins to come to life in your mind. And Jon is such a cinematic writer, it really became clear to me how we can make this a film.”

Because Meacham’s book covers a wide swath of American history — including the Civil War and Reconstruction, periods already been amply documented by filmmakers such as Ken Burns — Davison says she looked for periods about which Meacham wrote that might resonate especially strong in the tempestuous politics and society of recent years.

“In terms of honing the story, we were thinking, Let’s really focus on the 20th century,” she says. “Let’s begin in places where we have archival materials to support it, visual materials. Let’s also connect it to stuff that really is reflective of what we’re experiencing.”

The documentary then looks at the battles over issues such as women’s suffrage and isolationism before World War II, the internment of Japanese Americans, McCarthyism, and the Civil Rights Movement — antecedents of many of the topics that have riven the nation today.

“These are the moments we celebrate,” Meacham says of the righteous victories of the past. “These are the monuments. We don’t build monuments to people who close doors. We celebrate, we commemorate people who liberate.”

With Meacham filmed in class at Vanderbilt University where he holds an endowed chair in the American Presidency or on various stages talking about the book, the documentary also skillfully weaves in archival film and photographs, and a handful of well-chosen experts on the topics at hand.

Some, such as actor George Takei, who was interned with his family as a boy, and the late U.S. Rep. John Lewis, a longtime civil rights leader and the subject of Meacham’s 2020 book, also add the power and poignancy of personal experience.

A hopeful ‘Soul’

“The Soul of America” makes a persuasive case that when the tide of public pressure grows strong enough America’s “better angels” will overcome its worst impulses.

Meacham’s faith in the ability of the American people past, present and future to make the right choices even when difficult days precede them is solid.

“Nobody is celebrated for constricting access to the mainstream,” he says. “You are not celebrated for foreclosing the possibilities of full and equal justice under the law. You’re just not.

“I think, self evidently, read that book, watch that film, and tell me whether you want to be Joe McCarthy or Margaret Chase Smith. Do you want to be George Wallace of do you want to be John Lewis? It’s pretty straightforward.”

(In that spirit, Meacham says he’s voting for Joe Biden. “Trump is the embodiment of just about everything that we should avoid in American public life,” he says. “And I’m not a Democrat and I’m saying that. That’s my historical brain talking.”)

As for Davison, she says that making the film has made her personally more hopeful about the chapters yet to be written in American history.

“In the conversations that I’ve been having with people around this film and around where we are, I am getting a sense that people have faith that we can change,” she says. “And Jon, he speaks so wonderfully about faith and hope.

“He is able to look into the darkness and find the light.”

Jon Meacham’s Recommended Reading

We asked Meacham to recommend a few books in addition to his own for those who might want to dive more deeply into periods of American history that emerged successfully from periods such as our own.

Here are six books he recommended organized by the times they address:

Reconstruction through Jim Crow: “The Souls of Black Folks,” by W.E.B. Du Bois and “A Short History of Reconstruction, 1863-1877,” by Eric Foner.

Suffrage: “Sisters: The Lives of America’s Suffragists,” by Jean H. Baker.

Isolationism: “The Plot Against America,” by Phillip Roth.

McCarthyism: “A Conspiracy So Immense,” by David M. Oshinsky.

Civil Rights: “Walking With the Wind: A Memoir of the Movement,” John Lewis.