Audience members at a Saturday, May 5 afternoon session during the week-long PEN World Voices Festival of International Literature in New York City could be forgiven a bit of confusion when one participant opened with, “It is my honor to be here with Tony Kushner,” and the second responded with, “Thank you, Suzanne, I’m looking to talking to you about The Hunger Games.” With that exchange, the panelists – who in reality go by the names David Levithan and Brian Selznick – set the tone for a freewheeling discussion that careened from Selznick’s work specifically to film and literature in general, with plenty of laughs in between. (By way of clarification: Kushner – to whom Selznick bears something of a resemblance – was due to speak at a PEN event that evening.)

Levithan began by asking Selznick what draws him to the past – he set The Invention of Hugo Cabret in the 1930s, and his second novel, Wonderstruck, alternates between the 1920s and 1970s. “I have absolutely no interest in writing about today’s technology,” Selznick said in reply. “I like reading those stories. But when I’m thinking of a story or thinking about a character, what interests me is how they make their way in the world. No matter what we write about, we’re writing about ourselves, filtered through other lenses.”

Selznick did acknowledge an affinity with the technological revolution of the turn of the 20th century, when people like filmmaker Georges Méliès – who plays a key role in Hugo – were able to build their own movie cameras from scratch. Nowadays, on the set of the film version of Hugo for example, “One guy’s job was making smoke. Another guy’s whole job was making sparks fly off a train wheel,” he said. “It’s still hand-done, but when we look at technology today, the hand of the artist is lost.”

Even when he’s writing about a more recent era – the 1970s of Wonderstruck – there’s a certain distance between it and his readers. Selznick, who was a boy during that decade, said, “For kids today, the 70s are like what the 30s were to me when I was a kid – which is depressing.” But no matter what period he’s writing about, the author-illustrator said he does tremendous amounts of research before writing, then leaves much of it in the background, off the page. The goal is for his historical fiction to feel as natural is if it were being written at a contemporary time. For example, a writer like Edith Wharton (“She’s become a good friend of mine” – Selznick. “I hear she’s a bitch” – Levithan”), writing a story set in the early 20th century, would not make specific references to a character’s corset or bustle, because those items are assumed parts of her cultural landscape.

As their banter indicated, Levithan and Selznick have a long history together, and Levithan remembers when Selznick was contemplating his next move as an artist, pre-Hugo. Selznick had illustrated a number of picture-book biographies, beginning with Pam Muñoz Ryan’s Amelia and Eleanor Go for a Ride. He was eager for a change, but wasn’t sure what it would be. He mentioned to Levithan an idea that he’d had in the back of his mind for some 20 years – writing a story about a kid who meets Georges Méliès. “For those of us who knew you at the time,” Levithan said, “there was no doubt that that would be the next thing.

Hugo the book began, said Selznick, as a 100-page novella. “In thinking about how to illustrate it, I thought of books like Where the Wild Things Are, and the wild rumpus. It’s four scenes, and you move through those scenes wordlessly. It gives you power – turn the page, move the story forward.”

He also considered cinematic moments like Dorothy in her black-and-white home, opening the door to a Technicolor Oz, and the early days of motion pictures, when plot was driven by images rather than dialogue. He went back to his story and, he said, “took out all of the text that was doing purely descriptive work, and illustrated it. I came up with picture sequences that would replace that text. Three pages of text describing the train station became 48 pages of drawings,” and his 100-page novella blossomed to a 553-page tome. “Turning the page moves the story,” he said. “Or I can point out something important by zooming in like a camera lens. I was trying to make the book feel like a movie, and at the end, I wanted one continuous narrative – I wanted you not to remember what you read and what you saw.”

Ironically, considering that Hugo is such a visually driven, cinematic novel, Selznick confesses, “I always said it couldn’t be filmed. I was wrong.” (Martin Scorsese’s movie has the 11 Oscar nominations and five wins to prove it.) Accolades aside, the author feels that the film version is one of the most faithful adaptations of a book he’s seen – Nick and Nora’s Infinite Playlist, based on the novel co-authored by Levithan and Rachel Cohn, being another.

Selznick and Levithan have more than author-pleasing screenplays in common: each wrote an essay for the newly released anthology The Letter Q: Queer Writers’ Notes to Their Younger Selves. During the PEN discussion, Selznick mentioned that his boyfriend, David Serlin, works with two leading deaf scholars who were instrumental in his understanding of the deaf experience as it related to characters in Wonderstruck. Afterward, during the Q&A, an audience member asked whether Selznick brings up Serlin when he’s on tour, particularly at schools. Selznick said, “I’ve been with my boyfriend for 15 years. It’s very hard not to talk about him.” Of course, he added, being in a room with 500 eighth-grade boys can be nerve-wracking, but “I’ve realized that I like being out, and I like kids knowing. It feels like it shouldn’t be controversial. What’s most powerful for me is the casualness of the mention.”

Levithan – whose debut novel, Boy Meets Boy, takes a matter-of-fact approach to sexual orientation – agrees. “We would have had very different careers 20 years ago,” he said. “I would have had no career. I remember hearing gay authors accept awards and not mention [their partners], and it was disappointing. I thought, our generation will do it differently.” And they have: Serlin accompanied Selznick to the 2012 Academy Awards ceremony. “A friend asked if being gay influenced my work, and at first I said you’re crazy,” Selznick says. “But I kept a secret for a long time, and I wouldn’t change what I went through. It was really hard, but it made me who I am.”