It has been almost a year to the day since I began representing Milo Yiannopoulos as his literary agent. I have largely stayed quiet about the controversy that started right before the New Year, when the book deal became public, but I feel compelled to respond to Joy Peskin, whose soapbox “Drawing the Line” appeared in Publishers Weekly on February 6.

I’ve been continually shocked by the willingness of many in the publishing industry to stifle Milo’s opinions. The right to speak freely, even if your opinions are unpopular, should be the bedrock of our industry. As Bill Maher said recently in response to the UC Berkeley riots, “Free speech should be something we own.” The AAP, ABA, Authors Guild, CBLDF, FTRF, Index on Censorship, NCAC, and NCTE all agree with me. Unfortunately, many in publishing did not heed their sound and sage words.

To be clear: I do not agree with everything Milo says. As the cowriter of transsexual icon Amanda Lepore’s forthcoming memoir, I take particular issue with things he’s said about the transgender community. However, I have no desire to silence his opinion. On the contrary, whenever I have dinner with Milo, I debate him on this topic and others we see differently. That is what we as purveyors of intellectual thought are supposed to do.

If you have watched Milo’s q&as with audience members during his Dangerous Faggot college tour, you will know that he speaks eloquently and backs up his opinions with facts. “Facts not feelings” is a favorite phrase of Milo’s, and for good reason. Peskin states that Milo’s brand is “racism and misogyny,” a heinous charge she did not and cannot justify. As she makes abundantly clear in her attempted takedown of Milo, his critics often get lost in their hurt feelings and often resort to half-truths and flat-out lies.

Let’s clear up one of those lies that has often been repeated about Milo, including by Peskin. Milo never made racist statements about actress Leslie Jones and never asked his Twitter followers to do so. Yes, he made fun of Jones’s appearance, and it was unkind.

However, his words were no worse than what Chelsea Handler, Kathy Griffin, or any number of other successful celeb authors have used. Conservative voices are often held to this double standard. Jones was subsequently bombarded with racist and vile tweets; Milo condemned these tweets. He did not participate. He did not incite. While Twitter banned Milo, the company has never stated what exactly he said that led to the ban. The reason is likely that there was nothing he actually said to warrant his banning; Twitter just doesn’t like him and doesn’t like what he stands for. This should make those who have devoted their lives to the written word acutely uncomfortable. Instead, many in the publishing industry applauded.

Milo is provocative and charismatic, which has put a huge target on his back. His book is called Dangerous because, to many people, a gay Jew who doesn’t kowtow to the party line jeopardizes long-held beliefs that liberals are the party of inclusion, and the other guys are the party of hatred. This disruption of the status quo has left many feeling threatened. When protesters try to silence Milo, when they show up to his events and physically threaten him, or scream and smear fake blood all over themselves, or riot and destroy property, they are using tactics I, as a self-described progressive, have always chided others for using. I won’t stand for it when religious groups try to silence transgender supporters, and I won’t stand for it when so-called progressives try to silence conservative voices.

I am proud of Milo Yiannopoulos and believe in his core message that political correctness is causing more harm than good. As a gay man, I am proud of Milo Yiannopoulos for saying things about gay culture I have often thought but never before heard uttered in public. And that to me is my job as a literary agent: to find writers with talent, new ideas, and the platforms to share them with the world. You may not like what Milo has to say or how he says it. But, as a member of the publishing community, you should be extremely cautious of your willingness to shut down a book you haven’t even read. Unless, of course, you’re scared you might agree with him.

Thomas Flannery Jr. is an agent with AGI Vigliano Literary and a writer. His first book, Doll Parts, cowritten with Amanda Lepore, will be published in April by Regan Arts.