“Outrageous” and “copyright infringement” were the first two (unsolicited) emails PW received from independent publishers when word of Audible’s new program to run text along side its audiobooks began to spread. The program, called Captions, which requires the company to transcribe audio to text, was highlighted in a story in USA Today with a headline touting that Audible is looking to let customers “ ‘read’ an audiobook while [they] listen.” While the company disputes that description, saying Captions is not at all akin to the act of reading, publishers, literary agents, and organizations representing authors are skeptical.

While Audible said in a statement that Captions “does not replicate or replace the print or eBook reading experience,” publishers are unconvinced. “There are real copyright issues here and authors, publishers, and agents should review and clarify their positions,” said Dominique Raccah, CEO of Sourcebooks. “It seems unlikely that Audible was granted these rights.”

Although Audible said Captions was “designed primarily to fill an unmet need in education” by allowing students listening to a book to engage more fully with the work, publishers feel the intention of the program is moot. Declining to answer specific questions about whether Audible has the rights to display the text of works, the company insisted that publishers will better understand the program after its official launch in September. Currently in beta, Captions has not been shown to publishers as yet, and, Audible said in its statement, “Given that the feature isn’t live yet, we are in discussions with content providers to help address some confusion about how Audible Captions works and what listeners will experience.”

At least one major publisher, Simon & Schuster, has already deemed the program illegal. In a statement released by a spokesperson, S&S said: “We have informed Audible that we consider its Captions program to be an unauthorized and brazen infringement of the rights of authors and publishers, and a clear violation of our terms of sale. We have therefore insisted that Audible not include in Captions any titles for which Simon & Schuster holds audio or text rights.”

The Association of American Publishers and the Authors Guild issued statements that also said Audible’s contracts do not give the company the right to create a text product. “Existing ACX and Audible agreements do not grant Audible the right to create text versions of audiobooks, whether delivered as a full book or in segments,” the Guild statement noted. “The Captions program appears to be outright, willful copyright infringement.”

In its statement, issued on behalf of AAP CEO Maria Pallante, the association also stressed that Audible has no right to create a text version of its audiobooks. Noting that Audible can sell audiobooks only because of “explicitly licensed audiobook agreements,” the statement noted that the company “cannot also create wholly unauthorized, text versions of these audio performances unless the copyright owners agree that such adaptations are permissible. Such adaptations could serve to undermine their legitimate and separately licensed markets for books and eBooks.”

Both the guild and AAP said they are reviewing what further steps to take, with the guild noting that the Captions program “will inevitably lead to fewer e-book sales and lower royalties for authors.”

Audible released a video showing how Captions works, which you can see here:

Wendy Strothman, a literary agent with an eponymous firm, said Captions feels “like a rights grab.” Adding that most of her standard contracts with Audible do not give the company “the right to reproduce text,” Strothman said she feels Audible needs to look closely at its contracts. “I think Audible needs to read the license agreements it has with publishers and authors. They outline the rights that they’ve licensed, and I don’t think they’ve licensed these rights.”

Likewise, Robert Gottlieb, chairman of literary agency Trident Media Group, sounded the alarm about Audible overstepping. “If Audible has not cleared the rights to use the text, then they are impinging on rights granted to others,” he said.

While Audible sidestepped any discussion on the question of whether it has, or needs, certain rights to go forward with Captions, it reiterated that publishers and others in the literary community will be put at ease once they had a better understanding of the program. Nonetheless, as Strothman explained: “I don’t care that you might be able to only read the text slowly. You can turn off the sound and read the text. That sounds like an e-book to me.”

Publishing attorney Lloyd Jassin said Captions sounded like an example of copyright infringement. “The benefits of reading and listening are two separate revenue streams,” he said, noting that combining them without explicit rights to do so would be legally questionable. As to Audible’s insistence that Captions is a program intended largely for educational use, Jassin sees this as a way to “lay the foundation for a fair use defense.”

A number of industry sources who spoke to PW for this story said they also took issue with the notion that, because Captions is intended for educational use, it would somehow make the fair use question moot. “I don’t see where education comes into play,” Strothman said, noting that one of the few titles that Audible announced in its beta for Captions is Michelle Obama’s Becoming. That book, Strothman noted, is a bestseller, and not a textbook or a mainstay on school reading lists. Regardless, as other sources noted, Audible is very squarely in the consumer market and has built its business selling titles primarily to adult consumers.

Melville House’s Dennis Johnson compared Captions to Amazon’s text-to-speech (TTS) effort, a program that the tech giant, who owns Audible, unveiled 10 years ago on the Kindle 2. “Previously, the issue was one of Amazon turning texts into audio, when it didn’t own audio rights,” Johnson said. “Now, it’s doing the opposite of that. It’s offering the public something it doesn’t own—print rights.” Amazon gave up the effort to turn written content on the Kindle to audio after protests from various groups within the creative community. In making the change, Amazon said that, while it believed TTS was legal, its was dropping the policy in order to make rights holders more comfortable with the concept. Johnson is hopeful that if enough companies, organizations, and individuals speak out, Audible will also reverse course on Captions.