Anyone who writes a memoir is asking to be called a liar. Folks in Limerick, Ireland, claimed that Frank McCourt made up whole sections of Angela's Ashes. Sean Wilsey's stepmother sent a letter to the publisher of Oh, the Glory of It All claiming that the book contained more than 30 "actionably defamatory statements of fact." And last month, Augusten Burroughs was sued by members of his adoptive family, who charged that in his bestselling memoir Running with Scissors, he had "fabricated events that never happened and manufactured conversations that never occurred."

While there hasn't been any threat of litigation over The Glass Castle, my memoir of growing up with willfully eccentric parents, I can certainly relate to those writers' experiences. Everyone who has read my book seems to have an opinion about it. "Walls's book is very troubling to thoughtful readers and to anyone who has suffered the childhood she purports to have had," one woman wrote on Amazon. She went on to say, "I am angry at a memoir that seems to me deceitful, but I find myself wishing every word were true."

The charge that I exaggerated the facts of my life has left me perplexed, infuriated and even a bit amused. Having spent years hiding my past, I sat down to write about it after my mother dared me to "tell the truth." So far, no one I know, and certainly no one in my family, has challenged my account.

Even so, aspects of my childhood are so bizarre that people simply find them hard to fathom. I got some clue of what was to come when a lawyer for my publisher vetted my manuscript. "You are alleging child neglect," she said at one point. Not really, I shrugged. It was just the way I was raised. The events that she found most horrifying seemed pretty unexceptional to me. "Some people who've lived very normal lives refuse to believe those of us who haven't," Augusten Burroughs told me this past summer.

Then there's the question of the reliability of memory. People have asked me how I can recall something that happened to me when I was three years old. My answer is: How could I possibly forget being burned so badly that I was in the hospital for six weeks and had to have skin grafts?

The most important goal of a memoir writer is to tell the truth. But truth is subjective, especially when one is condensing decades into several hundred pages. My brother, my sisters and my mother have all said that while they felt my book was substantially true, any memoir they would have written would have been entirely different.

One of my most cherished memories was of Christmas when I was five. My parents had no money for presents, but Dad took each of us children out into the desert night and gave us any star we wanted. My brother—who is a year younger and has a steel-trap memory for most things—didn't recall the incident at all. My older sister saw it as yet another example of Dad conning us into thinking our hardships were in fact a great adventure. "That's just like that s.o.b.," she said. "Giving away something he didn't own in the first place."

Sometimes, people aren't upset by what you think will set them off. I struggled over whether or not to include a scene where we kids had gone without food for days and my brother caught Mom secretly chomping away on a giant chocolate bar. I thought it was pretty damning, but said something significant about our family, so I left it in. My mom never mentioned it. She was, however, quite indignant about my description of her atrocious driving. "Your father always used to make fun of my driving, and this is just like you to side with him on that," she said.

All in all, though, my mom said she was fine with the book. "There are certain things that I saw differently than you," she told me. "But then I realized, 'That's how it must have looked to a little girl.' You had to tell the truth as you saw it."

Jeannette Walls is the author of the bestselling memoir The Glass Castle, which will be out in paperback from Scribner in January.