By last count, the Justice Department estimates about 80,000 U.S. inmates live in some kind of restricted housing.
That means being confined to a cell for about 22 hours a day.
"You are going to eat, sleep and defecate in a small room that's actually smaller than the size of your average parking space," said Amy Fettig, a lawyer who runs the Stop Solitary campaign for the American Civil Liberties Union. "And you're going to do that for months, years and sometimes even decades on end."
Fettig said solitary confinement is brutal and expensive.
"So for example in places like Arizona that may spend $20,000 a year on a general population prisoner, they're spending $50,000 dollars or more on somebody they hold in solitary confinement," she added.
Big states such as California, Colorado, New York and Mississippi have been moving to limit the numbers of people they send to solitary — or to limit the amount of time inmates live in isolation. Those restrictions would cut back on the number of people like Louisiana's Albert Woodfox, who has spent 43 years in solitary. A federal judge ordered his release this week.
But corrections officials said they need restricted housing as an option to help control the most dangerous inmates. Charles Samuels, director of the Federal Bureau of Prisons, described the problem at a Senate hearing a few years ago.
"We believe with solitary confinement for the inmates who pose the most violence and disruption within the facility that we utilize it as a deterrent," Samuels testified.
Samuels and state corrections officers emphasized the need for a way to house murderers, terrorists, and inmates who hurt or kill people while incarcerated. And taking solitary off the table, Samuels said, can put staff members and other inmates at risk.
Yet advocates who fight solitary confinement said it's often not reserved for the worst of the worst. Instead, they said, it's become a routine tool for dealing with unruly inmates, the mentally ill, and juveniles.
Ian Kysel, who teaches at Georgetown law school, has been interviewing young people about solitary confinement for years.
"The children talked about wanting to die, about losing control, having hallucinations," Kysel said.
Corrections officials sometimes put juveniles in isolation to protect them. But Kysel said many juveniles caught up in the justice system already suffer from mental illness and trauma.
And he said putting them in solitary just makes those problems worse.
"Sometimes even access to a book or to visitors or family contact is cut off once a child is placed in solitary confinement and this really has an effect on children," he added.
The vast majority of people held in isolation live in state jails or prisons. So any changes to the practice happen incrementally, state by state.
Still, advocates who want to restrict solitary confinement say they've noticed more attention to the issue over the past few years. The U.S. Senate Judiciary Committee has held two hearings, and state legislatures, even more.
"I really think that solitary confinement is no longer the dark secret of the criminal justice system," Kysel said.
And Fettig, of the ACLU, said she's noticed that law enforcement authorities are more open to change too.
"Corrections leaders are waking up to the fact that solitary confinement simply doesn't work, that they need to do something else, that they need to give their staff more tools and they need to use their resources to rehabilitate people and not just warehouse them," she said.
After all, Fettig said, about 95 percent of people housed in solitary confinement eventually leave prisons and return to the streets.