WASHINGTON (CNN) -- Attorney General Eric Holder will announce Monday that the Justice Department will no longer pursue mandatory minimum sentences for "certain low-level, nonviolent drug offenders."

In a speech at the annual meeting of the American Bar Association's House of Delegates in San Francisco, he will make the case that the United States "cannot simply prosecute or incarcerate our way to becoming a safer nation."

Holder it set to announce that "drug offenders who have no ties to large-scale organizations, gangs, or cartels will no longer be charged with offenses that impose draconian mandatory minimum sentences."

They now "will be charged with offenses for which the accompanying sentences are better suited to their individual conduct, rather than excessive prison terms more appropriate for violent criminals or drug kingpins."

Lessening the use of mandatory minimums -- sentences that require a mandatory, "one-size-fits-all" punishment for those convicted of federal and state crimes -- could mark the end of the tough-on-crime era, which began with strict anti-drug laws in the 1970s and accelerated with mandatory minimum prison sentences and so-called three-strikes laws.

Holder is set to label these types of sentences as "draconian," "counterproductive" and "excessive."

The attorney general's speech will also hit upon the reality that the federal and many state budgets are experiencing -- increasing costs.

Legislation to lessen the use of mandatory minimums, Holder will say, "will ultimately save our country billions of dollars."

"Although incarceration has a role to play in our justice system, widespread incarceration at the federal, state, and local levels is both ineffective and unsustainable. It imposes a significant economic burden -- totaling $80 billion in 2010 alone -- and it comes with human and moral costs that are impossible to calculate," Holder will say, according to prereleased excerpts of his remarks.

In 2009, incarceration cost federal, state and local budgets $83 billion.

The administration hopes the move will also address racial disparities in the U.S. prison population, of which ethnic minorities are a majority.

President Barack Obama nodded to some of the issues in remarks he made after the Trayvon Martin verdict last month, giving voice to African-American concerns that "there is a history of racial disparities in the application of our criminal laws -- everything from the death penalty to enforcement of our drug laws."

Although Obama administration officials say the changes they are pursuing will not require congressional approval, some unlikely pairs of lawmakers have come together to push criminal justice changes.

Democratic Sen. Patrick Leahy of Vermont and Republican Sen. Rand Paul of Kentucky have worked together to allow judges to depart from mandatory minimum sentences when circumstances merit. Democratic Sen. Dick Durbin of Illinois and Republican Sen. Mike Lee of Utah have undertaken similar efforts.

In his speech, Holder will highlight the work of each lawmaker.

In recent years, there has been a rise in support among conservatives for reforms to the criminal justice system. While more flexible approaches to crimes have long held support among liberal Democrats, fear of being tarred as weak on crime by Republican opponents has long caused moderate Democrats, particularly those running for president, to avoid the issue.

In addition to changes to mandatory minimums, Holder will call for expanding the use of "compassionate release" of people in jail who "pose no threat to the public."

"In late April, the Bureau of Prisons expanded the criteria which will be considered for inmates seeking compassionate release for medical reasons," Holder will say. "Today, I can announce additional expansions to our policy -- including revised criteria for elderly inmates who did not commit violent crimes and have served significant portions of their sentences."

CNN's Carol Cratty contributed to this report.

™ & © 2013 Cable News Network, Inc., a Time Warner Company. All rights reserved.