Wells Fargo & Co. has agreed to pay more than $175 million to settle claims that some of its mortgages unfairly affected black and Hispanic buyers, the bank said Thursday.

The agreement with the U.S. Department of Justice marks the second-largest fair lending settlement in the department’s history, resolving claims that some minority borrowers who qualified for loans were charged higher fees or were improperly placed into subprime loans.

Those claims were based on a statistical survey of Wells Fargo Home Mortgage loans between 2004 and 2009, relating primarily to mortgages priced and sold to consumers by independent mortgage brokers, Wells said.

The San Francisco-based bank, which bought Charlotte’s Wachovia in 2008, denies the claims but agreed to pay $125 million to borrowers the DOJ said were steered into subprime mortgages or who paid higher rates and fees than white borrowers because of their race or national origin. The settlement is subject to court approval.

Wells Fargo will also provide $50 million in direct down-payment assistance to borrowers in communities across the country where the justice department identified large numbers of borrowers who might have been discriminated against, the DOJ said.

“The department’s action makes clear that we will hold financial institutions accountable, including some of the nation’s largest, for lending discrimination,” Deputy Attorney General James Cole said.

Separately, Wells Fargo announced that it will stop funding mortgages originated, priced and sold by independent mortgage brokers – who operate as independent businesses and are not employed directly by the bank – through its mortgage wholesale channel. Those mortgages represent 5 percent of the company’s home mortgage funded volume, Wells said.

Beginning Friday, the company will no longer accept new applications for such loans but will work to process and close existing applications, it said.

Wells Fargo is settling with the DOJ “solely for the purpose of avoiding contested litigation,” it said in a news release Thursday.

“We believe it is in the best interest of our team members, customers, communities and investors to avoid a long and costly legal fight, and to instead devote our resources to continuing to contribute to the country’s housing recovery,” Wells Fargo Home Mortgage President Mike Heid said.

As part of the agreement with the DOJ, the bank will launch an internal review of a small percentage of subprime mortgages delivered through its retail channel from 2004 to 2008.

The settlement also resolves pending litigation filed in 2009 by the state of Illinois on behalf of borrowers there and an investigative complaint filed in 2010 by the Pennsylvania Human Relations Commission.

Wells agreed to pay $50 million to community improvement programs in the city of Baltimore and other areas the DOJ identified as most in need of support after the housing crisis, including Washington, D.C., Chicago, Philadelphia and San Francisco.

The bank is entering into a collaborative agreement with the city of Baltimore in addition, in which the city will dismiss the lawsuit it filed against the bank in January 2009, Wells Fargo said.

A portion of the $50 million community improvement money will go to Baltimore, and the bank will grant the city $3 million in additional funds for local priority housing and foreclosure-related initiatives.