The city's program would have given black seniors in the city's Western Addition preference to move into a 98-unit affordable housing development this fall.
The Board of Supervisors and Lee enacted the neighborhood preference plan in part to answer the continuing departure of the city's African-American population.
The measure sets aside 40 percent of the new affordable housing units for qualified residents who already live in the supervisorial district where a new affordable development is being built.
San Francisco officials acknowledge that the city's affordable housing lotteries are very competitive -- they emphasize that the legislation improves the chances of residents moving into affordable housing in their own neighborhoods.
Lee wrote to Castro earlier this month, requesting the agency reconsider its decision.
"I believe that the decision is adverse to our mutual goals regarding equity, displacement prevention and the creation of opportunities for vulnerable populations," Lee wrote.
Now the mayor is following up with a delegation to HUD.
"We are not going to give up as we experience the outmigration of African-Americans and others from San Francisco," said mayoral spokeswoman Deirdre Hussey. "The mayor believes neighborhood preference is an anti-displacement policy."
The city's contingent will "take HUD up on their challenge to find anti-displacement policies that work for our city and our people."
The city is working with Feinstein and House Minority Leader Nancy Pelosi on the issue.
Feinstein sent a letter to Castro on Friday urging him to come to a solution with city officials by the end of President Obama's term.
"I believe it is incumbent upon your department to work with cities, particularly those in California that have seen dramatically increasing housing costs in recent years, to develop innovative strategies to prevent residents from being priced out of their historic neighborhoods," Feinstein wrote.
"San Francisco's tech boom has led to increased demand for limited housing, particularly in the Western Addition," the California senator wrote. "I am deeply concerned that, if left unchecked, this trend will destroy the fabric and culture of the individual neighborhoods that make San Francisco so unique. Please let me know if new legislation would be helpful to clarify the law on this issue."
Pelosi's office is also lobbying HUD.
"We support San Francisco's neighborhood preference and have advocated strongly with HUD," said Jorge Aguilar, a Pelosi spokesman. "We are disappointed by the department's decision and hope they will reconsider."
London Breed, president of the Board of Supervisors who co-authored the legislation, is calling on Pelosi and other Democrats to fight for the plan.
"We want action, we want housing," Breed, who's running for re-election, said during a news conference last Thursday. "We're going to fight tooth and nail to make sure that we get it."
Breed was joined by Supervisor Malia Cohen and the Rev. Amos Brown, president of the San Francisco NAACP.
"The soul of San Francisco's neighborhoods are on the line," Cohen, also a co-author, said at Thursday's news conference.
"The only intention of the mayor and the board was to make sure we stop the hemorrhaging and the loss of African-Americans from this city," Brown said in an earlier interview.
The Western Addition project, known as the Willie B. Kennedy development, is partially funded by HUD.
Adrian Williams would get preference to move into the building because she already lives in the neighborhood.
"I watch every day as I see seniors on the streets that have given back to this community, and now they're homeless," Williams said.
If the city moves forward on the preference plan, it risks losing HUD money, said Tim Iglesias, a law professor at the University of San Francisco School of Law, who specializes in housing issues.
"The city is caught between a rock and hard place," Iglesias said. "This is a collision between fair housing law and our ongoing chronic affordable housing crisis."
KQED's Matt Beagle and Mina Kim contributed to this post.
