Trump issued the threat during a meeting with state and local officials on school safety and gun violence. He told Attorney General Jeff Sessions and others that his administration has targeted members of the violent MS-13 gang, but has been "getting no help from the state of California."
"Frankly, if I wanted to pull our people from California, you would have a crime nest like you've never seen in California. All I'd have to do is say is, 'ICE and Border Patrol, let California alone,' you'd be inundated. You would see crime like nobody has ever seen crime in this country."
He added: "If we ever pulled our ICE out, and we ever said, 'Hey, let California alone, let them figure it out for themselves,' in two months they'd be begging for us to come back. They would be begging. And you know what, I'm thinking about doing it."
An ICE spokeswoman in California said she had no information about such a move and referred reporters to the White House for explanation.
The White House did not immediately comment on the president's suggestions.
In a statement, Sen. Dianne Feinstein (D-Calif.) called the remarks "outrageous," and said the state's crime rate is at a historic low.
“The president’s attacks are not only mean-spirited, they’re patently false,” she said.
Many cities have been openly defiant in the face of the threats, with lawsuits pending in Chicago, Philadelphia and California over whether the administration has overstepped its authority by seeking to withhold grant money.
The Justice Department has blamed "sanctuary city" policies for crime and gang violence, and criticized jurisdictions that aim to thwart the federal government's immigration enforcement.
Defenders contend these practices improve public safety by promoting trust among law enforcement and immigrant communities, and set aside scarce police resources for other, more urgent crime-fighting needs.
California Attorney General Xavier Becerra said the state actually works with federal law enforcement on a daily basis.
“We’re going after drug dealers, sex traffickers. We won’t stop doing that. My division of law enforcement is doing that right now," he said. "What we won’t do is change from being focused on public safety – we’re in the business of public safety, not deportation.”
Immigrant advocates have strongly supported California's "sanctuary state" law, enacted last year, that limits law enforcement cooperation with federal immigration authorities (except when a person has been convicted of one of hundreds of crimes deemed serious).
Jorge-Mario Cabrera, a spokesman for the Coalition for Humane Immigrant Rights in Los Angeles, said Trump is out of step with the Golden State.
"His erratic comments reflect an obsession with criminalizing immigrants and shows a deep lack of knowledge and respect for California values and immigration laws," said Cabrera.
In November, Trump called it "disgraceful" that a San Francisco jury found a Mexican man not guilty in the killing of a woman on a city pier, in a case that spotlighted its "sanctuary city" policy.
The defendant, who said the shooting was an accident, had been deported five times and was wanted for a sixth deportation at the time of the incident.
"No wonder the people of our Country are so angry with Illegal Immigration," Trump tweeted after the verdict.
Before the shooting, the man had finished a federal prison sentence for illegal re-entry into the United States and was transferred to San Francisco's jail to face a 20-year-old charge for selling marijuana.
The sheriff's department released him a few days later, after prosecutors dropped the marijuana charge, despite a request from federal immigration officials to detain him for deportation.
This post contains reporting from the Associated Press.
