Prime Minister Mohamed Basindawa stepped aside, accusing President Abdrabuh Mansur Hadi of being "autocratic", senior officials said.
State news agency Saba reported Basindawa's resignation, but without giving the reason.
However, the council of ministers released the text of Basindawa's resignation letter.
"The partnership between myself and the president in leading the country only lasted for a short period, before it was replaced by autocracy to the extent that the government and I no longer knew anything about the military and security situation," he wrote.
Mohammed Abdulsalam, spokesman for the Ansarullah rebels, also known as Huthis, said on his Facebook page that the seat of government had been taken.
He also said parts of "the military and security apparatus have supported the popular revolt", including "the general command of the armed forces, the radio", the government and other institutions.
A posting on the interior ministry's website urged security forces not to confront the rebels.
Interior Minister Abdo al-Tarib calls on "all members of the ministry not to confront Ansarullah," the statement said.
Tarib urged "cooperation" with the rebels "to strengthen security and stability, preserve public property and guard government installations... and to consider Ansarullah friends of the police".
One Western diplomat, who also reported rebel movements near Hadi's residence, said: "The situation is changing at speed with a capital S."
Earlier, shelling and gunfire in the north of Sanaa was heard across the city, as Sunni militiamen and troops battled the rebels, prompting an exodus of terrified residents, an AFP correspondent reported.
A week of fighting has left dozen of people dead on both sides and forced the suspension of all flights into and out of Sanaa airport, which is in the battle zone.
There was no let-up in the fighting on Saturday night despite Hadi ordering an after-dark curfew.
The clashes centred on the campus of Al-Iman University, a bastion of Sunni Islamists that the Shiite rebels have been trying to capture, witnesses said.
The violence came despite UN envoy Jamal Benomar announcing late on Saturday that a deal had been reached after "intense consultations with all the political parties, including Ansarullah".
Benomar did not specifically mention a ceasefire, nor did he say when a deal would be signed, although he said preparations were under way for the signing.
Signup to our newsletter and follow us on Facebook and Twitter!
But he did say the accord would be a "national document that will advance the path of peaceful change, and will lay the foundations for national partnership and for security and stability in the country".
On Sunday evening, Benomar entered a meeting with Hadi.
Forces allied to the government have been battling to halt the rebels, who swept into Sanaa from their mountain stronghold in the far north last month and set up armed protest camps across the capital to press their demands.
REBELS DEMAND KEY POSTS
Hadi has denounced the rebel offensive as a "coup attempt", but agreed to involve the rebels in the formation of a new government to replace the unpopular administration that imposed austerity measures, including a fuel price hike, earlier this year.
He has also agreed to partly reverse the price hike.
But the rebels have also demanded posts in key state institutions as part of a push for greater political clout.
The fighting in northern Sanaa has raged continuously since Thursday when nearly 40 people were killed in a single day.
Residents of northern districts have begun to flee their homes, an AFP correspondent reported earlier on Sunday.
The city's streets were largely deserted as shops remained closed and the education ministry ordered schools to suspend lessons indefinitely.
Sanaa University told students to stay away until mid-October after its campus was hit by shelling.
One of Sanaa's main markets, the Ali Mohsen Souk, has been closed for three days, which residents said had started to cause problems in obtaining fresh produce.
Yemen has been swept by political turmoil since longtime strongman Ali Abdullah Saleh was forced from the presidency in early 2012.
Al-Qaeda loyalists have taken advantage of the political turmoil in the impoverished Arabian Peninsula state to launch persistent attacks on the security forces.
Separatist protests have also rocked the formerly independent south.
The rebels hail from the Zaidi Shiite community, a minority in the mostly Sunni nation but the majority community in the northern highlands, including the Sanaa region.
Also known as Huthis after the name of their leading family, they have battled the government on and off for a decade from their stronghold of Saada in the far north.