"The delegation of the republic of Yemen has suspended its participation in Kuwait talks because of the continued violations by rebels and their takeover of Al-Amaliqa base," foreign minister and head of delegation Abdulmalek al-Mikhlafi said on Twitter.
He said the suspension will last "until guarantees for compliance were provided", without providing details.
A spokesman for Mikhlafi told AFP that the government delegation has suspended its participation in both "direct and indirect" talks taking place in Kuwait.
"The suspension will continue until guarantees are provided that the rebels will stop their ceasefire violations and withdraw from the base," Mane al-Matari told AFP.
A statement by the government delegation called on the UN envoy, Kuwait and other Gulf states and countries backing the peace process "to apply pressure on the other side to comply with peace requirements."
The United Nations said it was informed by the government delegation that it will not attend a round of talks scheduled for later Sunday.
UN envoy Ismail Ould Cheikh Ahmed said he made contacts with members of both delegations and that he had received assurances that they would continue to try to resolve their differences but without face-to-face meetings.
Huthi rebel spokesman Mohammed Abdulsalam however criticised the Yemeni government's decision.
"Those who don't want peace ... are the ones who create false justifications and reasons to obstruct," the talks, Abdulsalam wrote on Twitter.
Signup to our newsletter and follow us on Facebook and Twitter!
On Saturday, Yemen's warring parties held their first face-to-face talks since the negotiations in Kuwait began on April 21.
The UN envoy said these direct talks were "productive" and had touched on key issues.
But later Saturday, the rebels and their allies overran Al-Amaliqa base in northern Yemen after hours of clashes, tribal and military sources said, adding that the fighting caused casualties.
The sources said the commanders of the 600-strong brigade at the base, located in the rebel stronghold province of Amran since 2014, had "chosen to remain neutral" as pro-government forces, backed a Saudi-led coalition, clashed with the insurgents across Yemen.
"The attack against Al-Amaliqa brigade torpedoes the peace consultations in Kuwait," Mikhlafi has said on Twitter.
Yemen's warring parties have repeatedly traded blame for ceasefire violations.
Government loyalists said they have recorded "3,694 ceasefire violations by the Huthis and their allies" -- troops fighting in support of former president Ali Abdullah Saleh.
Meanwhile, the rebels accused government forces and the Saudi-led coalition backing them of 4,000 breaches.
The coalition in March 2015 began a military campaign against the rebels, who have seized the capital Sanaa among other parts of the country.
The UN says that more than 6,400 people have been killed since then and around 2.8 million displaced.