Pro-government forces recaptured an oil terminal as well as the city of Mukalla, which was considered a jihadist stronghold, military sources said.
"The operation resulted... in the death of more than 800 Al-Qaeda members and some of their leaders, while some others fled," Arab coalition commanders said in a statement published by SPA, the official Saudi news agency.
The death toll could not be independently confirmed and no indication was given of civilian casualties.
The operation was part of a wider offensive aimed at securing parts of the country captured by jihadist militants who have exploited a 13-month war between Gulf-backed loyalists and rebels supported by Iran.
It coincides with UN-brokered peace talks in Kuwait after a ceasefire entered into effect on April 11, but from which jihadists groups are excluded.
"We entered the city centre (of Mukalla) and were met by no resistance from Al-Qaeda militants who withdrew west" towards the vast desert in Hadramawt and Shabwa provinces, a military officer told AFP by phone from the city the jihadists seized last April.
The officer, who requested anonymity, said residents of Mukalla, home to an estimated 200,000 people, had appealed to the jihadists to spare the city the destruction of fighting and to withdraw.
Yemeni military sources said Emirati military vehicles were used in the operation and that troops from the Gulf country, a key member of the Saudi-led coalition, were among the forces that entered Mukalla.
AFP could not immediately confirm these reports from officials in the United Arab Emirates.
The Arab coalition battling rebels in Yemen since March 2015 carried out air strikes against Al-Qaeda positions in Mukalla to pave the way for the ground troops, military sources said.
Troops also recaptured Mina al-Dhaba oil terminal in Shehr further east, the sources said.
Earlier Sunday, military sources said pro-government forces seized Riyan airport and an army brigade headquarters Al-Qaeda had held for a year on Mukalla's outskirts.
Signup to our newsletter and follow us on Facebook and Twitter!
Yemen-based Al-Qaeda in the Arabian Peninsula (AQAP) is regarded by Washington as the network's most dangerous branch, and has carried out deadly attacks on the West in the past.
Last month, a US air strike on an Al-Qaeda training camp in Hadramawt province killed dozens of fighters in a major blow to the jihadists.
A provincial official in Shabwa said jihadists also fled from the town of Azzan on Sunday which they seized in February.
Bomb kills 7 troops
As the anti-jihadist offensive gained momentum, a bomb-laden vehicle exploded Sunday, killing seven soldiers and wounding 14. They were in a convoy entering another southern jihadist stronghold -- Zinjibar, capital of Abyan province, said military sources, blaming Al-Qaeda for the attack.
The coalition, led by Sunni powerhouse Saudi Arabia, has deployed Apache helicopters to support loyalists fighting on the ground.
Forces loyal to internationally recognised President Abedrabbo Mansour Hadi's government have retreated from Zinjibar after entering it late Saturday, an officer in Abyan told AFP.
"The withdrawal was decided following information that Al-Qaeda was preparing other car-bomb attacks against our troops," added the officer who requested anonymity.
Coalition-backed forces have also driven militants from Aden, the southern city declared by Hadi as Yemen's temporary capital after the Shiite Huthi rebels overran Sanaa in September 2014.
And last week, government forces expelled AQAP militants from Huta, the provincial capital of Lahj.
When US President Barack Obama met Gulf leaders on Thursday in Saudi Arabia, they discussed the wars in Yemen and Syria.
During the visit, Ben Rhodes, one of Obama's closest advisers, urged all warring sides in Yemen to participate "constructively" in the Kuwait talks that began on Thursday, saying that a political solution would "allow for a focus on AQAP in Yemen".