Troops and militia retook the central prison and deployed on main roads across the Mansura residential district after a three-hour gunbattle with the jihadists, the sources said.
There was no immediate word on casualties.
The Sunni extremists of Al-Qaeda have exploited conflict between the government and Shiite rebels who overran the capital Sanaa in September 2014 to expand their control in the south.
A Saudi-led coalition, which intervened in support of President Abedrabbo Mansour Hadi when he fled into exile in March last year, concentrated its firepower on pushing the rebels and their allies out of Aden and neighbouring southern provinces, and the jihadists took advantage.
But in recent days, the coalition has carried out a series of air strikes against Al-Qaeda in cities it has seized including Hadramawt provincial capital Mukalla and Abyan provincial capital Zinjibar.
Five militants were killed and three wounded in Monday strikes on Mukalla, a major port city that the jihadists seized last April, provincial officials told AFP.
Signup to our newsletter and follow us on Facebook and Twitter!
Zinjibar residents told AFP that Al-Qaeda fighters were evacuating public buildings in the city on Tuesday in apparent fear of new strikes.
The coalition raids follow a US strike against an Al-Qaeda training camp outside Mukalla last week that killed 71 militants, according to provincial officials.
On Tuesday, hundreds of people took part in an Al-Qaeda-organised protest in Mukalla against the US raid, witnesses said.
"US raids will not defeat jihad," banners carried by the demonstrators said.
But other residents resisted the jihadists' efforts to get them to join the protest, the witnesses said.
There has been no let-up in the longstanding US air war against Al-Qaeda's Yemen-based branch, which it regards as the jihadist network's most dangerous.
US strikes have taken out a number of senior Al-Qaeda commanders in Yemen over the past year.