In Tawahi, the flag has been hoisted over the police station and is flown on cars ferrying bearded men across one of the largest districts of the port city, residents say.
"Armed men of Al-Qaeda control our whole district, even if there are only a few dozen of them," said Raefat, a 32-year-old employee of Yemeni radio and television.
"It's no surprise given there's no state or government," he said.
With the capital Sanaa under the control of Shiite Huthi rebels for the past year, the exiled government returned to Aden as its "provisional capital" in September, only to take flight again within weeks.
Prime Minister Khaled Bahah and his team returned to Riyadh after a deadly October 6 attack claimed by the Islamic State jihadist group on an Aden hotel that was being used to house the government.
A government security official told AFP that the jihadists are active in several areas "like Crater, Khor Maksar and Brigua, where their presence is growing stronger by the day".
The official, declining to be named, said he "fears the city will fall under their total control in the absence of the state".
Bahah's departure was a setback for government forces and the Saudi-led Arab coalition supporting them that had recaptured Aden from the Huthis and their allies in July.
Three months later, hundreds of young gunmen now control most public buildings in the city, according to residents, contacted from Sanaa.
Signup to our newsletter and follow us on Facebook and Twitter!
They claim allegiance to the pro-government "Popular Resistance", a loose coalition of former regular army soldiers, tribal militias, Islamists and southern secessionists.
The coalition has battled against the Iran-backed Huthis but without necessarily pledging loyalty to President Abedrabbo Mansour Hadi, also exiled in Saudi Arabia.
- Fear of masked gunmen -
Aden police chief General Mohamed Mussaed says his men are "cooperating with our Popular Resistance brothers to resolve the security problem and restore police posts... a big challenge".
But Aden residents say they fear the masked gunmen who roam the streets, especially after assailants on motorbikes gunned down at least six people, with Yemen's Al-Qaeda franchise the prime suspect.
"We are counting on the cooperation of the youth of the Popular Resistance to restore security," said the city's police chief.
While Badi has called for such irregular forces to be integrated into the army, impoverished Yemen in the thick of a conflict which has cost about 4,500 civilian lives since March does not have the financial resources for such an operation.
"These armed men claim to be part of the Popular Resistance but nobody knows what group they belong to," protested Majed Ahmed, a resident of Dar Saad, another Aden district where Al-Qaeda's flag flutters over a disused police post.
To finance their operations, Al-Qaeda fighters have seized $6 million worth of diesel from a local port and sold it on the city's black market, according to an Aden refinery official.
Elsewhere in southern Yemen, Al-Qaeda has had control of Mukalla, Yemen's third largest port and capital of Hadramawt province, since April.
At the start of October, its fighters occupied an administrative complex in Zinjibar, capital of Abyan, one of five southern provinces recaptured from the Huthis.