Men on motorcycles fired seven times at Colonel Abdullah al-Tamimi in the Fuah district, killing him on the spot, before fleeing the scene, the source said.
Mukalla is the capital of the southeastern province of Hadramawt and is a major port city.
On September 30, gunmen from the Al-Qaeda-affiliated Ansar al-Sharia group seized an army headquarters there after a suicide bomber rammed a car into the entrance.
Signup to our newsletter and follow us on Facebook and Twitter!
The army soon retook most of the building, except for the top floor, where the militants held the soldiers captive.
Days later, troops assaulted the top floor in an operation in which the government said all the gunmen and an unspecified number of hostages died.
Al-Qaeda in the Arabian Peninsula has intensified its attacks on security forces in Yemen, mainly in its lawless southern and eastern provinces, where jihadist groups are active.
The Islamist network has taken advantage of the weakening of the central government since a popular uprising that toppled president Ali Abdullah Saleh in 2011.
Washington regards AQAP as the global jihadist network's most dangerous affiliate and has stepped up drone strikes targeting the militants.