"We see them making gains on the ground there as they try to take territory," said Carter, who was in Japan as part of a visit to Asia for talks with regional allies.
Yemen has descended into violence over recent months, with Huthi rebels seizing power in the capital Sanaa in February.
The Huthis, allied with army units loyal to ex-president Ali Abdullah Saleh, have been fighting forces supporting President Abedrabbo Mansour Hadi, who has fled to the Saudi capital Riyadh.
Late last month Saudi Arabia launched a campaign of air strikes, amid fears Yemen will slip into Huthi control and shift into the orbit of Shiite Iran, Sunni Saudi Arabia's regional rival.
Signup to our newsletter and follow us on Facebook and Twitter!
Observers say Al Qaeda and other groups are exploiting the instability, in which the World Health Organisation says at least 540 people have died since March 19.
"The terrorism threat to the West, including the United States, from AQAP (Yemen-based Al-Qaeda in the Arabian Peninsula) is a longstanding and serious one (..) that we will keep combating," he added at a press conference alongside his Japanese counterpart, Gen Nakatani.
"Obviously it’s always easier to conduct CT ops when there is a stable government willing to cooperate.
"That circumstance now obviously doesn’t exist in Yemen but that doesn’t mean that we don’t continue to take steps to protect ourselves. We have to do it in a different way, but we do and we are."
Carter expressed hope that peace would be restored "not only for that reason but also (because) there is a lot of suffering in Yemen".
At the end of last week AQAP, which the US views as the most dangerous wing of the Sunni Muslim extremist group, captured the army headquarters and the southeastern port of Al Mukalla.