Officials had yet to confirm if anyone had been killed, but Tehran mayor Mohammad Bagher Ghalibaf told state television that between 20 and 25 firefighters were thought to be inside when the building fell.
Some 200 had been tackling the blaze at the Plasco building, which dated from the 1960s and included a shopping centre and clothing workshops.
The head of Tehran's emergency services told state TV that at least 70 firefighters had been injured and 23 hospitalised as they rushed to evacuate the building.
"I was inside and suddenly I felt the building is shaking and is about to collapse. As we gathered colleagues and got out, a minute later the building collapsed," said Ali, a firefighter at the scene.
Dramatic images showed flames pouring out of the top floors of the building just before it came down.
"The building's caretaker and some firefighters were inside when the building collapsed," said Ahmad, the owner of one of its shops.
"I've lost my entire stock. Thousands of families have been ruined," he added.
The steel skeleton of the building could be seen bending down to the ground as around 100 fire engines and dozens of ambulances surrounded the area.
"A friend of mine has a shop there. I keep calling him but there's no answer. I think he's been trapped," said Mohsen, an onlooker.
President Hassan Rouhani called for an immediate investigation into the causes of the incident, calling it "unfortunate and sorrowful".
Signup to our newsletter and follow us on Facebook and Twitter!
"More than 30 times we warned the building's owners that it was not safe, but unfortunately they did not pay attention," said Shahram Gilabadi, spokesman for the municipality.
Tehran's first high-rise
Fire brigade spokesman Jalal Malekias said the building was known to breach safety standards.
"Even in the stairwells, a lot of clothing is stored and this is against safety standards. The managers didn't pay attention to the warnings," he told state television, adding that it lacked sufficient fire extinguishers.
The Plasco building contained some 400 business units as well as workshops.
It was the first high-rise and shopping centre in Tehran and was the city's tallest building when it was finished in 1962, before being dwarfed by the construction boom of later years.
It was built by Habibollah Elghanian, a prominent Iranian-Jewish businessman who was arrested for ties to Israel and sentenced to death and executed after the 1979 Islamic revolution.
The fire is thought to have begun on the ninth floor and spread quickly to workshops above.
Firefighters were initially able to bring it under control but it quickly flared up and the building fell four hours after the fire had started, at around 11:30 am (0800 GMT).
A crisis meeting was being held at the scene with Tehran's governor, police chiefs and officials from the Red Crescent. The army was also deployed to assist with the rescue effort.
"There were a number of people inside but we don't know how many and the fire brigade organisation is going to announce how many were there," Tehran police chief Hossein Sajedinia told AFP.
"Even one would be too many," he added.