The site is believed to be a church or cathedral dating back to at least the 7th century.
The Gaza ministry of antiquities and a number of prominent archaeologists called for all building to be suspended until the relics can be protected and preserved.
The ministry of endowments and Islamic affairs, however, owns the land and wants the commercial development to continue, with bulldozers working on Tuesday despite protests.
Hyam al-Betar, an archaeologist who works with the antiquities ministry, screamed at bulldozers to stop on Tuesday as they roughly moved marble columns from under the sand, breaking one.
Technicians from the ministries of antiquities were hurriedly taking columns, as well as ornate marble bases, to the Qasr al-Basha -- the only museum in Gaza -- to be cleaned and restored before being presented.
Mohammed Al-Zarad, a researcher in archaeology at Gaza's Islamic University, said it was a "very important site which must be protected."
"We found many of the rock layers had fossils in them dating back to the bronze age," he added, with other discoveries including plates and pottery – some of which were perhaps smashed by bulldozers.
Signup to our newsletter and follow us on Facebook and Twitter!
Zarad said there were not enough resources to search for relics within Gaza and called on the world heritage body UNESCO to step in and "save the monuments of Gaza from loss."
Jamal Abu Raida, from the ministry of antiquities, said the pieces found include a marble column engraved with leaves, pillars and a foundation stone bearing a Greek Christian symbol.
Gaza, like much of the Palestinian territories and Israel, is filled with antiquities.
The territory has at various times come under the rule of Romans, Byzantines, Crusaders, Mamluks and Ottomans.
Betar said it was important to try to find a solution.
"We extend our hand to all to cooperate in research into Gaza and its history because Gaza is one of the oldest cities in the world."
"There are treasures beneath our feet," she added.