The rally was organised by Israel's Peace Now settlement watchdog and held across the street from the residence of Prime Minister Benjamin Netanyahu.
Peace Now said the rally was "an opportunity to let the democratic camp's voice be heard" against the draft law, which embodied "nationalism, racism and aggression" threatening to "ruin our country."
As the demonstration got underway, police said that arsonists had torched an Arab-Jewish school in Jerusalem and scrawled racist anti-Arab slogans on its walls.
"Go home, release us from your oppressive, racist, extremist and inciting regime," MP Tamar Zandberg of the opposition Meretz party said at the demonstration.
Critics say the new bill -- endorsed by the cabinet on Sunday -- will come at the expense of democracy and institutionalise discrimination against minorities, including Arabs.
Netanyahu insists the law would balance Israel's Jewish and democratic characteristics.
Protesters held signs reading "we won't let you ruin the country" and "the nation-state law of the right-wing government is democracy for Jews only" in the crisp evening.
Police said approximately 800 people attended the demonstration.
Signup to our newsletter and follow us on Facebook and Twitter!
Meanwhile firefighters extinguished the fire at the "Hand-in-Hand" school, a short distance from Netanyahu's residence.
Slogans such as "death to Arabs" and "no coexistence with cancer" had been scrawled on the walls of the Arab-Jewish school, police said.
Several Israeli officials condemned the arson attack.
Education Minister Shai Piron described it as a "violent and despicable incident, which could undermine the foundations of Israeli democracy".
It is "a serious affront to the fabric of Jewish-Arab relations," he said in a statement.
Jerusalem Mayor Nir Barkat said: "We won't let pyromaniacs and radicals disrupt public order."
"We will continue to denounce extremists and do everything necessary to restore security," he said in a statement.
The school has been targeted with racist graffiti in the past, most recently during Israel's summer war with Hamas militants in Gaza.