Fifty-seven members of the parliament, or Knesset, voted to approve the draft legislation in the first of three readings, while 51 were against it.
Supporters of the bill submitted by the party of Prime Minister Benjamin Netanyahu on Wednesday called it a step towards eventual annexation of most of the West Bank.
Some 400,000 Israeli settlers currently live in the West Bank, excluding annexed east Jerusalem, along with 2.6 million Palestinians.
The United States, UN officials and the European Union have warned that continued settlement building is eating away at the possibility of a two-state solution to the conflict.
All Israeli settlements are viewed as illegal under international law and major stumbling blocks to peace efforts as they are built on land the Palestinians see as part of their future state.
Israel differentiates between those it has authorised and those it has not. The bill would legalise nearly 4,000 settler homes built on private Palestinian land.
Key figures in Netanyahu's coalition, considered the most right-wing in Israeli history, openly oppose a Palestinian state.
Education Minister Naftali Bennett, the bill's main backer, has advocated annexing most of the West Bank, like other Israeli religious nationalists who point to the Jewish connection to the land from biblical times.
Netanyahu says he still supports a two-state solution to the decades-old Israeli-Palestinian conflict, but is nonetheless supporting the bill.
He spoke Wednesday with French President Francois Hollande to rule out a proposed meeting with Palestinian president Mahmud Abbas on the sidelines of a peace conference in Paris later this month.
Netanyahu said however that he was willing to meet Abbas "directly, without preconditions," his office said in a statement.
"Israel will not participate in an international conference that will not contribute to peace," it added.
The bill has advanced despite concerns from attorney general Avichai Mandelblit, who has said he will not be able to defend it before the courts and warned it would violate both Israeli and international law.
- 'Profoundly damaging' -
Signup to our newsletter and follow us on Facebook and Twitter!
US State Department spokesman Mark Toner said Tuesday: "We hope that it does not become law.
"Enacting this law would be profoundly damaging to the prospects for a two-state solution.
"We've also been troubled by comments that we've heard by some political figures in Israel that this would be the first step in annexing parts of the West Bank."
Walid Assaf, the Palestinian minister responsible for monitoring Israeli settlements, has called the bill "the most dangerous law issued by Israel since 1967".
Israel occupied the West Bank in the Six-Day War of 1967 and subsequently annexed east Jerusalem in a move never recognised by the international community.
But while comments by Bennett and others have sparked international condemnation, opinion polls have shown a significant percentage of Israelis support annexation.
A recent survey published by the Israel Democracy Institute think-tank says 44 percent of Jewish Israelis support annexing all of the West Bank, while 38 percent are opposed.
Defence Minister Avigdor Lieberman, speaking to EU ambassadors on Wednesday, criticised them for focusing on settlements instead of other global issues.
According to settlement watchdog Peace Now, the bill would legalise some 3,881 housing units.
Most of the homes are in Israeli-approved settlements but were built on Palestinian land. Around 750 are located in outposts which Israel has not yet approved, Peace Now says.
US Secretary of State John Kerry has said there are currently around 100 wildcat outposts in the West Bank and the bill would give retrospective Israeli approval to 54 of them.
An agreement to remove one wildcat outpost from the bill has allowed the legislation to move forward.
The outpost known as Amona, home to around 40 families, is under a court order to be evacuated by December 25 since it was built on Palestinian land.
Some members of Netanyahu's coalition said they could not support the bill if Amona remained part of it because of the court ruling against it.
A solution is still being sought for Amona settlers.