The Islamist movement which runs Gaza is believed to have held Hisham al-Sayed and Avraham Mengistu since April 2015 and September 2014 respectively, after they sneaked into the blockaded Palestinian enclave.
Both men suffer from serious mental health issues, the report said.
Hamas media has claimed the two were soldiers, but HRW said Mengistu was rejected from Israel's military on medical grounds and Sayed was discharged after a few months.
HRW said a third Israeli citizen, Jumaa Abu Ghanima, entered Gaza in July 2016, but there is no further information on him and it is unclear if he was arrested or joined a militant group.
Hamas has never officially confirmed it is holding Sayed and Mengistu, but videos the group has published on social media have included their images.
The Islamist group, which took control of Gaza in 2007, has not released any details on their condition or allowed rights groups to visit them.
Israel bans its nationals from entering Gaza for security reasons.
HRW said Hamas is demanding the release of a number of its members from Israeli jails before it will release information about the men.
"No demand can justify disappearing and bartering over the lives of men, particularly those with serious mental health conditions," said Omar Shakir, HRW director for Israel and the Palestinian territories.
Shakir said Hamas should either release the men or provide evidence of their alleged crimes.
Signup to our newsletter and follow us on Facebook and Twitter!
"While they are being detained they must treat them humanely and provide them with contact with their families," he told AFP.
Sayed's father Sha'aban said his son, a Bedouin Muslim, has schizophrenia and had previously disappeared to Jordan and even Gaza multiple times, eventually being returned by the authorities.
"We just want to ask Hamas why are they holding him? What is the reason?" he told AFP.
"They say he is a soldier, an officer, in the Israeli army. This is a lie."
Hamas did not immediately respond to AFP's request for comment.
HRW said Mengistu, an Israeli Jew of Ethiopian descent, climbed over a fence along the Gaza-Israel border to enter the enclave.
Sayed is believe to have walked across a relatively open area of the heavily fortified border.
Last year, old images of the two men appeared in a Hamas video along with pictures of two soldiers who they claim were kidnapped alive in the 2014 war with the Jewish state.
The Israeli army said the two soldiers were killed, but that Hamas has the bodies.
In 2011 Israel exchanged more than 1,000 Palestinian prisoners in exchange for Gilad Shalit, an Israeli soldier Hamas had detained for five years.
Hamas and Israel have fought three wars since 2008.