Rouhani was elected last year having pledged to be more moderate on social issues after his conservative predecessor Mahmoud Ahmadinejad's eight-year tenure, but he has encountered resistance.
Iran has a policy of filtering online content, which leaves popular websites such as Facebook, Twitter and YouTube inaccessible without the use of illegal software.
Iranian authorities are also often accused of deliberately slowing down the Internet to make many websites harder to access.
The decision of Rouhani's government to approve faster 3G mobile Internet licences for two Iranian companies last month was seen as a first step toward making Internet access easier.
But he went further Sunday and said filtering was counter-productive.
"Some people think we can fix these problems by building walls, but when you create filters, they create proxies," Rouhani said, referring to proxy servers in other countries used by Iranians to circumvent national regulations.
"Iranian authorities are often accused of deliberately slowing down the Internet"
Signup to our newsletter and follow us on Facebook and Twitter!
"This (current policy) does not work. Force does not produce results," he added, in a speech broadcast live on state television.
The 3G licenses decision caused controversy with conservative clerics and officials arguing that video call functions on smartphones could expose youngsters to "immoral content".
Iran's ministry of telecommunications, technology and information later stated that video calling would not be available, despite such services -- including FaceTime and Skype -- being accessible on regular Internet connections.
Rouhani also hit out at the recent decision of Tehran's municipality to segregate staff by gender, a step implemented by the capital's mayor and hailed by conservatives.
Referring to those who "constantly say we must separate girls and boys," he said the Islamic republic's founder, Ayatollah Ruhollah Khomeini, vehemently opposed universities adopting such a policy after the 1979 revolution.
However, Rouhani defended the veil and use of hijab, which women use to cover their hair and the loose robes they wear to cover their body, saying that it protects women.
Since Rouhani took office last August he has been accused of being too soft on hijab -- in October he asked police to be moderate when enforcing the requirement.
Recent years have seen many women wear a thin veil that barely covers the hair and tight clothing or coats reaching mid-thigh instead of the long coat or chador, prompting MPs to write to Rouhani to demand tougher police action.