Segregation and social exclusion are at “worrying levels” and are fuelling inequality in some areas of Britain, a report has found.
Women in some communities are denied “even their basic rights as British residents”, the Casey Review said.
Dame Louise Casey accused public bodies of ignoring or condoning divisive or harmful religious practices for fear of being called racist.
Communities Secretary Sajid Javid said he would study the findings “closely”.
Dame Louise’s review into the integration of minorities was commissioned by former Prime Minister David Cameron as part of the government’s efforts to tackle extremism.
Among her recommendations were that immigrants could take “an oath of integration with British values and society” and schoolchildren be taught British values.
Her review said there was a sense that people from different backgrounds got on well together at a general level, but community cohesion “did not feel universally strong across the country”.
She found “high levels of social and economic isolation in some places, and cultural and religious practices in communities that are not only holding some of our citizens back, but run contrary to British values and sometimes our laws”.
Her report highlighted the plight of women in some Muslim communities, who she said were less likely to speak English and more likely to be kept at home.
“Misogyny and patriarchy has to come to an end,” Dame Louise said, adding that public institutions must not fear being racist or Islamophobic.
Faeeza Vaid, from the Muslim Women’s Network, said migrant communities should not be blamed for failing to integrate.
“We also see segregated white communities,” she told the BBC’s Victoria Derbyshire programme.
“Integration is everyone’s responsibility.”
Halima Begum, a Muslim woman from London, said she was “worried” that the report singled out Muslim men.
“There is a problem with men and I see it,” she told the BBC’s Asian Network. “We need to have an open debate without placing too much blame on communities.”
Muslim areas under spotlight
By BBC political correspondent Ross Hawkins
Afraid of being dubbed racist, afraid of losing support, afraid of challenging minority communities – that is Dame Louise Casey’s view of Britain’s decision-makers.
Criticising politicians and officials is the easy bit. The significance of this report is that it targets individual communities and faiths.
It contrasts with the language of people living “parallel lives” – a term used in an earlier report and designed to be neutral, placing no more blame on one community than another.
Dame Louise makes clear her outrage at what she calls “regressive practices” targeting women and girls. She accepts she is putting Muslim areas under the spotlight.
The question for Dame Louise – an official not immune to criticism herself – is how would she achieve change? What if an elected politician refused to take her oath? What if some people – of whatever community – simply prefer to live and educate their children separately?
Iqbal Bhana, a government adviser on anti-Muslim hate crime, rejected the report’s claim that Britain was becoming more segregated.
“I don’t think we are divided,” he told BBC Radio 4’s Today programme.
“If you go to places like Bradford or Leicester you see second and third-generation people moving out of some of these communities.”
During her research, Dame Louise spoke to women in areas of Birmingham and Manchester that she said could not leave their homes without their husband’s permission.
She blamed those in authority for “ducking the issue” of women’s inequality out of a wish to respect different cultures.