The Cabinet minister insisted it is a safe place for women to work as he backed the resignation of Tory colleague Neil Parish for twice watching pornography in the Commons.
Commons Speaker Sir Lindsay Hoyle is calling for “radical” reform to working practices after a string of bullying and sexual misconduct offences involving MPs.
They have triggered women working in Westminster to share accounts of their treatment, with Cabinet ministers describing men acting like “animals”.
Senior Conservative Caroline Nokes, who chairs the Women and Equalities Committee, has accused the Tories of “institutional sexism”.
But Mr Kwarteng, speaking on Sky’s Sophy Ridge on Sunday, denied there is a culture of misogyny at the centre of British democracy.
“I don’t think there is a culture of misogyny; I think the problem we have is people are working in a really intense environment, there are long hours and I think generally most people know their limits,” he said.
He told the BBC’s Sunday Morning show that Parliament is a safe place for women to work, adding: “I think we’ve got to distinguish between some bad apples, people who behave badly, and the general environment.
“There are some bad apples, there are people who have acted very badly, and they should be held to account.”
The Sunday Times carried a report describing drunken and disorderly behaviour, including a senior MP accused of repeatedly licking the faces of researchers in parliamentary bars.
A minister being overheard frequently having “noisy sex” in his parliamentary office, an MP being warned over his use of prostitutes, and a female Tory being sent a “dick pic” by a colleague were also detailed.
Mr Kwarteng accepted that the allegations were “extraordinary and unacceptable” but ruled out closing Parliament’s many bars to tackle the sleaze.
“No, they shouldn’t all be shut, I don’t think we should have an excessively puritanical, severe regime in that regard,” he told Ridge.
Senior Tories have been pushing to get more women MPs, but Mr Kwarteng said he is not “a fan of quotas” to boost their numbers.
Calling for urgent action, Sir Lindsay suggested staff should no longer be employed by the parliamentarians they work for to address a series of “serious allegations”.
He said he is considering an outside body employing aides as he moved to establish a “Speaker’s conference” bringing MPs together to discuss an overhaul.
His bid for change was echoed by former Commons leader Andrea Leadsom, who in 2018 spearheaded the creation of the Independent Complaints and Grievance Scheme (ICGS), which looks into allegations of bullying and sexual harassment.
“Things haven’t changed and that’s because there aren’t enough cases coming through and it’s taking too long for investigations to come to an end,” she told the Sunday Times.
“It’s only when you see people getting done for being blind drunk and subject to the appropriate sanctions that people will start to think twice about their behaviour.”
Labour leader Sir Keir Starmer said he will participate in the Speaker’s bid, and called for “political leadership” from Prime Minister Boris Johnson to fix the problem “because the fish rots from the head”.
Cabinet minister Anne-Marie Trevelyan this week described once being “pinned up against a wall” by a former MP, while colleague Suella Braverman said some men act like “animals”.
On Saturday, Mr Parish announced his resignation as MP for Tiverton and Honiton after two colleagues reported having seen him watching porn.
The farmer by trade, who chairs the Environment, Food and Rural Affairs Committee, said he first accidentally viewed porn in the Commons chamber after looking at tractors online, before later doing so deliberately.
His departure will pave the way for a by-election in the Tory safe seat and he still faces an investigation by the ICGS.
It was the latest in a series of damning developments that have dogged the Commons in recent weeks.
Three Cabinet ministers are among 56 MPs reportedly facing allegations of sexual misconduct that have been referred to the Independent Complaints and Grievance Scheme.
Senior Labour MP Liam Byrne is set to be suspended from the Commons for two days for bullying a member of staff.
David Warburton had the Conservative whip withdrawn after allegations of sexual harassment and cocaine use emerged.
Former Tory Imran Ahmad Khan resigned after being convicted of sexually assaulting a 15-year-old boy.