White House budget director Mick Mulvaney stood behind President Trump and chief of staff John Kelly's initial defense of White House staff secretary Rob Porter Sunday, saying it was "normal" to give a colleague the "benefit of the doubt."

"Go back and watch what happened this week, and I think it's a very normal reaction to the circumstance," Mulvaney told "Fox News Sunday."

"Someone who we know and trust and work with, Rob Porter, came to the president of United States and the chief of staff and said, 'Look I'm being accused of these things, they are not true,'" Mulvaney continued. "For the president and the chief of staff to give that person the benefit of the doubt is probably a very normal and human reaction."

Porter resigned from his White House post on Wednesday following reports he emotionally and physically abused his two ex-wives, raising questions about why Porter was allowed to operate in the West Wing with only a temporary security clearance given his alleged history of domestic violence.

John Kelly has also been criticized for the manner in which he has dealt with the accusations, at first praising Porter "as a man of true integrity and honor" when the complaints came to light Tuesday.

But Mulvaney reiterated there was no place for abusive behavior in the Trump administration, particularly in his purview of the Office of Management and Budget.

"We don’t tolerate it at all," Mulvaney said, before adding he had not had to deal with the issue in his position as budget director. "You cannot put trust in people who would do that to their spouse of either gender."

In addition, Mulvaney pushed back on rumors Trump was considering replacing Kelly.

"Absolutely not," he said. "I think talk about the chief's departure is much ado about nothing."