The sexual indiscretions of Michael Kelly, D.C. Mayor Vincent Gray’s choice to lead the District’s housing department, while he ran the Philadelphia Housing Authority threatened to obstruct the troubled Pennsylvania agency from moving forward, a federal official said Wednesday.

“Just the fact that you have a whistleblower calling in illustrates that there is a perception issue,” said Jerry Brown, a U.S. Department of Housing and Urban Development spokesman. “When you’re talking about a housing authority that is in receivership where the goal is to rebuild morale and to show the team that everything is aboveboard, this wasn’t the right direction.”