After a surprise loss to a little-known primary challenger, Virginia Republican Congressman Eric Cantor is stepping down as House Majority Leader.

In his final speech as Majority Leader Thursday, Cantor thanked his colleagues on both sides of the aisle for their service and support. Most notably, he praised Speaker of the House John Boehner for his "patience" while working together during the past five years.

Cantor also extended his support to his "closest confidant and good friend," incoming Majority Leader Kevin McCarthy, R-Calif., saying, "I know your leadership will serve as an inspiration for all of us."

Putting aside political differences, Democratic Whip Steny Hoyer, D-Md., took time Thursday to thank Cantor for his service.

"I always knew he had the best interest of his country, his state, his community, his family and our neighbors in mind when he acted," Hoyer said. "His voice will still be a voice of influence, and he will make a difference."