There's a theme running among former chiefs of staff -- none of them particularly wanted the job. On Wednesday night, the National Archives and the Aspen Institute brought out five men who formerly held the position -- Donald Rumsfeld for President Ford, Ken Duberstein for President Reagan, Mack McLarty for President Clinton, John Podesta for President Clinton and finally Josh Bolten for President George W. Bush. And of the five, only one admitted to being excited to take the position.

"I was the budget director and a lot of people came up to me after I took the [chief of staff] job and said, 'Oh my gosh, you've been serving in this administration for five years ... you must be completely worn out,' " Bolten told moderator David Gergen. "And I'm thinking, 'thank God I don't have to be the budget director,' " Bolten said laughing.

Bolten also detailed one of the "real treats" he experienced during his time working in the Bush White House. Near the end of his tenure, and before the 2008 election, Bolten suggested that all the living chiefs of staff come have breakfast in his office to impart their wisdom on the poor schmuck who would be taking the job. (He even had the conference table extended by the White House carpenter so they all could fit).

"And sure enough President Obama won, Rahm Emanuel was named chief of staff," Bolten said. "Rahm kindly consented to come to breakfast and said he would behave, which it turned out he did." Joining Emanuel was 13 of the 16 living former chiefs of staff, including former Secretary of State Rumsfeld and Vice President Cheney, who, like Rumsfeld, held the position under President Ford.

"I seated Rumsfeld at one end, Cheney at the other, Rahm Emanuel in the middle, and we went around for advice," Bolten recalled. Rumsfeld noted that it was important to pick a successor early and make sure the President approved of that person. "And then we went around the table and got some other bits of advice and we got to Vice President Dick Cheney," Bolten said. "And there was a long pause and he says, 'Ehhhh ... watch out for the vice president.' "