Sen. Lamar Alexander, R-Tenn., on Tuesday added his voice to those calling for the removal of a bust of Nathan Bedford Forrest from his state's capitol building, a bust he once defended when he was governor of the state.

Alexander told those attending an event in Nashville that the state's capitol was not an appropriate venue to house the bust of Forrest, a lieutenant general in the Confederate Army who was also an early member of the Ku Klux Klan and believed to be a Grand Wizard.

"As I look at it now, I think it's not appropriate for Gen. Forrest's bust to be in the place of honor in the state capitol," Alexander said. "But for a place of honor, such as the place in the state capitol, between where the Senate and House meet, that's a place that ought to be reserved only for Tennesseans who inspired all of us and bring out the best in all of us."

The Tennessee senator said battlefields, museums, and birthsites are more appropriate places to remember the Civil War and urged the country and state to spend more time learning from history than trying to erase it.

"All the people of this country were involved in the Civil War at that time," Alexander added. "There's a place to remember all of it."

Alexander's current tone marks a shift from when he was governor and told protesters seeking to remove the bust at the time, "There are a lot of things we don't like in our past, but that's not a good reason to remove the bust."

He said a lot has changed since he made that statement in 1979, and pointed to the appointment of the first black state Supreme Court justice and approving a holiday to honor Martin Luther King Jr. – all milestones that occurred during his tenure as Tennessee's governor.

Alexander also criticized a newly reformed white supremacist group that plans to protest the possible removal of a Confederate statue in Knoxville this weekend.

"They have a constitutional right to assemble," Alexander said. "Their views are wrong; their views are not welcome. We need to say that loud and clear."