(CNN) – South Carolina Sen. Lindsey Graham has always enjoyed a little back-and-forth with belligerent audiences. He was at it again on Monday night as he faced down an angry town hall crowd in Greenville packed with libertarians and Tea Party activists who accused at the Republican senator of ditching conservative principles by working with Democrats on issues like climate change and voting to send Sonia Sotomayor to the Supreme Court.
But Graham stressed a mantra he's repeated many times since his friend John McCain lost the presidential election last November — that the GOP must reach out to different constituencies, or face extinction.
"I'm not going to leave the Republican Party," Graham said when one questioned asked him why he hasn't yet joined the Democrats. "I'm going to grow it. We're not going to be the party of angry white guys."
His comments were met with a salvo of boos and shouts of "Ron Paul!"
"I love this party," he responded. "I'm not going to be let it be hijacked by Ron Paul."
Graham emphasized his conservative bona fides — particularly his record of opposing abortion rights and support of gun rights — but said GOP needs to build a broader coalition. Otherwise, he warned, those trying to purify the party will turn Congress over "to the most liberal people in the world because somebody disagrees with you."
"I'm going to find people in Maine, Delaware, Illinois and other places that can win," he said. "And I'm going to help them, and we're going to move this party and this country forward. If you don't like it, you can leave."