CHARLESTON, S.C. -- Sen. Rand Paul of Kentucky is turning down a coveted invite to a gathering of wealthy conservative donors this weekend, citing his plans to be on the campaign trail.

"I'm supposed to be in Iowa," Paul told The Washington Post in a short interview Monday. "We've been invited -- it's just hard to make [these] decisions, because you can't be everywhere."

The weekend conference, being held at a luxury resort in Southern California, is expected to draw hundreds of donors who support a political network organized by the billionaire industrialists Charles and David Koch. Five GOP presidential contenders are scheduled to attend: former Florida governor Jeb Bush, Sen. Ted Cruz of Texas, former Hewlett-Packard chief executive Carly Fiorina, Sen. Marco Rubio of Florida and Wisconsin Gov. Scott Walker. Paul was invited but had not yet committed, Politico reported earlier Monday.

Paul's absence will make it harder for him to gain ground with rich conservatives who were disappointed by his appearance at the last conference in January. During a forum with Rubio and Cruz, Paul's casual attire and meandering answers left some attendees unimpressed.

He still has fans within the Koch-backed network, however. On Monday, he appeared at a town hall in South Carolina hosted by Concerned Veterans for America, a group that receives support from donors in the organization.

Still, the senator from Kentucky appears to be struggling to win over the kind of deep-pocketed backers who are emerging as influential players in the presidential contest. A super PAC supporting his candidacy raised just $3.1 million in the first half of 2015, with two-thirds of the total coming from just two donors.