But this weekend, it shouldn't be too hard to find a Bush in town.
The extended clan is descending on this coastal village to mark Barbara Bush's 90th birthday and the 91st birthday of former president George H.W. Bush, which is Friday. Golfing, tennis and some yoga is likely in order, though the former president isn't expected to jump out of an airplane this year, according to a spokesman.
Instead, the formal celebration includes a lobster luncheon on Monday to mark Barbara Bush's birthday. Her literacy foundation also has been raising money to mark its 25th anniversary.
This year's celebrations are drawing more attention than usual as Jeb Bush, the family's second-oldest son, prepares to launch his presidential campaign on June 15. His plans mean this coastal town is likely to endure a new wave of political tourism -- the third wave in roughly three decades.
"It’ll be phenomenal for business," said Steve Kingston, owner of the Clam Shack, a lobster and clam restaurant and market located on the bridge connecting Kennebunkport to neighboring Kennebunk.
Walker's Point, the Bush family compound, is what visitors come to see. It's about two miles south of the town center along Ocean Ave. and has been in the family since the late 19th century, when the 41st president's maternal great-grandfather, David Davis Walker, and grandfather -- also named George Herbert Walker -- purchased the estate.
Primary residents George and Barbara Bush are usually in town six months every year, flying the Texas flag below Old Glory to signal their presence. Back in the day, the former president would be frequently seen out on the water aboard his beloved cigarette boat, "Fidelity." Former president George W. Bush and former first lady Laura Bush are also frequent visitors.
As president, George W. Bush visited here 11 times spanning all or part of 43 days, according to records kept by CBS News White House correspondent Mark Knoller. His most notable visit came in July 2007, when the Bush family hosted Russian President Vladimir Putin at a rocky moment in U.S.-Russian relations. Locals dubbed the meeting the "Lobster Summit."
Kingston catered the summit -- and is also preparing lobsters for Monday's birthday lunch. Reporters visiting town for the Bush-Putin summit were so intrigued by the seafood menu that Kingston agreed to sit for live interviews with Russian television and Al Jazeera.
During the first Bush presidency, Kingston's ice cream shop up the street was a popular choice for visiting reporters. "I used to serve [then-ABC News White House correspondent] Brit Hume coffee," he recalled. "It was horrible coffee, absolutely old terrible stuff and he said it was just like it came from the newsroom."
During the second Bush presidency, then-NBC correspondent David Gregory visited frequently. "We used to ship lobsters to his wife’s doctor after she gave birth," Kingston said. "He was a good customer during that period."
"It's kind of fun to think that more of you will be coming up again," he said to yet another visiting reporter.
But Kingston -- and many other locals who didn't want to give their names -- said they don't know much about Jeb Bush. Kingston said he knows Bush's children better.
"Jeb Jr. loves my lobster roll, so I know him face-to-face," he said. "But Jeb, I’m not sure."
The former Florida governor doesn't visit Kennebunkport as frequently as his siblings. He's in town this weekend along with his wife, Columba, and his own growing family. But on Sunday, he drove about 45 minutes south into New Hampshire, to attend a Sunday Catholic church service in Portsmouth, N.H., aides said. On Monday, he leaves for a week-long trip to Germany, Poland and Estonia -- a journey designed to bolster his foreign policy portfolio.
On previous visits, he's been known to entertain at the family home.
Joseph A. Califano, the famed New York attorney who served in Jimmy Carter's cabinet, recalled visiting Walker's Point in 2009 when his Maine vacation coincided with Jeb and Columba Bush's annual visit. The couples first met when Columba Bush joined the board of Califano's National Center for Addiction and Substance Abuse. That night, they all agreed to meet for dinner, starting the evening with cocktails at the family compound.
"We had a drink in the living room," he recalled. "The house is built over a pile of rocks with three sides of glass. A spectacular view. We had a drink and were there for about 90 minutes. We talked with Jeb, Columba, George H.W. and Barbara. They were terrific, and we talked about everything in the world."
At one point, Califano's wife, Hilary, piped up and urged Jeb Bush to run for president in 2012. Barbara Bush interjected.
"Hilary, the country is all Bushed out," the former first lady said, Califano recalled.
A Bush family spokesman declined to comment. But in the years since, Barbara Bush has retracted such comments and is fully supporting her second-oldest son's presidential aspirations.
Out at Walker's Point this past week, the compound was bustling with activity. Construction workers could be seen working on a new, $1.4 million home being built in Jeb's name. The new construction dwarfs other nearby cottages. Aides said that it will be occupied by other relatives and guests when Jeb isn't in town.
About a quarter mile to the west of the compound is an area for gawkers to pull over and park for 15 minutes. This week, cars with license plates from Massachusetts, New Hampshire, New Jersey, New York and Quebec pulled in. Couples posed for photos with the compound as the backdrop. One elderly couple quizzed a passing jogger about the neighborhood. A tourist trolley breezed by with passengers snapping photos.
At the site, there's also a placard and anchor comprising a modest memorial dedicated to George H.W. Bush by the Kennebunkport Conservation Trust.
It reads: "As he was for our nation and world during four years of tumultuous and historic change, so, too, has Kennebunkport served, in the words of St. Paul, 'as an anchor of the soul, both sure and steadfast' to him."