Prince Charles has been filling out hius schedule for his return trip to the United States in September. After attending Harvard University's 350th anniversary celebration, he will then travel to Chicago for a charity ball, a department store promotion and a polo match.

Compared with last year's visit, this one should be fairly subdued: Princess Diana, the real attention-getter of the family, will not be along.

And yesterday the British Foreign Office canceled Princess Anne's planned Novermber trip to Peru because of concerns about security. Anne was to make the visit, which also was to include stops in Bermuda and Ecuador, in her capacity as president of the Save the Children Fund. She probably won't visit the other two nations, either, but a final decision has not been made.

A Maoist insurgency in Peru, which has calimed 7,000 lives in six years, has recently heated up. "It was not felt that the trip was viable at this time," the Foreign Office said. Villechaize's Arrest

Actor Herve Villechaize, the 3-foot-11 costar of the old "Fantasy Island" television series, was arrested for kicking and threatening a 6-foot-3 man who tried to serve him court papers from his ex-wife, police said yesterday.

Villechaize, 43, was booked Thursday on suspicion of battery after Eric Borwnson filed a complaint, police Lt. Talbert Kanigher said.

Brownson, 40, of Burbank, told investigators that he and Villechaize's former wife, Donna Hagen, 29, approached the actor on the street to serve papers involving a civil action against him by Hagen.

"Villechaize threw the papers down and stomped on them," Kanigher said.

When Brownson tried to retrieve the documents, Villechaize allegedly grabbed the man's arm, kicked his leg several times and pointed his finger at him, saying, "You're dead." The Pothole Planter

A Burlington, Vt., man identified sumply as the "pothole bandit" has claimed responsibility for planting evergreen trees in the city's potholes. The mysterious bandit, who said he conned mask and gloves to do the deed, contacted a local radio station Thursday to explaon that he was "taking vengeance against potholes because I'm tired of seeing cars wreched."

Police Lt. William Luare, who said officers found trees in three locations, said he did not think the bandit could be charged with any crime unless the trees had been stolen. "We could get him for blocking vehicular traffic, but that's exactly what a pothole does, so we'd be hard pressed on that one. Our big concern is that if he's playing 'caped crusader' in the night he could get run over." End Notes

If he's going to refer to famous personalities, President Reagan might be better off sticking to easily pronounced names like "Rambo." At a meeting yesterday at the White House to urge financial support for Nicaraguan rebels, Reagan quoted the French philosopher Albert Camus, saying, "The spirit is of no avail against the sword but . . . the spirit together with the sword will always win out over the sword alone." Reagan may have lost some of his credibility, however, when he pronounced the existentialist writer's name as "Kay-mus" rather than "Kah-moo" . . .

The Rolling Stones may be going through a midlife crisis, according to band member keith Richards. In an interview with London's Daily Express, the group's lead guitarist said he "can't help feeling miffed" that his colleague, lead singer Mick Jagger, doesn't want to tour in support of the band's latest album, "Dirty Work." Said Richards, "We used to pride ourselves on overcoming a crisis and watching the rest of the bands squabbling." This, the guitarist said, "is the menopause of a rock 'n' roll band" . . .

The 1,500-pound Texas state sesquices tennial mascot -- a longhorn bull with a Texas-shaped marking on its forehead -- died suddenly Thursday. The bull, named Texas U.S.A. suffered a fall lost month but appeared to be recovering under treatment at an Oklahoma clinic. "They gave him water Thursday morning. He drank it and laid his head over and that was it," said Larry P. Smith II of LS Ranches, who lent the longhorn to Texas for the state's 150th anniversary.