The wind whipped across the bow of the pilot boat Virginia yesterday as Democratic candidate for governor Gerald L. Baliles, warmed by a gray cardigan sweater, stood topside bracing himself against the choppy waters of Hampton Roads.

"This is my Chuck Robb sweater -- literally," Baliles joked later as he handed it back to Democratic Gov. Charles S. Robb.

"That's right, it's mine," laughed Robb as they headed for the next stop on a two-day statewide campaign swing for Baliles, who faces Republican Wyatt B. Durrette in Tuesday's election.

The sweater was symbolic for Baliles, the former state attorney general whose campaign has attempted to wrap him in Robb's popularity. The link -- dubbed "The Robb Factor" by politicians -- is a key element in the Democratic campaign, and yesterday Baliles and Robb were seeking to make the most of it at every stop.

"The point here is: He and I have been on the same team for four years," said Robb, who under Virginia law cannot succeed himself.

Baliles "has a legitimate claim to the results" of the Robb administration, the governor said during a chartered flight to Richmond.

At a rally attended by about 125 persons at the Amtrak station in Alexandria, Baliles noted that he and the governor had just arrived by rail from Fredericksburg. They were "testing the rails" and, on Tuesday, "would ride the Republicans right up out of this state," Baliles said.

Republicans have chafed at Robb's high popularity -- more than 70 percent of all persons recently polled by The Washington Post said he is doing a good job -- and Durrette has conceded that the governor's success has made the GOP's campaign more difficult. Durrette has pulled in Republican stars, from President Reagan, also highly popular in Virginia, to former governors Mills E. Godwin and John N. Dalton.

Yesterday Durrette campaigned in the Shenandoah Valley with former President Ford, who spoke at a $100-a-person reception and dinner in Winchester.

The event was expected to gross $29,000 for the Republican ticket.

Responding to questions about published reports that some national Republicans have "written off" the Virginia election, Ford said: "That's hogwash. I didn't come here to preach gloom and doom. I came here to preach optimism.

"The fact is, that polls aren't always reflective of what's going to happen."

Although Republicans had hoped to use appearances by President Reagan to counter Robb, Durrette told reporters yesterday that he does not expect President Reagan to make a return visit to the state to promote the GOP ticket.

"We certainly want him," Durrette said, adding, "I never counted on him coming back."

Reagan appeared in Arlington in early October and helped raise about $600,000 for the Republicans.

Robb's role is something relatively new for the Democrats. They have not had a sitting governor on the campaign trail for them since 1969, when Godwin, then a Democratic governor, sought unsuccessfully to keep Linwood Holton from becoming the state's first Republican governor.

Robb, a son-in-law of President Johnson, broke a 12-year string of Republican victories in 1981, and party officials are seeking to make the most of him in the election.

"It's very important anytime you get a governor as popular as Robb," said House of Delegates Majority Leader Thomas W. Moss Jr. of Norfolk.

"If he could run again, he would be elected overwhelmingly. It would be a big political mistake not to use him."

Baliles and his running mates, state Sen. L. Douglas Wilder of Richmond, the party nominee for lieutenant governor, and Del. Mary Sue Terry of Patrick County, the nominee for attorney general, are leading their Republican opponents in most polls, in some cases by wide margins.

"What we're dealing with is clearly a phenomenon . . . if the numbers hold up," said Darrel Martin, Baliles' campaign manager. "No one would have predicted this in June." Durrette has insisted that the race is close, although a recent Republican poll showed him trailing by 13 percentage points.

Robb, who leaves office in January, is widely thought to have national political ambitions, despite his repeated suggestions that he has "no plans" for seeking election. He turned aside such questions again yesterday, saying only that he will return to private business in Northern Virginia.

Moss and other Democratic leaders credit Robb's administration for maintaining fiscally conservative policies and sensitivity to social issues, and for laying the groundwork for this year's political climate.

"Jerry in his own right is a first-rate candidate," said former state Democratic chairman Alan A. Diamonstein, a Newport News legislator who is considered part of Robb's inner political circle.

Robb sought to play down any suggestion of influence he might have over Baliles. "I have tried to be very careful not to look like I'm trying to continue my own influence over state government," he said when asked about his alignment with Baliles. "Jerry has staked out his own campaign. It is his vision of Virginia."

Wilder received some good news in Richmond, where a special grand jury declined to take action on public nuisance complaints about a row house he owns. Radio ads run by his Republican opponent, state Sen. John H. Chichester of Stafford County, have accused Wilder of owning "slum" property in the city.

"We do not find a nuisance, your honor," the grand jury's foreman, Robert Ellett Dandridge, told Richmond Circuit Judge Willard I. Walker on Monday.

The grand jury was appointed after eight persons in the Church Hill neighborhood asked the court to investigate. State law requires appointment of a grand jury if more than five residents of a locality request such an investigation.

The panel could not have indicted Wilder on the charge, but it was empowered to make a "presentment" that prosecutors said would have been the basis for a trial.

Wilder has been charged twice in the last year with violations of the city building code at his property, which is one of eight row houses built in the 1880s. Those charges were dropped after he brought the property into compliance with the code.