Virginia Gov. Charles S. Robb has made preliminary plans to build a state prison in southern Fairfax County, angering local officials who said the governor never told them of his efforts.

Northern Virginia Rep. Stanford E. Parris yesterday revealed the plan, releasing a copy of a July 15 letter from Robb to President Reagan asking for 107 acres in Fort Belvoir for a facility to house convicted felons. State officials later confirmed they have identified the wooded federal property as an ideal site for a prison.

Parris, a Republican who is fighting for his political life against former Democratic Rep. Herbert E. Harris II in next Tuesday's election, labeled the proposal "dangerous and dumb" as well as "ill-conceived and idiotic." He faulted Robb, a Democrat, both for proposing to build a prison near densely populated suburbs and for pursuing the idea without consulting local elected officials.

Fairfax Democrats said the timing of Parris' well attended press conference in his Capitol Hill office was clearly political, but the revelation seemed to leave them without much stomach to fight back. Democratic County Supervisor Sandra L. Duckworth, an ally of Robb and Harris, said she had no idea Robb was proposing a prison for her Mount Vernon district.

"My immediate reaction was, 'Good grief!' " Duckworth said. "Of course I don't support it, and I'm very disappointed that he Robb didn't get in touch with me and the county Board of Supervisors."

Before any prison could be built, the General Services Administration would have to agree that Virginia has a stronger claim to the site than several federal agencies that have also expressed interest in it, Parris said. The GSA and the state would then have to negotiate a fair price for the land, and Virginia's General Assembly would have to appropriate funds for construction, said state Sen. Adelard L. (Abe) Brault.

"I'm not going to support such a thing if the people of Fairfax don't want it," said Brault, Northern Virginia's senior Democratic legislator who said he was unaware of the proposal until yesterday.

Harris, who served in Congress from 1974 until 1980, managed to oppose Robb's plan with a twist of his own. "The apparent site that is being considered by the Reagan administration I think has no merit whatsoever," Harris said.

Asked whether the issue could damage him politically, Harris jovially faulted Parris, against whom he is running for the third time, for learning of the plan three months after Robb proposed it. "I think the Reagan administration should do a better job of keeping the congressman informed of this kind of thing," Harris said.

The District of Columbia owns and manages a prison at Lorton in southern Fairfax, near the proposed Fort Belvoir site, but, except for work camps like the 150-prisoner facility near Fairfax City, Virginia has no prisons north of Richmond. Since at least 1970, state plans have called for a prison in north Virginia because about 15 percent of the state's prisoners come from Alexandria and Fairfax, Arlington, Prince William and Loudoun counties.

Still, local officials, such as Fairfax Board Chairman John F. Herrity, a Republican, said they did not know until yesterday that the state had focused on one site. The 107-acre piece of Fort Belvoir, at the intersection of Route 1 and Telegraph Road, has been declared surplus by the federal government.

In his July letter to Reagan, Robb said the site would be close enough to inmates' homes to allow family visits and is heavily wooded to allow buffering and protection for nearby neighborhoods. "Virginia corrections has a reputation for its low escape rate and for providing protection to citizens located near our penal institutions," he wrote.

Virginia Secretary of Public Safety Franklin White and spokesmen for the governor and the state corrections chief all said yesterday the site remains a candidate for a prison, although the federal government has made no final decision on selling the land. All three state officials said they did not know why local officials were not told of the proposal.

Parris said he would urge Reagan not to allow the GSA to transfer the land to the state for use as a prison. In a letter to the president released yesterday, Parris said the site is not only "in the middle of Virginia's major population center" but is also in a historic district. George Washington's Pohick Church is across the street and the world's first telegraph line was constructed nearby.

Parris recently proposed relocating the District's Lorton facility to a site inside the District, which he acknowledged would require building a prison within a major population center.

"But the District's problem is the District's problem, and the District penitentiary is to serve residents of that city," Parris said. "In Virginia, you've got lots of rural areas . . . "

Parris said the timing of his revelation was unrelated to next week's election. He said his staff discovered Robb's letter only last week while searching government files to find surplus land in the District where the District's prison might be moved. "I did not select the subject, nor the location, nor the timing of this proposal," Parris said.

Democratic state Sen. Brault said Parris is using the letter "as an 11th-hour thing to try to get people stirred up."