Virginia's legislative elections are four months away but the General Assembly already has been reshaped in a way that could threaten the power base of House Speaker A.L. Philpott.

Party primaries earlier this month brought changes that some party leaders say privately could spell trouble for Philpott, the House speaker since 1980. Longtime Democratic powers -- and Philpott allies -- L. Cleaves Manning of Portsmouth and Claude W. Anderson of Buckingham County were ousted in the primaries.

In addition, Democratic Del. Richard M. Bagley of Hampton, perhaps the most influential House member behind Philpott, had announced earlier that he was leaving the legislature to run for governor in 1989.

The loss of Anderson, Bagley and Manning are seen as blows to Philpott, who could face a challenge for speaker by either current House Majority Leader Thomas Moss of Norfolk or Del. Richard Cranwell of Roanoke. Philpott, who comes from Henry County in the Southside, last mustered his strength in 1982 to scare off a threatened challenge by Moss.

Neither Moss nor Cranwell have made any overt public attempt to gauge Philpott's power this year.

But the combined influence of Manning, Anderson and Bagley, chairman of the key appropriations committee, was far larger than just their three votes in the Democratic caucus that controls the speaker's selection, legislators say.

That increases the chances of a move against Philpott, says one state official close to the speaker. "If they're going to strike, this would be the year," he said. Any challenge, however, probably won't take shape until after the November elections when House Democrats assess their successes and failures, he said.

Even without a direct challenge to Philpott, the loss of the three veteran legislators is expected to lead to a shakeup in committee assignments that will alter and most likely weaken the Democratic powerbase that put Philpott in the speaker's chair.

Republicans, who have dreams of taking control of the House but have not made any gains there since 1982, are hoping to add as many as six seats to improve on their 34-member minority. Democrats, who control 65 seats in the House, label that wishful thinking.

(In addition to the major parties, one member of the House is an independent.)

House Minority Leader Vincent L. Callahan of Fairfax said this week that at best he expects the Republicans to pick up from three to six seats, a change that would not signal significant shifts in the overall House. In the past, some Republicans have predicted the GOP would win control of the House either this year or after the elections of 1987.

"I think we're several elections away" from any potential control of the House by Republicans, Callahan said this week.

Of the 100 House seats, 58 members currently are unopposed for the Nov. 5 elections, according to Democratic and Republican officials.

There are six open seats.

In addition to the defeats of Anderson and Manning and the retirement of Bagley, both Republican W.R. (Buster) O'Brien of Virginia Beach and Democrat Mary Sue Terry of Patrick County are giving up their seats as their party's nominees for state attorney general.

Democrat Floyd Bagley of Prince William County also is retiring from the legislature.

In Northern Virginia, Callahan said Democrats apparently are expected to make their most serious run at Republican Del. Gwendalyn F. Cody of Annandale, but Callahan said he agreed with some Democratic leaders that a candid assessment of the races suggest that few if any incumbents of either party are in serious jeopardy.

State Sen. Clive L. DuVal II said this week the Democrats' major effort this fall is to keep the seat held by Del. Dorothy S. McDiarmid of Vienna, who is in line to succeed Bagley as chairman of the appropriations committee.

"We're raising a lot of money for Dorothy," said DuVal, who is cochairman of her campaign finance committee.

DuVal said the campaign is expected to spend as much as $40,000, considered high for a House race.

DuVal said the Democrats expect to put up strong fights to keep seats where the Republican powerbase is growing in Fairfax County, including seats held by Dels. Vivian E. Watts of Annandale and Gladys B. Keating of Franconia.

DuVal said he believed the Democrats' strongest challenge of an incumbent in Northern Virginia is the race between the GOP's Cody and Democrat Leslie Byrne, a former Fairfax County League of Women's Voters president.

The 40-member Senate is not scheduled to face elections until 1987.

A special election will be called after the November balloting to fill the seat of either Democratic Sen. L. Douglas Wilder of Richmond or Republican Sen. John H. Chichester of Fredericksburg, who are both competing for lieutenant governor.