Two Virginias who worked in President Carter's 1976 campaign have begun organizing a movement in the state to draft Sen. Edward M. Kennedy as next year's Democratic presidential nominee.

The action by James Gibb of Alexandria and Dr. Robert R. Richards of Virginia Beach makes Virginia the 20th state where a "Draft Kennedy" organization has been formed. Gibb and Richards filed papers with the Federal Election Commission a week ago for their Kennedy group, called "Virginia Democrats for Leadership and Commitment."

Both men, while expressing respect for Carter, said they were switching to Kennedy because they are dissatisfied with Carter's performance as president.

"He hasn't been able to get a grasp of federal government and make it work," said Richards, a professor in the state extension service run by Virginia Polytechnic Institute and State University. "We're looking for someone who has tried and true leadership abilities on a national level, and Kennedy has it."

While Richards and others are channeling all their efforts to promoting a Kennedy candidacy, many Democratic leaders in the state say they think the Massachusetts senator is too liberal to attract much broad based support in the conservative Old Dominion.

"Those people in Virginia who have publicly announced their support of Ted Kennedy couldn't organize a two-car funeral," was how one district chairman in the party was quoted as saying.

Kennedy's chief support is said to be coming from Northern Virginia, where liberal Democrats have had a following in the past and where a large number of federal workers reside.

Similar groups committed to Kennedy's candidacy have been formed in the District of Columbia and Maryland.

D.C. has two committees. The D.C. Committee for a Democratic Alternative, headed by Mary Ann Keefe of Ward 3 and Barry Campbell of Ward 4, filed with the FEC on Sept. 5. "Since Kennedy has sent out smoke signals, we are completely Kennedy," said spokesman C. Grove Smith.

Another group, the D.C. Kennedy for President Committee, filed with the FEC in late July and is headed by Community activist Norman C. Neverson and political fund-raiser Mark Plotkin.

The Maryland for Kennedy Committee filed with the FEC on Monday and will hold its first statewide coordination meeting Oct. 14, according to Jack Bregman, the group's head.

Richards said yesterday that those who think Kennedy is too liberal to garner much backing from Virginias should be answering his telephone these days.

News of the Kennedy committee "broke Sunday, and since then I've been answering phone calls from people who want to get on the bandwagon and help out," said Richards.

Plans call for the draft Kennedy organizers to get under way in earnest after Virginia's Nov. 6 local government and legislative elections. Kennedy supporters raised more than $2,000 at an Alexandria dinner last month for their New Hampshire counter-parts, who are readying for a crucial primary there early next year.

Richards said he was amazed at how callers have echoed his own feelings about Carter and his decision to work for Kennedy's nomination. "They're disappointed in Carter now," he said. "They like him personally, but they don't think he's doing a good job as president."

What they want instead, Richards said, is "someone with that magic combination to make people respond."

Though Virginia backed Ricahrd Nixon over John F. Kennedy nearly 20 years ago, Richards insists that voters in the state will take more warmly to a Kennedy candidacy this time.

"This state almost elected Henry Howell," he said, referring to the gubernatorial contest that was won six years ago by Republican Gov. Mills E. Godwin. "The people of Virginia aren't looking for labels, they're looking for leadership."