In the continuing drama of racial politics that has turned President Carter's Georgia hometown church into a shambles, Plains Baptist Church yesterday voted against rehiring the Rev. Bruce Edwards - a friend of the President who was forced to resign as pastor in February.

But the fight that has, according to many in Plains, "split this town wide open and ruined the church," did not end with yesterdays 87-to-51 defeat of the motion to rehire the pastor. Pro-Edwards and Carter forces, led by state Sen. Hugh Carter, the President's cousin and a church deacon, are considering breaking off and starting a new church. They will vote on such a plan next Sunday.

"Jimmy's side has been defeated. It looks like we can no longer worship here in this attitude of hatred," said Hugh Carter.

The White House had no comment on the situation.

The rift, which has received world-wide attention, began last fall after a black minister from nearby Albany, Ga., tried to integrate the church just before Jimmy Carter was elected President. The congregation invoked a 1965 resolution barring blacks to keep him from the church. Carter and Edwards vigorously opposed that action.

Yesterday, charging that the opposition to Edwards "anti-black" and anti-Jimmy," Hugh Carter said Edwards was "crucified" in a meeting stacked with voters who never go to church. "We couldn't win because our people can no longer vote; they've quit the church in protest, said Hugh Carter.

"Probably 40 to 50 people who never come to churh except when a vote is taken were there today," Edwards said after the vote.

Church treasurer Clarence Dodson noted that since Edwards left, many active members had quit, offerings had dropped drastically and the church was "inreal trouble" financially.

During yesterday's emotional debate over the fate of Edwards - who sat through the meeting with his wife, Sandra - Marlie Carter, the teenage daughter of the President's brother Billy, said many of the young people had quit because "we do not respect our adults" who fought to oust Edwards.

The Plains Baptist Church, a simple, white-frame structure with beautiful stained-glass windows, ironically looks the very symbol of small-town serenity. Its current racial troubles go back 11 years to when church deacons voted to bar "Negroes or any other civil rights agitators" from attending services or becoming members. That vote would have been unanimous except of the opposing votes of Jimmy Carter and his family.

Last fall deacons invoked the ban to bar the Rev. Clennon King, a black nondenominational minister.

Edwards, like most in this hamlet of 683, thinks King's struggle to enter Carter's church was a Republican-inspired political setup but said yesterday, "You have to respond in a morally responsible way even if that act was crooked. I could have stopped King but that would have been immoral."

After Carter was elected President, he returned to Plains and helped overturn the 1965 ban on blacks. The church later turned down King's application and to date has approved no black members. Resentment toward Carter and Edwards continued to fester and in February, at a conference ostensibly called to rule on paying a $300 bill, Edward's critics called for him to resign.

Pro-Edwards members charge it was an "underhanded, stacked" session. A. O. Williams, a major peanut competitor of the Carters and critic of Edwards, raced down the aisle, according to several in attendance, and threatened, "If y'all don't vote on this today, I'm never coming back." Edwards said that to prevent national embarassment, he resigned rather than be fired.

George A. Harper, a leader of Edward's opposition, still contends that a resentment of "politics injected into the church" was their only motivation. "Carter polarized the church by using the church in his campaign. He would call ahead of time and thell them when he wanted to teach Sunday School, and the papers and TV would be there, and all that about King appealed to the black vote, of course. Politics has no place in church," Harper said.

Those who refues to be quoted are more acerbic and speak in racial terms. One man sitting in Billy Carter's gas station said, "I ain't going to give you my name, but Carter and Edwards tried to integrate. We don't believe in that."

A deacon strongly opposed to Edwards said, You Know, he adopted that baby - I don't call it white and I don't call it black. I call it gray. Now most folks don't hold to that sort of thing."

Edwards, 30, and his wife, adopted a half-Polynesian boy, Philip, last October. For the Edwards family, the past months have been troubled and sad. They still live in the white-frame house behind the church (his resignation is effective April 30) but they no was blacked off the mailbox after longer go to church and their name "nasty words" were written on it.

There has been hate mail. Some neighbors "pass us and turn their heads and stick their roses up and go on," said Edwards. His older son, Paul, 6 1/2, is "very distrubed" at what has happened in Plains, he said.

Edwards said his son came home from school distressed because one of his friends said he couldn't come to Sunday School because "your daddy wants to let blacks in."

Edwards, who grew up in then-segregated Jacksonville, Fla., smiles ruefully. "That boy goes to a school that is 75 per cent black [the school Amy Carter attended] and yet he can't go to Sunday School if blacks are welcome," he said.