Contract negotiations between General Motors Corp. and the United Auto Workers got off to a rocky start today as the union accused the company of tolerating "flagrant violations" of a pledge not to interfere with UAW organizing of new plants in the South.

After what was described as an angry session with top GM officials, UAW President Douglas A. Fraser recessed the barely opened contract talks for four days so the company could give full attention to resolving a particularly bitter dispute at a huge new assembly plant in Oklahoma City.

Chief GM negotiator George B. Morris denied any violations of the three-year-old "neutrality pledge" but told reporters that a team of company officials was being sent from here to Oklahoma City this afternoon.

The new plant's 2,200 workers will vote Thursday on whether they want the UAW to represent them A defeat would be a serious setback to the union's influence in the auto industry and to the labor movement's efforts to gain some organizing headway in the largely non-union Sun Belt.

The dispute could also erode the generally good relations between the nation's largest auto maker and the powerful 15-million-member UAW. After a stormy start in the 1930s, this relationship has come to be based as much on mutual trust as economic muscle. Rarely if ever have GM negotiations been interrupted by such a dispute.

It was an inauspicious start for the auto industry's summer of bargaining for a new contract by a Sept. 14 deadline. Morris said, "It didn't create the most harmonious atmosphere . . . not the most ideal." Fraser made clear he wasn't including Morris, GM Chairman Thomas A. Murphy or President Elliott M. Estes in his charges of a "conspiracy" on the part of plant-level company officials and anti-union "forces," but added: "This is frustration as hell, and it makes us angry."

Fraser and Irving Bluestone, UAW vice president for GM operations, said they had repeatedly complained of neutrality-pledge violations, most recently on Saturday. This morning, said Fraser, the union got complaints of cartons of "anti-union literature and T-shirts by the thousands" being carried into an executive parking garage at the Oklahoma plant and distributed by anti-union employes.

Denying even that the plant has an executive parking garage, Morris said, "There is no conspiracy . . . no collusive effort. They are doing their damnedest to remain neutral."

After Fraser and Bluestone met again this morning with Murphy and Estes on the Oklahoma problem, the largely ceremonial bargaining kickoff got started about two hours late with ritual handshakes over the boat-shaped, 52-foot-long negotiating table on the fifth floor of GM headquarters here.

While always among the nation's most important, the auto talks - to be extended to Ford Motor Co. tomorrow and Chrysler Corp. on Wednesday - are especially crucial this year.

A costly settlement could shred all vestiges of credibility from the administration's already battered voluntary wage-price guidelines, which Fraser contends have "self-destructed." And a strike, which many economic analysts are predicting for GM this fall, could exacerbate the recession that the administration now concedes is under way.

In addition, both Fraser and Morris said economic uncertainty may increase pressures for a contract shorter than the normal three years, which could undermine stability in the auto industry.

The UAW's 460,000 workers at GM are paid an average $9 an hour, and about $5.50 hourly in fringe benegits. They are seeking improvements in cost-of-living protection, especially for retirees, as well as wage increases, improved benefits and shorter hours.

The UAW will not choose a target for an industry-wide settlement until a few weeks before the current contract expires Sept. 14, but most observers expect it to be GM. CAPTION: Picture, Ritual handshake opens negotiations between United Auto Workers president Douglas A. Fraser, left, and General Motors Corp. vice president George B. Morris. AP