Gene Mayer, who has made no secret of his annoyance with the fact that the pretournament attention in almost all the major tennis championships these days centers almost solely on Bjorn Borg and John McEnroe, grabbed the spotlight the same way he hits all his ground strokes -- with both hands -- on the first night of the $400,000 Volvo Grand Prix Masters tournament by upsetting McEnroe, 3-6, 7-6, 6-2.

Mayer joined Borg, Jimmy Connors, and Ivan Lendl as winners on opening day of the round-robin portion of this eight-man playoff for the top finishers in the 1980 Grand Prix tour. He put McEnroe in the unevitable position of having to beat Borg Thursday night to have a reasonable chance to reach the semifinals. If Mayer beats Jose-Luis Clerc in the opening match of Thursday evening's session, McEnroe would have to beat Borg -- his adversary in last year's memorable five-set Wimbledon and U.S. Open finals -- to stay alive.

Mayer, 24, has climbed to No. 5 in the computerized world rankings of the Association of Tennis Professionals, but never before had beaten any of the players ranked ahead of him. He was 0-7 against Borg, 0-6 against McEnroe, 0-2 against Connors, and 0-5 against Guillermo Vilas. But tonight was the occasion Mayer chose to put anonymity behind him. Returning serve superbly with his oversized graphite racquet, which he swings with deft double-fisted touch off both wings, he impressed a crowd of 15,637 at Madison Square Garden.

"This was my biggest win, and it came in one of the biggest tournaments in the world," said Mayer, who plays with his upper right side heavily taped to protect against the reoccurance of a nagging hamstring injury. "I always thought I could beat the top players, but it's one thing believing and another to be able to do it on the court. Tonight I was able to do it on the court."

Mayer's victory overshadowed triumphs earlier in the day by Borg, who beat Clerc, 6-3, 6-4; Lendl, who served fitfully but still overpowered Harold Solomon, 6-3, 6-1, and Connors, who dispatched Vilas, 6-2, 4-6, 6-0. h

Mayer was heretofore known to tennis aficionados largely for his legendary appetite, which has made him the bane of restaurants serving all-you-can-eat buffets. Tonight he showed that he was just as hungry for a victory over a top-ranked player.

McEnroe, who was hampered by an ankle that he twisted in the first set, made a single service break in the eighth game stand up for the first set, but he was in trouble after losing his service rhythm in the second set tie breaker.

There were no breaks of service through 6-6 in the set, but McEnroe was able to win only one point on serve in the tie breaker. Mayer, keeping the pressure on with hard and nicely controlled returns, took a 6-2 lead and held on to win the tie breaker, 7-5, when McEnroe knocked a forehand return wide.

McEnroe served 17 aces, but he died by his seven double faults. One of these came in the tie breaker, and three more in a row cost him his serve at love in the first game of the third set.

After that abysmal service game, McEnroe strode to the changeover with his head down, shaking his mop of reddish curls. He was strangley subdued, lacking his usual fight and fire, but even so, Mayer had to go out and win the match.

He only wavered once. In the fourth game, after McEnroe had held serve easily to 1-2, Mayer was down 15-40, and faced three break points in all. On two of them, he missed first serves, but McEnroe overanxiously tried to attack the returns and drove them into the net. It was clearly not his night.

The audience was getting on McEnroe for arguing several line calls, though the brash New Yorker was really quite controlled in his deportment. It was really more a matter of Mayer being a fresh and appealing, albiet bland, face, and the crowd was vocally cheering the underdog.

"They found someone new. They didn't like some of the things McEnroe had done, and they got behind me, especially at the end," said Mayer, reveling in his new-found recognition.

In the fifth game of the final set, trying desperately to hand in despite his sore left ankle and stomach cramps, McEnroe served an ace to go up, 40-30. Then he double faulted again to make it deuce. He hit a good first serve wide to the forehand, but Mayer lunged and sent another splendid return winner winging down the line. Then McEnroe butchered a backhand volley for the break.

Mayer was growing in self-confidence as McEnroe wilted. He served two aces and a service winner to hold for 5-1, and delivered another ace in serving out the match.

Mayer was jubilant, but reserved, in victory. He knows that this match will do wonders for his reputation, but is equally cognizant of the fact that there is still a long way to go in this tournament. "As you play better, you get more publicity," He said diplomatically. "Hopefully, mine will come as I play better."

Borg, Lendl and Connors all won their opening matches without having to produce their best tennis. They were all pleased to win comfortably while getting acclimated to the playing conditions in preparation for the bigger challengers ahead.

Connors had the toughest test, and in stretches looked like his former world-beating self, but it was difficult to gauge how well he was playing because the lopsided final set of his match was attributable mostly to a collapse by Vilas.

Vilas lost his serve in the second game of the final set, hitting a second serve a foot and a half long at 30-40, but promptly had Connors down, 15-40, in the next game. The failure to cash either of those break points, and Vilas' los of concentration following a bad call on the first of them, turned the match back in Connors' favor.

Vilas glared furiously at the linesman he felt had victimized him, and on the next point just missed a forehand service return. Connors went on to hold serve while Vilas -- visably and vocally upset -- quit trying.