When the privileged gentry who buy their way into the Atlantic Coast Conference basketball tournament each year relive tournament memories, they rarely begin by mentioning a great play or a dramatic moment.

Usually, they start by complaining.

Their seats were no good.

They had to park too far away.

The referees cheated.

The other fans were obnoxious.

This is a festival, a three-day basketball orgy, a major social event. But the 16,034 who cram into the arena each day and call themselves basketball fanatics have little in common with the crowd at an Urban Coalition League summer basketball game in Washington. Their connection with the eight schools who field teams here is, in almost all cases, strictly financial. They Contribute.

College students have little to do with this college tournament: approximately 700 students get into the building each of the three days--and that's if you count the players.

This isn't a basketball event so much as a money event. It takes money to get in here, money to get the talent to win here and money to afford the social whirl that is as much a part of this as long-time North Carolina coach Dean Smith. The people who come here expect to be treated royally.

To get into this tournament you have to be someone or know someone. At the University of North Carolina, if you joined the school's booster club this year, you would have to contribute $20,000 in 1982 to be guaranteed the right to pay $120 for two tournament tickets. Even at a school like Duke, with a relatively small booster club, it would cost you $1,000 to be able to buy two tickets.

"If they pay that kind of money to buy the tickets, I say more power to them," said Virginia student Steve Schoemaker, aware of the fact that at Virginia the current minimum contribution is $2,500 to be guaranteed tickets. "I hope someday I'll have that kind of money so I can contribute big dollars and get my tickets."

Everyone here, it seems, wants to belong. When the ball goes up in this building, you can turn in any direction and see money and power. On one side of the building this afternoon sat James Hunt, governor of North Carolina, an N.C. State graduate but a North Carolina fan today. In front of him sat Skipper Bowles, former gubernatorial candidate in North Carolina who has moved on to more important work: He is in charge of raising $30 million to build a new coliseum in Chapel Hill.

On the other side of the court sat Louis Goldstein, the comptroller of the state of Maryland. Friday, when Maryland played and lost, Goldstein wore a red shirt and a Terrapin tie and exhorted the Terps to "be patient, be patient," while many others booed the slowdown game.

Goldstein also has a press pass, which gains him entry to the press room and complimentary food and drinks. "I used to be president of the Maryland-Delaware press association," Goldstein explained.

Friday, when Marvin (Skeeter) Francis, the tournament director, walked by his seat, Goldstein gave him a warm handshake and his card. Francis smiled. "I've got 10,000 of these," he said, waving the card as he walked away.

Each year, it seems, the tournament is engulfed by a controversy that has nothing to do with the games themselves. Last year it was Prince George's County's unwillingness to exempt the tournament from the county amusement tax. This year it is a new seating system, which puts four of the schools wholly downstairs and four wholly upstairs.

"I'm sure I would have enjoyed the tournament if I could have gotten in," said Dr. Thomas Hulvey III, an orthopedic surgeon from Abingdon, Va., who is a member of the Virginia booster club. "I'd look downstairs only I might get a nose bleed."

As he spoke, Hulvey sat three rows from the top of the coliseum and from there, even 7-foot-4-inch Virginia Cavalier Ralph Sampson appeared to be about the size of a gnat. The Virginia people are not pleased that they are seated upstairs while, by luck of the draw, North Carolina, today's opponent in the final, is downstairs.

"Luck of the draw my ---," was the consensus in the upstairs section.

Understand this about the ACC tournament: everyone is convinced they are getting gypped and, in most cases, everyone is convinced UNC is the one doing the gypping.

What strikes newcomers when they walk in here are three things: the garish regalia most fans insist on to clearly label their loyalties; the almost total lack of co-mingling among the fans from various schools and the dozens of souls standing outside the building holding signs which inevitably say, "Need Two."

The rest of the country may look at scores in the ACC and say the league is losing its appeal because of stallball and it may note that half the teams are mediocre, but in this section of the country the ACC has never been hotter.

"I've been coming since 1937 when this used to be the old Southern Conference tournament," said Walter Keller, whose wife, Anna Keller, is director of admissions at N.C. State. "I remember when you used to be able to go down the night of the games and buy a ticket at the door. Of course, that's changed just a little bit the last few years."

Indeed. There has not been a public sale of tickets since the mid 1960s. One scalper arrested this year was asking $300 for tickets--and getting it. One particularly clever scalper bought 100 tickets this year, many of them reportedly from Georgia Tech students, and was asking $250 per ticket when he was arrested. The man was going to raise an entrapment case when the judge pointed out to him that if he did, he (the judge) might consider an investigation into who got ACC tickets and how. The man pleaded guilty and paid a $100 fine.

This ticket mania continues even though every game in the tournament is televised. Two years ago, C.D. Chesley paid $1 million for TV rights to conference games. This year, Metrosports paid $3 million. Next year, the conference will begin a three-year, $18 million contract with Raycom Inc. in addition to a three-year, $4.5 million contract with NBC for selected games. In short, each school will receive about $1 million a year from television alone.

Clearly, money makes the ACC go around. Because of their desire to belong, ACC fans will go to amazing lengths and expense for their team. Robert and Rhoda Osternak, longtime North Carolina fans, have a life-sized Ram doll (the school mascot). They bring it to each UNC home game. The Osternak's each have a front row seat in Carmichael Auditorium. So does the doll. The doll is not at the tournament. It only goes to home games.

Many fans travel many miles at great expense to see their team play. One of Maryland's biggest fans is political columnist Robert Novak (an Indiana graduate) who rarely misses a Terrapin game anywhere. Novak is one of a small group granted access to the Maryland locker room after games.

For a fan, that is the ultimate status. "To be associated with a fine group of young men who represent the university well is quite enjoyable," said Durwood Chase, who, when he is not wearing orange pants and glad-handing with Virginia basketball players, is an investment counselor who generally handles million dollar-plus deals. "But I also love the basketball."

The students do, too. Although minimally represented, they are the most enthusiastic fans. Many of the University of Virginia students stood, sat and huddled in line for five days to get tickets.

"You do it so you can be here, so you can be part of it, right down there in it, not just watching, but getting into the games," said Laura Feldman, a junior from Richmond. "It isn't the same unless you are right there."