After 20 years on "Gunsmoke," did James Arness' star quality bite the dust? That question plagued executives at Metro-Goldwyn-Mayer a couple of years back when they were considering the former Matt Dillon for lead role in their pilot series "How the West Was Won."

Several top MGM executives hudled together one hectic day at their New York executive office to ponder the predicament. "My boss called me in," recalls Charles Gersch, now MGM's director of research. "He asked me what was the appeal of James Arness with 'Gunsmoke' off the air. We wondered if anyone remembered him at all."

For answers, Gersch turned to a fat, yellow book known as Performer Q. By turning its pages, he could check up on the popularity and familiarity ratings of Arness and more than 600 other movie, television and sports celebrities. The only syndicated "personality measurement" service in the country, Performer Q is considered a "valuable casting tool" by some network executives and many advertising agencies. But it's anathema to many others in the entertainment industry, generating strong emotions. One veteran actor called it a "blacklist." A major network failed to renew the rating service under intense pressure from the Screen Actor's Guild. A leading screenwriter accuses it of promoting "Further diiocy" on television.

But, for Gersch, - and for Arness - the news from the yellow book was all good. "We found out that Arness was not only well-known but well-liked as well, even after the long 'Gunsmoke' stint. I said certainly he's somebody you'd want for that lead," Gersch recalls. "The rest is history."

Today, in part thanks to the Performer Q survey, james Arness is riding the TV comeback trail. After appearing on the pilot of "How the West Was Won," Arness starred in three two-hour shows on the same basic theme a year later. Next year, ABC is planning a 10-week, two-hour series of "How the West Was Won" specials, all starring the reborn marshal of Dodge City.

Across the country in an office above Sunset Boulevard in Hollywood, Mike Mamakos, agent and long-time friend of Telly Salavas, was listening with rapt attention to the latest Performer Q figures on his star. The statistics weren't too good. Over the last year Salavas' popularity, according to the Performer survey, dropped 30 per cent from a "Q" of 32 to one of 20.

"Telly places no importance on these figures, if he's a 96 or a 1. He isn't the least bit interested in his 'Q'," Mamakos said gruffly. "His philosophy is he goes out there and does his best and he can't control the reaction of the public."

Whiles minimizing the importance of the performer rating. Mamakos took pains to assert that Telly Salavas is still very popular - much so than, say, Meadowlark Lemon of the Harlem Globetrotters or Paul Michael Glazer of the "Starsky and Hutch" television series. As it happens, however, both Lemon and Glazer outpolled Salavas by a wide margin in the 1977 "Q" test.

"I can guarantee you I could go anywhere in Hollywood today and ask who Meadow Lark Lemon is and ask who Telly Salavas is and I'll bet Telly is better known," Mamakos insisted. "And as for that Paul Michael Glazer, I don't think on his greatest day he could equal Telly's talent in any department."

The source of all this sound and fury is a small market research firm located in an unimposing colonial-style office building on a tree-lined side-street of Port Washington, a Long Island town 40 minutes from Manhattan. The company, Marketing Evaluations Inc., occupies a small office with six or seven secretaries preparing the lists that go into the yellow book.

Marketing Evaluations was founded in 1962 by Jack Landis, a former NBC executive. Landis developed a system of questionnaires sent to 2,000 families across the country which purports to measure the popularity and general awareness of celebrities of television, sports and movies.

These families get a list of more than 600 names - 300 taken from the casts of shows written up in TV Guide and the rest submitted by various Performer Q clients - and rate their opinions of them on a one-to-five scale. In return for their work, the families receive numerous sample products to test from some of the commercial clients of Marketing Evaluations.

Over the years this study, published annually under the trade name Performer Q, has developed a dedicated clientele among network executives, advertising agencies and studios. The "Q" numbers, a statistical measurement of the popularity of the stars by the Marketing Evaluation families, have come to have a important influence in casting decisions for movies, television shows, commercials and even newscasts.

The "Q" itself is a mathematical quotient derived by dividing all opinions by the number of people who consider an individual "one of my favorites." In this way a person's "popularity" is measured independently of his or her fame. Therefore, a celebrity can be exceedingly well-known - say, Muhammed Ali - but nevertheless have relatively few people who consider him "a favorite," thus leaving the heavyweight champ with a small "Q" even though he is among the world's best-known people.

Five years ago, Steve Levitt, a young Hofstra University graduate who was brought into the business by Landis, took over the president's job at Marketing Evaluations. He believes the "Q" figures are an important and accurate assessmant of public attitudes, a marketing device that is invaluable in the celebrity-oriented commercial climate today.

"We say it's like the Bible," Levitt, 37, said, scrathing his beard. "You don't use it every day. It's like an encyclopedia, you're using it all the time you need it." Levitt added that it costs his clients between $2,500 to $5,000 a year to purchase the book - the exact fee is determined on a sliding scale with the larger clients paying a higher price.

Among the subscirbers to this "Bible" of celebrity are at least two major networks, ABC and NBC, MGM studios and scores of advertising agencies across the country. While sensitive to critism of their use of the system, performer Q's clients admit it does have at least a limited effect on key casting decisions. "We don't like to get specific about these things. It's just another tool we use," said George Hoover, an NBC spokesman. "It's not usually the barometer we use to make casting decisions but there are cases when it has happened."

It is this power given to numbers which disturbs many leading actors, directors and producers in Hollywood. Performer Q has become sort of a cause celebre, especially among activistsa t the Screen Actor's Guild and the Screenwriters Guild who see it as a dangerous, inhuman intrusion into the creative process.

"What they're doing is casting shows on the basis of Performer Q," complains Burt Freed, vice president of the Screen Actors Guild. "They're leaving it to mechanical means. It has nothing to do with talent. There is no esthetic value here, no art here."

Freed and the 36,000-member union he re-presents have led a strong campaign to get networks and individual producers to drop Performer Q. Charging that the system discriminates against both unknowns and "character" actors. SAG has won some signifigant victories against the mighty "Q."

Among the converts to an anti-Performer Q positions is George Shaeffer, an independent Hollywood producer. Schaeffer recalls bitterly the time six years ago at NBC's Burbank studios when he was "being bothered all the time" by netwoek executives to stock specials with people blessed with a high "Q" rating. "I won't yse that thing any more," the producer said. "It's a ridiculous and non-productive way to cast a show."

More importantly, perhaps, SAG's campaign has led ABC recently to cancel any continuation of their current contract with Performer Q. But other keu Marketing Evaluation customers have so far refused to bow to the pressure. "Despite what the so and sos over at SAG say, this isn't a terrible, secret study," says Performer Q client and MGM executive Gersch. "It's an open study - anybody who's got the money in their pockets can buy it."

The assault on his clients disturbs Marketing Evaluations's Steve Levitt, who considers himself the victim of a political power play. "Their arguments are silly," Levitts says. "They say we are keeping people from work, but don't you think the Neilsen's do the same? I just think they're prejudiced. Isn't it a coincidence that Kathleen Nolan (president of the Screen Actor's Guild) isn't in this year's study? Nobody asked for her."

Nolan, in Europe, could not be reached for comment.

Despite the reaction against the lists, Levitt believes Performer Q has a bright and beautiful future. In recent years new clients have come to him from various television news departments around the country who, in local rating wars, seek to find the most appealing on-air personalities.

But most important, Levitt believes, are the increasing numbers of advertising agencies which are, he says, lining up to buy the fat, yellow book. Already, he notes agencies are signing up celebrities by the score - including the long-time untouchable and king of "Q" list, John Wayne - to sell their products to celebrity-crazed consumers.

One advertising company executive, Bill Long of the firm Darcy, McManus and Masius, said Performer Q played a key role in making Jack Nicklaus the current spokesman for Pontiac cars. Using the Performer Q study, Long compared Nicklaus to several other possible celebrity spokesmne - including Bob Hope, Muhammed Ali and O.J. Simpson - and found Nicklaus the right man for the job.

"There's no better way to do it," Long said. "If someone says to you, 'What do you think about a guy?' it give you a way to quantify certain attitudinal aspects around that person."

While Performer Q may well have its strenghths in selling cars or dog food (they even used the "q" to see if Sylvest Stallon could sell dog food), there are many in the creative end of the arts who wonder if it's influence is undermining the mass culture and the entertainment business.

"The real problem with the "Q" is that popularity becomes the criteria for quality," says David Rintels, president of the Screen Writers Guild. "It calls to mind Gresham's law when the worst, most-vulgar, violent show can drown out Shakespeare. It may end up as the final stranglehold on creativity."