When Jimmy Carter set up the Senior Executive Service nearly three years ago, it was touted as the ideal place for the government's top managers, an elite niche where Uncle Sam's best and brightest would toil at greater risk but with the promise of greater rewards.

It hasn't worked out that way, according to numerous senior executives. In a recently released survey of 1,000 randomly selected SES members, more than 80 percent said that, although they like their jobs, they believe there are insufficient incentives to retain top quality government executives. Further, 46 percent of the current executives said they are thinking of leaving government service within the next two years.

The written survey, mailed out to senior executives by the Office of Merit Systems and Studies, found a growing dissatisfaction with the SES, which replaced the so-called "super-grade" system and was part of the Civil Service Reform Act passed in late 1978. The 6,700-member SES includes most managerial, supervisory and policy-making positions equivalent to GS 16 through Executive Level V in the Executive branch.

The survey detailed several SES shortcomings:

* The current pay cap of $50,112 has resulted in "pay compression," so that all SES members -- and many of their subordinates -- earn the same salary, preventing distinctions based on differing responsibilities and authority.

* The pay ceiling has meant that some executives have accepted "promotions" with no increase in pay; conversely, some have been less willing to accept promotions.

* The pay cap has hastened the retirement of senior executives, whose pensions are based on the top salary earned. Though they can't improve their base pensions at this point by staying on longer, they can look forward to having their pensions boosted by regular cost-of-living increases once they do retire.

* The SES bonus system, a type of merit pay intended to motivate and reward executives for exceptional performance, has been so severely curtailed by Congress and the OPM that at least half the members of the executive work force feel they have no real chance of getting a bonus in the coming year.

Despite the theory that the SES would mean greater risks for greater rewards, most top performing executives feel they have not been adequately compensated for their work and, as of July 1981, only one career executive has been removed from SES for poor performance. Part of the idea behind the reorganized system was that it would attract and keep the best government executives by increasing their pay and perquisites while eliminating many of the protections against dismissal for poor performance.

The survey reported that 78 percent of the executives questioned are dissatisfied with their pay compared to that of private sector executives. Ninety-one percent said they enjoy the job itself and the kind of work they do, and 80 percent said they believe they get results and have a positive impact on government performance.

On the negative side, 63 percent said they are dissatisfied with the bonuses and rank awards systems.

The vast majority of eligible career executives chose to join the SES when it was set up, particularly since it was understood at the time that failure to do so could scuttle their career advancement.

"People are really down," said E. Jerry Shaw, president of the Senior Executive Association. "There's this tremendous feeling of betrayal, and executives just cannot see a future in government if they are going to keep getting kicked in the head."

Shaw said the treatment of career executives should be all the more troubling to officials because "these are the successes in government, not the failures."

Archie Grimmett, one of those successes, has worked for the federal government for 22 years. He loves his job, regards his career as a higher calling and until recently had never once thought of looking elsewhere for employment.

But now the 46-year-old chief of the Department of Army's staffing and career management division for civilian personnel says he is becoming "very disillusioned about the whole thing." For the first time, he and his wife have begun discussing the possibility that he might want to leave government service unless, as promised, government pay and benefits improve.

The continuing failed promise of the SES has made it "alarmingly unattractive" to mid-level federal employes, the pool from which a large number of future SES members will likely be drawn, according to the survey. And even Donald J. Devine, director of the Office of Personnel Management, has told congressional officials that the government is losing its best, most seasoned executives at a time when it needs them more than ever.

Senior executives, to be sure, are not the only government employes feeling abused these days. Studies of mid- to high-level civil servants indicate that morale in general is suffering, particularly now that more than 48,000 federal employes have hit the $50,000 pay cap.

"The brain-drain problem is a much broader problem than the SES," said Rep. William Ford (D-Mich.), chairman of the House Post Office and Civil Service Committee. He said officials in the Reagan administration have come in "thinking that bureaucrats are to be purged at all costs, and this makes them incapable of making any judgment on the impact of that policy on the government."

Still, the problems of the SES -- where the last substantial pay raise was in March 1977, followed by a 5 1/2 percent cost-of-living increase in October 1979 -- are being felt in every agency.

"We've lost a number of top people," reports Ed Nicholas, director of personnel at the National Institutes of Health and a charter member of the SES. He has been in government service for 35 years.

Nicholas noted that while other government workers just received a 4.8-percent pay increase, SES salaries have been frozen. NIH recently lost one of its top directors, who left to head up a medical school, reportedly at double his salary, and Nicholas said about 30 vacancies have remained unfilled at the agency because it can't find qualified people to take them.

While few are being attracted to government service, others are seizing every chance to leave it. OPM's statistics show that between July 1979 and June 1981 1,100 career executives elected to take optional retirement. Another 363 resigned outright, according to OPM.