Amid growing concern about the qualifications of U.S. envoys abroad, a group of retired senior diplomats is forming an organization to rate prospective ambassadors on their qualifications for office.

The group, to be called the American Academy of Diplomacy, will be officially announced sometime in the next few months, said Elliot L. Richardson, who served briefly as undersecretary of state during the Nixon administration before moving on to a variety of Cabinet posts, including attorney general.

As envisioned by Richardson and others, the academy will consist of 60 or 70 of the country's "most distinguished" names in diplomacy and will bestow awards and undertake policy studies besides rating the abilities of potential ambassadors.

Richardson and others would like the academy to serve the same kind of informal screening function that is provided for federal judiciary candidates by a standing committee of the American Bar Association.

The Senate long has acted as a virtual rubber stamp for ambassadorial appointments, and, according to Senate historians, has refused to confirm only two in the nation's history. Murat Halstead, a newspaperman, was rejected as minister to Germany in 1889, and Martin Van Buren was turned down as ambassador to England in 1831, six years before he became president.

The academy's supporters do not expect President Reagan or any other president to agree at first to submit nominations to the academy, but they hope that the group's prestige will eventually bring acceptance, at least by the Senate Foreign Relations Committee