Edward A. Curran, the Reagan administration's controversial nominee for the chairmanship of the National Endowment for the Humanities, yesterday was narrowly rejected by a Senate committee whose members questioned his qualifications and credibility.

The vote by the Republican-controlled Senate Labor and Human Resources Committee was a rare defeat for the White House. It came after months of intense lobbying by national associations of scholars and academics who had opposed Curran as a poor choice to fill a post often described as that of the nation's leading humanist.

A White House spokesman said the administration was "surprised" and "disappointed" by the vote, which appeared to effectively scuttle the nomination. Said the spokesman, "As for Mr. Curran, it's my understanding that he will go back to the Peace Corps," where he is deputy director.

Curran, 52, has not commented publicly in the nine months since his nomination and could not be reached yesterday.

Some of the scholars and academics who had opposed him, however, were jubilant. "What you're hearing in the joy of the reaction is a group of people who thought nobody cared about the humanities," explained Phyllis Franklin, director of the 37,000-member Modern Language Association.

Curran, a former headmaster of the National Cathedral School, was rejected in two tie votes, the deciding ballot in each case cast by Sen. Robert T. Stafford (R-Vt.). His objections stemmed from Curran's stormy tenure at the Department of Education. Stafford was the lone dissenter among the panel's nine Republicans. All seven Democrats on the committee voted against Curran.

Curran made headlines in 1982 when he wrote to President Reagan suggesting the abolition of the National Institute of Education. He was director of the agency at the time, and his letter, which criticized the institute as a tool of the political "left," came just months after he had assured members of the same Senate panel that he would do his best to promote the institute's work. Curran was forced to resign over the letter.

The seeming gap between that promise and his performance at the Education Department provoked lengthy debate during his confirmation hearings last month, where committee members and scholarly witnesses questioned his intentions and credentials.

"There's a saying, once shame on you, twice shame on me," Sen. Lowell Weicker (R-Conn.) said then. (He voted for Curran yesterday, "with trepidation.") At the hearing, Curran had described his proposal to abolish the Education Institute as "a stupid move on my part," and promised he would not recommend the same fate for the Humanities Endowment. "I don't pretend to be the nation's leading scholar," Curran said, and promised to be objective and avoid politicizing the endowment. Although he made subsequent trips to Capitol Hill to reassure his congressional critics, Curran apparently was unable to dislodge the doubts.

"While serving as director of the National Institute of Education four years ago," Stafford said yesterday, "the record shows that Mr. Curran did everything possible to undermine the program and to embarrass the secretary of education . . . In light of Mr. Curran's past actions, I could not be confident he would perform the duties of endowment chairman in a manner the American people have the right to expect."

The White House nominated Curran to succeed William J. Bennett, who left the endowment to become secretary of education. Curran became headmaster at the National Cathedral School in 1968 after working as a teacher at a private boys' school in Houston, where he made the acquaintance of Vice President Bush. In 1980, he worked as a volunteer on the Reagan-Bush campaign before becoming associate director of presidential personnel at the White House.

Stafford had urged Curran to withdraw his name during a meeting the two had several weeks ago, and others had hoped that the White House would withdraw his nomination after the lengthy confirmation hearings, rather than force a vote.

The NIE ruckus was not Curran's only problem, however. Critics of the nomination, including Sen. Claiborne Pell (D-R.I.) and Sen. Christopher Dodd (D-Conn.), said Curran's secondary school background, scholarly credentials (a master of arts in teaching, or MAT, from Duke University) and limited experience with scholarly research and writing made him the wrong man for the post.

In addition there were reservations about Curran's tenure at the Peace Corps, where he has clashed with Peace Corps Director Loret M. Ruppe, as well as his description of that relationship.

Their dispute has been described as both ideological and personal, and reached its nadir last year when Ruppe was discovered to have secretly taped a conversation she had with Curran about their continuing problems. Since then, Ruppe reportedly has given Curran few key assignments and excluded him from most meetings.

In recent weeks as the Curran nomination looked vulnerable, Ruppe visited the White House to to tell administration officials that she did not want Curran back at the Peace Corps if his endowment nomination should fail, according to congressional and Peace Corps sources.

At his confirmation hearing, Curran testified that he had a good working relationship with his boss.

Pell, one of the sponsors of legislation that created the endowment 20 years ago, had vigorously opposed the nomination and praised yesterday's vote. "It was the correct vote," Pell said. "I believe that whoever the nominee is, if he is not a scholar himself, he must have the respect of the scholarly community."

Members of the Senate committee reported receiving hundreds of letters from scholars around the country, in some cases more than those received over the administration's controversial choice of Edwin Meese as U.S. attorney general.

"I don't think this should be a reflection on poor Mr. Curran," Pell added. "It's more a reflection on lack of judgment by the White House."

The vote appeared to surprise some of Curran's supporters, including committee Chairman Orrin Hatch (R-Utah), who had known of Stafford's opposition but brought the matter to a vote yesterday, hoping that at least one committee Democrat would defect.

The National Endowment for the Humanities is a federal agency with an impact and visibility that far exceeds its relatively small $140 million budget, and the periodic search for its chairman has often produced political fireworks. Former president Carter once complained that finding a chairman for the Humanities Endowment was more difficult than finding someone to run the Department of Defense. The chairmanship has often served as a pulpit for administrations' views on the role of liberal arts scholarship in American life.

The leadership of the endowment is a matter of great interest to the nation's scholars, who say the agency is the single largest source of support for scholarly work in the humanities.

The opposition to Curran's nomination included such groups as the National Humanities Alliance, the American Council of Learned Societies and the American Society of Aesthetics.