When, in 1974, little-known author John Irving needed help, the National Endowment for the Arts gave him a $5,000 literature fellowship. Now, Irving, whose 1978 novel "The World According to Garp" made him a wealthy man and earned him a cult following, will give some money back to the NEA, literature program director Frank Conroy yesterday told a congressional subcommittee.

Irving confirmed yesterday that he would make a contribution but refused to specify the amount.

"That's between me and the endowment," said Irving, reached by phone in New York.

Asked why he was making the contribution, Irving said, "I have every reason to believe that it's a good organization. There are few organizations that support people in the arts. With the license the goverment is taking toward giving no money to poor people . . . it's time for people in the arts to think about giving money to the arts--especially those organizations who've helped people out when they needed it."

"What happens if a writer who had a fellowship sells to the movies and gets rich?" Rep. Sidney Yates (D-Ill.), chairman of the subcommittee, had asked Conroy yesterday at a hearing of the House appropriations subcommittee on the interior devoted to the proposed NEA budget for the next fiscal year.

" John Irving had a grant from us many years ago," said Conroy. "He hit with 'The World According to Garp,' and he's going to give the money back."

Conroy, who later said that Irving did not actually mention a figure, said that he called Irving a couple of weeks ago. "I asked him to support the endowment," Conroy said. "I know he's a big supporter of the endowment. He said he would give us some money."

Conroy added that Irving would be giving the money next year: "He's out of tax deductions this year." Irving said he had no comment on that.

Irving said he is also giving the money "as a gesture of my feelings about Frank. I hope he'll stay around there for a while."

Conroy, the author of the 1967 book "Stop-Time," called Irving "an old friend of mine."

Conroy said he will ask for contributions from other writers who have received NEA grants, but cautioned that this was not "like a big fund-raising drive." He refused to name people he would call.

"I'll be trying to get support from writers who've gotten grants and done very well financially ," said Conroy. "That's a very short list. I'm also going to be asking writers to support the endowment modestly as a gesture of their support. It's not like they're going to give thousands of dollars. They're going to say they support it and back it up with a little cash. There are very, very few rich writers in the United States at all."

NEA literature fellowship winners over the last 16 years include Pulitzer Prize-winning poets John Ashbery, Donald Justice and Maxine Kumin, and writers Walter Abish, last year's winner of the PEN Faulkner Award; Tim O'Brien, author of "Going After Cacciato"; and Maxine Hong Kingston, author of "Woman Warrior."

The question of the responsibility of artists who receive government grants and later become wealthy from their art has been raised in hearings before. Generally, few artists fall into that category.

"Usually when somebody makes it big, really big," said Mary MacArthur, assistant director of the NEA literature program, "they pay it back in their income tax.