Sen. Jesse Helms (R-N.C.), who has irritated other senators repeatedly by blocking appointments, is getting a little of his own medicine.

Helms' closest North Carolina ally, Thomas Ellis, was nominated Feb. 8 to the Board for International Broadcasting, a prestigious panel that oversees Radio Liberty and Radio Free Europe.

But the Senate Foreign Relations Committee is holding up the nomination. Republicans, fearing that Democrats will pounce on the nomination and embarrass Helms and the party, are apparently behind the delay.

"A bunch of people want to pay him back," one Republican leadership source said. "They know Helms wants this appointment, and they're holding it up just because he is Helms' man."

Ellis, a Raleigh attorney, managed Helms' 1978 election campaign as well as President Reagan's 1976 North Carolina primary victory. He is chairman of Helms' National Congressional Club, one of the nation's largest conservative political action committees.

Ellis' biggest problem with the Senate Foreign Relations Committee stems from his association with the Pioneer Fund, a tax-exempt foundation that has supported research into theories that blacks are genetically inferior to whites.

Democrats became interested in the nomination when they found a 1977 article in The News & Observer of Raleigh that quoted Ellis as saying he had served on the fund's board of directors for a year as a favor to an old college friend.

He told the newspaper that he hadn't done anything for the fund except "put my John Hancock on some grants."

Nominations to oversight panels like the Board for International Broadcasting are often confirmed routinely without a hearing. But in the Ellis case Sen. Claiborne Pell (D-R.I.), the committee's ranking Democrat, asked for a hearing, and one was scheduled Feb. 24.

The hearing didn't occur. Committee Republicans, aware of Ellis' connection with the Pioneer Fund, stepped in.

"For Helms opponents, it would have been terrific," one Republican committee staff member said. "It could prove very embarrassing for Jesse Helms to have his main man embarrassed by the committee."

No new hearings have been scheduled, and the nomination is in limbo. Ellis is out of the country and couldn't be reached for comment.

Helms is no stranger to delaying tactics. He has blocked the appointments of moderates to State Department posts repeatedly, irritating the Reagan administration and other senators.

The nine-member board normally meets three or four times a year, takes an annual trip to Europe, and is paid on a per-diem basis.