Rep. Frederick W. Richmond (D-N.Y.), whose activities are being investigated by a federal grand jury in Brooklyn, has been asked to halt his work as chairman of the $5,000-a-person Speaker's Club, a fund-raising operation run by the Democratic Congressional Campaign Committee.

Rep. Tony Coelho (D-Calif.), chairman of the campaign fund-raising group, said yesterday that Richmond had been asked "not to do anything, to take a back seat" in the operation.

"I'm not prejudging him," Coelho said, "but in the business of fund-raising, we've got to be as much on the table in appearance and in fact."

The grand jury is investigating whether Richmond knowingly used his influence to help an escaped Massachusetts convict, Earl W. Randolph Jr., get a job on the House payroll in January, 1981, under an assumed name. Randolph is in custody again after being arrested for male prostitution in New York in March, 1981, while in possession of Richmond's car.

It is a federal crime to knowingly aid an interstate fugitive.

The congressional campaign committee is the first House group to acknowledge it has taken action with regard to Richmond.

Sources say, however, that staff members of the House Committee on Standards of Official Conduct have been collecting information on Richmond's activities.

Coelho said he first spoke to Richmond about two months ago, after the first press reports of Randolph's arrest in New York, and his identification as an escaped prisoner and a Richmond patronage employe in the House.

Prior to that, Richmond had played a role in getting big donors to contribute to the new $5,000-a-person club which was started in July, 1981.

In the fall, according to Coelho, Richmond entertained "about 50 members" at a dinner at his house.

Late last month, a dinner, scheduled for club members at Richmond's home here, was switched to the home of Rep. Corinne C. (Lindy) Boggs (D-La.). According to one source, the change was necessary because the chairman of a major House committee did not want to attend a dinner at the home of the New York congressman.