A fact-finding panel, formed to investigate charges of racial discrimination in coaching and front-office assignments in the National Football League, criticized the NFL yesterday for refusing to meet with it and said it may ask Congress to take over the investigation.

"Congress is always reluctant to get into these things, hoping they will be settled elsewhere," said Rep. Augustus F. Hawkins (D-Calif.), a member of the fact-finding panel and chairman of the employment opportunities subcommittee of the House Education and Labor Committee.

But if the NFL continues to refuse to meet with the panel, Hawkins, said, "it may have to express its views to a committee of the Congress."

His subcommittee, Hawkins said, has the authority to subpoena witnesses and conduct such a probe. He said he had no doubt that members would agree to an investigation should it become apparent that a pattern of discrimination does, in fact, exist in the NFL.

The charges of racial discrimination came in a study commissioned by the NFL Players Association released last fall. Johns Hopkins University sociologist Jomills Braddock said an exhaustive survey of NFL hiring practices over the last two decades demonstrated a pattern of discrimination against blacks.

William Pollard, director of civil rights for the AFL-CIO, was named to head the fact-finding panel, but Jack Donlan, executive of the NLF's management council, said the panel was stacked against the NFL and that union members were overrepresented.

NFL officials have denied charges of discrimination.