Hard-line conservatives will meet this weekend in Dallas to air their disappointment in Reagan and perhaps lay some groundwork for challenging him if he seeks reelection in 1984.

"We've either got to fish or cut bait," said Howard E. Phillips, chairman of the Conservative Caucus. "Either we get some changes out of the administration or we have to go in a different direction."

Phillips said the purpose of the meeting of about 20 conservatives would be to "see if there is a consensus among conservatives about where we go from here."

Phillips and conservative publisher and fund-raiser Richard Viguerie are openly urging Reagan not to run again in 1984.

"I would think the conservative cause and the Republican Party would be better served if the president doesn't run for reelection," said Viguerie, who says he now doubts that Reagan will run.

The meeting was called by Clymer Wright, a Houston lawyer who was Reagan's Texas campaign finance chairman in 1980. Last spring, Reagan sent Wright a letter rebuking him for calling on conservatives to pressure the president to fire Baker as White House chief of staff.

Viguerie said he has no particular candidate in mind to lead a challenge to Reagan, but his magazine, "Conservative Digest," often has praised Sens. Jesse Helms of North Carolina, Orrin G. Hatch of Utah and William L. Armstrong of Colorado and Rep. Jack Kemp of New York.