John Hannah told the New England Patriots yesterday he is retiring from football. Raider John Matuszak, another big, big man of 32 who quit the NFL just hours earlier, meanwhile followed up with a collision--his 1983 Cadillac into a Los Angeles power pole--hospitalizing him with a probable concussion and possibly bruised heart.

Hannah, voted the best offensive lineman in the game by his peers for five successive years and making $300,000-plus salary, says he has had it with losing: "The system in the league stinks . . . It should be that the owners can't make any money unless they win . . . Everything I had in me has vanished."

The Patriots tried to put Hannah in a trade to Baltimore for John Elway before Denver swung the deal. And, so help him, Hannah says: "I'd sell the family business before I'd go to the Colts" . . .

The 6-foot-8, 290-pound Matuszak, No. 1 pick in NFL draft '73 (Oilers) the year New England took Hannah, cited disk surgery that took him off the Raiders' '82 defensive line:

"You not only have to lift people up by their shoulder pads . . . but you also have to be able to throw them as far as you can. I could still throw 90 percent of them, but that's not enough." Then, the crash . . .

Defensive phenom Billy Ray Smith, Chargers' No. 1 pick, threatens to follow Arkansas teammate Gary Anderson, San Diego first-rounder signed by USFL's Tampa Bay Bandits, to the other league (Oakland Invaders), unless "properly compensated." Chargers owner Gene Klein vows to make a competitive offer . . . Doug Williams, Tampa Bay Bucs' quarterback, says he hopes to re-sign with the NFL team but has not ruled out USFL . . .

Dallas Cowboys try to stay one step ahead, hiring an FBI agent to help players cope with--or, better, avoid--drugs, gambling, etc. Don't call him a company cop, says Larry Wansley, officially retired Monday after 10 years as a G-man, tabbed "Mandrake" for skill in FBI undercover work . . . One that Mandrake won't have to worry about: discontented tight end Jay Saldi--traded last night to the Chicago Bears for an undisclosed 1984 draft pick.