Owner Robert Irsay finally ended weeks of speculation yesterday by firing Ted Marchibroda after five years as coach of the Baltimore Colts.

It was learned that Penn State's Joe Paterno, long a target of pro teams, is the leading candidate to replace Marchibroda, although the club realizes it faces an uphill battle in its effort to persuade Paterno to leave his present job.

If Paterno is hired, he would be given player personnel responsibilities similar to Marchibroda's. Otherwise, it was learned, the front office will be restructured to separate player personnel duties from coaching chores.

Those considered prime candidates if Paterno turns down the Colt advances are San Diego's Jim Hanifan, one of the NFL's most respected assistant coaches; Mike McCormack, former Philadelphia head coach just fired from an assistant's job at Cincinnati; Frank Kush, longtime coach recently fired at Arizona State; Temple Coach Wayne Hardin, formerly at Navy, and Pittsburgh assistant George Perles, architect of the Steel Curtain defense.

Long shots include Navy's George Welsh and New England assistant Raymond Berry, the former Colt receiver. George Allen and two highly regarded league assistants, Jerry Rhome and Dan Reeves, are not being considered for the job, it was learned.

Marchibroda was notified of his firing yesterday morning by Dick Szymanski, Colt general manager who was carrying out Irsay's orders. Marchibroda, who had three years left on a $150,000 annual contract, had won three division titles and compiled a 41-33 record with the club.

Marchibroda's downfall came after back-to-back 5-11 seasons in which quarterback Bert Jones, the mainstay of the club, rarely played.

But Irsay couldn't wait any longer for Jones to heal, at least not with Marchibroda as head coach.

Irsay said in a prepared statement that he agonized over the decision for the past two weeks.

"This is one decision I had hoped I would never have to make," he said. "I have a deep personal affection for Ted and gratification for the three division championships he brought to Baltimore.

"But there are times in professional sports that the public voice must be heard."

Said Marchibroda: "They all have deep affection for you when they let you go. In the next five years I coach, I hope to win three more championships."

The Colt average attendance of 37,000 per game was the worst in the league and was only 60 percent of capacity. The Colts had about 50,000 noshows at home this year, another NFL tail-ender, and their last game attendance of 25,684 was the poorest home crowd in 25 years.

Baltimore finished the season ranking 20th in total offense, 26th in rushing offense and 15th in total defense in the 28-team league.

Jones, who has been fighting shoulder injuries for the last two years, played in just four games. The Colts won three of those. In 1978, he appeared in only three contests.

With Jones as quarterback, the Baltimore record is 36-16. Without him during the period in which he has been injured, they are 5-20.

Marchibroda's problems ran deeper than Jones' injuries. Since he assumed control of the personnel department after winning a power struggle in 1976 with then General Manager Joe Thomas, the Colts have had questionable drafts.

Toss in the trading of Lydell Mitchell after a spat with Irsay and the loss of John Dutton because of contract problems, and Marchibroda slowly lost what had been a Super Bowl-contending team before 1978.

His last three No. 1 draft choices -- Randy Burke, Reese McCall and Barry Krauss -- have not played up to expectations. This season, only two draft choices -- defensive backs Larry Braxiel and Nesby Glasgow -- contributed much to the team's success.

Before joining the Colts in 1975, Marchibroda had served as an assistant coach in the NFL for 14 years, including two stints with the Redskins. He began his professional career as Redskin backfield coach under Bill McPeak in 1961, moved to Los Angeles in 1965 under George Allen, and returned to Washington with Allen in 1971.