U.S. Representative Gladys Noon Spellman (D-Prince George's), who had been considering a run for Republican Sen. Charles McC. Mathias' post in the 1980 election, has decided instead to seek a fourth term in the House next year.

Spellman, 61, whose overwhelming reelection victories in recent years have marked her as perhaps the most formidable Democratic politician in the Maryland suburbs of Washington, said many of her supporters had been pushing her to run for the Senate.

"I gave the idea considerable thought this winter," Spellman said in a newsletter to her campaign workers this weekend. "And I decided that what I should do is stay right where I am in the House of Representatives."

Mathias, a liberal Republican who is finishing his second term in the Senate, has cultivated such strong support from organized labor and blacks - normally the constituencies of Democrats - that many potential Democratic candidates appear hesitant to challenge him.

Aside from Spellman, other Democrats mentioned as possible Democratic challengers include U.S. Rep. Barbara Milkulski (D-Baltimore), who lost to Mathias in 1974, former Maryland Senate President Steny H. Hoyer and Baltimore City Council President Walter S. Orlinsky. CAPTION: Picture, REP. GLADYS SPELLMAN to stay "where I am"