The House declared vacant yesterday the 5th Congressional District seat of ailing Rep. Gladys N. Spellman (D-Md.), the first time in history a vacancy has been declared as the result of disability.

The action clears the way for a special election to choose a successor to Spellman, who was overwhelmingly reelected to a fourth term last November, but was unable to be sworn in because of massive heart arrest she suffered four days before the election.

The resolution, sponsored by House Speaker Thomas P. (Tip) O'Neill Jr. and other members of the Democratic leadership, passed by a voice vote, with about two dozen members present. Before the vote, four colleagues delivered brief statements of praise for Spellman, who remains semiconscious at Walter Reed Army Medical Center.

Majority Leader Jim Wright (D-Tex.) said "all of us share the sadness" that necessitated such a resolution. Minority Leader Robert C. Michel (R-Ill.), noting that "this is the first time in my 25 years here we have had to face up to such a thing," called Spellman "a wonderful member."

Two of the 18 women remaining in the House lauded Spellman in brief remarks. Rep. Marjorie S. Holt (R-Md.), who represents the adjorning 4th District, said she will "continue to miss her intelligence, diligence and warmth" and Rep. Millicent Fenwick (R-N.J.) said she would remember Spellman's "calm way of talking which is much needed in this place. She is a wonderful politician and a fine public servant."

The last time Congress enacted a similar resolution was in January 1973 when it declared vacancies resulting from the plane crash in Alaska the year before in which Reps. Nick Begich of Alaska and Hale Boggs of Louisiana were lost. An Alaskan judge had issued presumptive findings of deaths, but the state of Louisiana refused to recognize the order, so Congress acted.

Ordinarily, in the event of a vacancy resulting from death or resignation, the governor of the affected state signs an order calling for a new election. But because Spellman had been elected, it was necessary for Congress to exert its constitutional right of judging the qualifications of members to declare the vacancy, according to a House parliamentarian.

In Annapolis, a spokesman for Maryland Gov. Harry R. Hughes said an order was being drawn up calling for a primary election "at the earliest possible date."

Maryland election law requires the primary to be held on a Tuesday, more than 35 days after the governor certifies the vacancy. Another 35 days must pass before the general election can be held, also on Tuesday. Those regulations mean that the earliest dates for the elections would be April 7 for the primary and May 19.

Spellman aides spent yesterday packing the former congresswoman's personal memorabilia. During her six years in Congress, Mrs. Spellman had collected so many plaques and awards, nearly all of which were displayed in her office, that Edna McClellan, her administrative aide, recalled that "Gladys always said that when she left, we would have to take the walls with us."

Aides emphasized, however, that they will continue handle pending and new constituent affairs, from the suite in the Cannon House Office Building and the branch office in Hyattsville.

Included in yesterday's resolution is a provision for Spellman's staff to be assigned to the clerk of the House, and to continue on the federal payroll, until a successor is chosen. Spellman's own salary ended with the adoption of yesterday's resolution.

Meanwhile, the scramble to pick a successor had begun in earnest.

Rueben Spellman, the incumbent's husband, announced his candidacy last Saturday. Although a number of the 93 women who have served in the House first were elected to succeed their husbands, including Rep. Beverly B. Byron of Maryland's 6th District, Rueben Spellman, if he were successful, would be the first husband to succeed his wife in the House.

Spellman said he has received "overtures" to bow out in return for a staff position but said there will be "no deal" to drop his candidacy.

McClellan, Mrs. Spellman's administrative assistant, said she telephoned former State Senate president Steny Hoyer yesterday to complain about one report that Hoyer wanted to employ all of Mrs. Spellman's staff in return for Spellman bowing out.

"We feel we are professionals," said McClellan, "not chattels to be passed around." She added that all of the congresswoman's staff are "supporting Rube" for the nomination.

Hoyer told several people yesterday that he would announced his candidacy on Thursday or Friday.

State Sen. Edward T. Conroy, the unsuccessful Democratic nominee for the Senate last November, earlier said he will enter the race. Other Democrats reportedly weighing a race include former Bowie mayor Leo S. Green and Prince George's County Councilwoman Sue Mills.

Potential Republican nominees include John Burchman, who lost to Mrs. Spellman in 1974 and 1976, and Lawrence Hogan Jr., son of the Prince George's County executive.