Voters in Maryland's 5th Congressional District today will choose a replacement for Gladys Noon Spellman, the former congresswoman who suffered heart failure last October and has been semicomatose ever since.

The special election between Democrat Steny H. Hoyer, Republican Audrey Scott and Libertarian Tom Mathers is the second such election since Ronald Reagan was elected president and its outcome will be viewed in part as a test of the Reagan administration's popularity.

According to election officials, nearly 40 percent of the 173,196 voters in the district are expected to cast ballots in today's contest. Polls throughout the district, which includes northern Prince George's County and the Takoma Park section of Montgomery County, will be open from 7 a.m. until 8 p.m. Final Results should be known by 10:30 p.m.

Today's vote is the culmination of a six-week general election that was centered for the most part on national issues, particularly the effects of the Reagan administration's proposals on the country and the 5th District.

Hoyer repeatedly spoke in favor of reductions in government but said that many of the Reagan proposals are inflationary or too severe. Instead he said he supports some of the alternative cuts proposed by House Democrats. Scot said she favors the Reagan package as the best way to revitalize the economy, but opposes any cuts that would harm the 5th District. r

Because the campaign has focused on the administration and its proposals, the national Republican Party invested considerable time and effort in Scott's candidacy, with the Scott campaign sending out letters from Reagan and running commercials of endorsement by Vice President Bush. The GOP is hoping a victory in the race will document the popularity of the president and his efforts.

Democratic officials, who gave assistance to Hoyer, are likewise counting on keeping the seat in the party and showing they have rebounded from the 1980 election. A Hoyer victory, the Democrats say, would also demonstrate growing public concern with many specific Reagan cuts.

Because nearly 70 percent of the 5th District's voters are Democrats, and the district voted overwhelmingly for Jimmy Carter in the 1980 presidential election, Hoyer has been given the edge throughout the race. Polls released by both candidates showed him easily ahead and Hoyer's campaign workers said they expect to win by as much as a 10-point margin.

However, the Republicans maintain that Hoyer's lead has steadily eroded in the last two weeks and predict a close one or two-point margin of victory. Republican National Committee Chairman Richard Richards said in an interview yesterday that he is confident Scott will score an upset and demonstrate the President's popularity.

Today's vote is also the closing chapter in the long political career of one of the county's most popular politicians, Gladys Spellman. Spellman, a Democrat, suffered heart failure last October while campaigning for election to a fourth term. She was reelected with over 80 percent of the vote in November but has never regained full consciousness and remains today at Walter Reed Medical Center.

Spellman's continuing incapacitation prompted Congress last February to vacate her seat. Shortly after that, Maryland Gov. Harry Hughes called the special election and 19 Democrats, 12 Republicans, one Libertarian (listed as "Other Candidate") and one official write-in candidate entered the race. It was the most crowded and condensed congressional race in Maryland history.

On April 7, Hoyer and Scott were primary victors, and the general election, featuring a host of famous political names and costing nearly a half million dollars, began. Then early this month, they were joined by a third candidate, Mathers, who won a court battle to appear on the ballot