For the first time ever high school students from the Washington area have won both of the nation's toughest scholastic competitions this year in mathematics and science, contests that have traditionally been dominated by New York City and its suburbs.

Randall Dougherty, a 17-year-old senior at W.T. Woodson High School in Fairfax County, scored highest in the country in the seventh U.S.A. Mathematical Olympiad, a three-hour test of mathematical reasoning and creativity given May 3.

It was the final event in a two-stage national competition involving about 370,000 students.

In early March, the nation's most prestigious high school science competition, the Westinghouse Science Talent Search, was won by Michael Briggs, 17, of Adelphi, a senior at High Point High School in Prince George's County.

The Westinghouse contest, now in its 37th year, drew science projects from about 13,000 students, many of them winners in local science fairs.

"It's been a good year for Washington," said Alfred B. Wilcox, executive director of the Mathematical Association of America, one of the sponsors of the math Olympiad. "We shouldn't make too much over one year's results. But New York City has long been the intellectual capital of the country. Now the Washington area is getting a high concentration of intellectual, professional families. Most of them wasn't educated here, but now their children are . . .

"In the arts, you know, a lot of people say the center of gravity seems to be moving in Washington's direction," Wilcox continued. "The same thing may be happening in intellectual (fields) too."

In addition to Dougherty, the second place finisher in this year's math contest also comes from the Washington suburbs. He is Ehud Reiter, 17, of Rockville, a senior at Thomas Wootton High School in Montgomery County.

Next month Dougherty and Reiter will lead a team of eight American students to the International Math Olympiad in Burcharest, Romania. They will compete against high school math teams from 20 other countries.

The U.S. team members will be honored in Washington today at the National Academy of Sciences. Then they will start a three-week training session at the Naval Academy in Annapolis.

Dougherty, an unassuming prodigy who could multiply three-digit numbers in his head before entering kindergarten, has won a place on the U.S. Olympiad team for the past two years. He had a perfect mark in the international competition last year and helped the United States win its first victory.

This is the first time, however, that Dougherty has scored highest in the U.S. competition.

Reiter, who said he was unsure whether he will major in college in mathematics or history, scored among the top 100 in the nation in last year's math contest, but did not make the Olympiad team.

In the Westinghouse contest, Briggs submitted a project on game theory, dealing with the ways in which the maximum number of mathematical opportunities are achieved. Briggs took the first round test in this year's math competition, but did no score high enough to be invited to take the second-round test for the Olympiad.

The Olympiad problems call for creative proofs that require rigorous logic. Despite their difficulty, all of them can be done with a knowledge of only algebra and geometry and do not require calculus.

"It's a matter of ingenuity and the method of approach," Dougherty explained, "not just the solution itself."

The third place finisher in the Olympiad this year was Mark Kleiman of Stuyvesant High School in New York City. Kleiman had scored highest in the contest in both 1976 and 1977.

"At this level they're all so good that it's very close," said Samuel Greitzer, a retired mathematics professor at Rutgers University, who is chairman of the Olympiad committee. "But Randy Dougherty was the only one with a perfect paper.

"He's definitely somebody special," Greitzer continued. "He has an inborn skill at analyzing a problem and then selecting an attack for it. It's a very, very rare ability, and he is going to do some really creative work."

The son of a Navy captain, Dougherty has lived in six different states, and has gone to school in Fairfax County only since 10th grade. He took all of Woodson's senior math courses as a sophomore. Since then he has studied advanced calculus at George Mason University, but is still taking all his non-mathematics courses in high school. Dougherty said he has gotten all As except for a B in English, while usually doing less than an hour of homework at night. Next year he will attend the University of California at Berkeley.

Dougherty said the mathematics he does gives him "a good feeling."

Last year, after he placed second in the U.S. Olympiad, he said he usually loks for "a trick solution, something that would make it real simple. Then if I can't find one, I just work my way through."

He remarked last week: "You usually don't call it a trick solution. You call it an elegant solution. It sounds better that way but it's really the same thing. The elegant solutions are almost the simple ones."

Briggs was born in Washington, and has gone to Prince George's public schools since kindergarten. His father, a mechanical engineer, is a civilian employe of the Navy; his mother, a Prince George's elementary teacher.

Reiter was born in Israel, moved to the United States at age 4, and then went to schools in four different states as his father, now a geophysicist for the Nuclear Regulatory Commission, moved to different graduate schools and jobs. The family came to Rockville from Oklahoma two years ago.

"I think the moving made Ehud depend on his own self for learning," his mother Harriet, a librarian, remarked, "and not on the schools."

Reiter said one important factor in developing his mathematical ability since he came to Rockville has been the competitions he has entered as a member of his hggh school and countywide math teams. The teams take part in problem-solving meets, modeled on sports contests. This weekend Reiter, Briggs and Dougherty are in New Jersey for the year's most rigorous math meet on the East Coast with "all-star" teams from 15 states.

"I like the math all right," Reiter said, "but I really like meeting the people. High school is fine but there are only one or two people who can understand me. When I'm with the county math team, all the people can talk to me. It's great."