Oklahoma Gov. David L. Boren, who a few months ago was considered a shoe-in for the Senate, was forced into a runoff Tuesday in one of the dirtiest campaigns in the state's history.

Boren, a former Rhodes scholar with conservative leanings, will face Ed Edmondson, a former congressman attempting a political comeback, in the Sept. 19 Democratic runoff.

In the only other major election Tuesday, former Alaska governor Walter Hickel held a narrow lead over Gov. Jay S. Hammond in that state's Republican gubernatorial primary, pictured as the most costly and confusing election since Alaska became a state.

The outcome will remain up in the air until absentee ballots are counted. With 364 of 403 precincts reporting Hickel, a former secretary of the interior, held a 565-vote lead, 26,031 to 25,466.

Both predicted victory yesterday.

"We estimate there'll probably be about 8,000 absentee ballots," said Lt. Gov. Lowell Thomas Jr., who serves as director of the state elections office. "And there could be as many as 5,000 questioned ballots as well - people who voted at the wrong precinct for one reason or another. It's going to be a week before we really get into that, so I think it'll be that long before we know who wins."

The race was a rematch for the two candidates, who faced one another in 1974. It also was an environmental referendum, with "boomer" Hickel pushing for opening up the Alaskan wilderness to further development, and "doomer" Hammond favoring a "go-slow" policy.

The Oklahoma Democratic primary, fouled by charges of homosexuality against Boren, ended on a bizarre note when a man stepped out of an election party crowd Tuesday night and fired a rusty pistol at Attorney General Larry Derryberry, a candidate for governor.

The pellets were loaded with red paint. Derryberry emerged soiled, but unhurt.

"I just wanted to show him I could kill him if I wanted to," said Kelly Spencer Ward, who was arrested.

Derryberry finished second in the race, with 37.8 percent of the vote. Lt. Gov. George Nigh collected about 50 percent and may narowly avoid a runoff, which is required in Oklahoma when the leading candidate fails to receive more than half the vote.

In the race to fill the seat being vacated by ailing Republican Sen. Dewey F. Bartlett, the Boren camp had hoped that charges of homosexuality against the popular 37-year-old governor would create a sympathetic backlash that would enable him to avoid a runoff.

Boren denounced the charges as "utterly ridiculous and categorically untrue."

Voters gave him a healthy margin over his two closest competitors - Edmondson, who has twice run unsuccessfully for the Senate, and state Sen. Gene Stipe, a colorful trial lawyer and power broker in the state - but not enough to avoid a runoff.

Edmondson is running as a traditional Democrat who "stands up for Democrats." He charges that Boren is a Republican disguised as a Democrat.

The winner of the runoff will face Republican Bob Kamm, a former president of Oklahoma State University, and four independent candidates on Nov. 7.

In other state races former Oklahoma University football star Ron Shotts easily won the Republican gubernatorial nominations, and Mike Synar, a real estate agent, upset two-term Democratic Rep. Ted Risenhover, who had been characterized as a Washington playboy.

Following are the winning candidates in Tuesday's congressional primaries in Oklahoma. Incumbents are indicated with an (I).