The Reagan administration's fiscal 1986 budget contains money to develop a prototype nuclear power reactor that eventually could be used to power the remote radar stations of the modernized, $1.2 billion, distant early warning (DEW) line across the top of northern Canada, according to American and Canadian government sources.

Under current plans, the modernized DEW line will consist of 39 unattended, short-range radars and 13 longer-range ones with small operating crews. The DEW line is designed to provide early warning in case of a Soviet nuclear attack.

Thirty-one existing, older radars, in place for more than 20 years, are powered by electric generators that require regular deliveries of diesel fuel. The proposed nuclear power reactors would last for years without refueling, saving personnel and fuel costs.

But the possible introduction into Canada of the small nuclear power systems has created a controversy there, a Canadian embassy official in Washington said yesterday.

"We are working on ways to power the new North Warning System," the official said, "but no decision has yet been made."

The budget contains a new $19.5 million item labeled "defense nuclear energy." One of the programs listed under that title is "development of a 15-20 kilowatt terrestrial reactor as an alternate power supply to update the DEW line," one budget document reports.

The project, according to the document, "will complete fabrication of the demonstration reactor core and design of the prototype reactor system."

The Canadian Embassy official said yesterday that he was "unaware" that the U.S. program "had gone that far." He said there has been "some talk on developing a nuclear power source . . . one of these degenerating systems" such as is used in Soviet satellites. One of those satellites crashed in a remote part of Canada in 1978, spreading tiny amounts of radioactive debris.

Three weeks ago, the Canadian official said, representatives of the U.S. Department of Energy and the Canadian Atomic Energy Corp., met to discuss the issue but this represented "preliminary talks from one agency to another."

Meanwhile, officials in Washington and Quebec City have been trying to work out a cost-sharing agreement on the DEW line modernization program that would be ready for signing when President Reagan visits the Canadian capital March 17-18.

One stumbling block has been questions raised in the Canadian parliament about the relationship of the DEW line system to Reagan's Strategic Defense Initiative, the "Star Wars" program.

Another project to be carried out by the Energy Department with proposed fiscal 1986 funds is development of a 50- to 1,000-kilowatt reactor to be used in space. The goal is to provide power to run a space-based radar system for up to 10 years.