Sen. Gary Hart (D-Colo.) yesterday termed President Reagan's defense program "excessive and inappropriate" and recommended cutting it by $100 billion over the next five years.

Reagan's five-year budget plan for defense would come to about $1.6 trillion. Hart, a declared presidential candidate, said he would start with cuts of $6 billion in budget authority for fiscal 1983 and $21 billion more in fiscal 1984.

He issued a long list of weapons systems he would terminate to save money, including the Army's Bradley infantry fighting vehicle, the Navy's F18 fighter and the Air Force's B1 bomber.

"The policies pursued by the administration and reflected in this year's proposed defense budget will weaken, not strengthen, our military forces," Hart said in a Senate speech and, earlier, at a news conference. He said that in three to five years Reagan's program "will produce simultaneous crises in procurement, readiness and retention."

Big bills for weapons now being ordered will pile up in the latter part of this decade, straining the economy as well as the Pentagon budget, Hart said. He predicted this problem would be compounded by cost overruns on weapons.

Reagan's defense plan would short-change the operations and maintenance accounts that keep forces ready to fight, Hart continued, noting that it would lower the accounts' proportion of the defense budget from 30 percent in fiscal 1981 to 25 percent in 1988.

Hart said that after the president and Congress have raised military salaries to the point that skilled people have made the service a career, "the administration has moved to shatter that carefully built credibility" by proposing to deny the raise originally projected for fiscal 1984. The senator said this would send a bad signal to the men and women in uniform.

The Senate Budget Committee today may take up the question of how much to recommend for defense in fiscal 1984. Democrats are pressing for no more than a 5 percent increase, after inflation, in military spending rather than 10 percent as proposed by Reagan.

Senate leaders say they believe it will be an extremely tough fight. Some warn that the Democrats may win over enough Republicans on the committee to make the 5 percent limit stick.