Sen. Herman E. Talmadge, an opponent to date of President Carter's bill to control hospital costs, said yesterday he hopes to be able to "march in unison" with the administration on the issue.

Health, Education and Welfare Secretary Joseph A. Califano Jr. called the Georgia Democrat's mildly conciliatory statement -- plus a strong endorsement Monday from House Ways and Means Committee Chairman Al Ullman (D-Ore.) -- hopeful signs that the long-sought measure can be passed.

Talmadge promised nothing firm, telling Califano, in a hearing before his Senate Finance health subcommittee, that "we have some problems" with the administration bill. But he said he hoped Califano's staff would work with his subcommittee's staff to try to adjust differences, and Califano said his aides would indeed do so.

While the administration has been seeking some kind of hospital cost control measure since 1977, Talmadge has been pressing his own, very different version. He backs a less rigorous approach that would affect only federal Medicare and Medicaid payments, rather than all hospital spending.

The Talmadge -- or Talmadge-Dole bill, since it is also backed by Sen. Bob Dole (R-Kan.) -- would tie a hospital's Medicare-Medicaid payments for some of its services to a target rate related to costs at similar hospitals.

The Carter bill would let hospitals as a group raise their total 1979 revenues by 9.7 percent, and start mandatory controls next January in any state where hospitals fail to meet this goal.

Talmadge approvingly noted that the administration's new bill has borrowed some features of his approach.

This includes some consideration of the special problems of various kinds of hospitals -- smaller, rural hospitals, for example. It also includes some financial penalties and incentives. Hospitals that overspend might be held to an even tighter budget in the following year; those that control their costs might be allowed to budget more for improvements.

Talmadge nonetheless maintained that the administration bill would "reward the rich and penalize the poor" by allowing all hospitals -- large or small, big spenders or small -- an equal percentage increase. Sen. Malcolm Wallop (R-Wyo.) said this "inherent unfairness" could allow a small hospital in a state like Wyoming with growing population to spend little more, despite large numbers of new patients, while giving larger, often less efficient, hospitals large sums.

White House officials and lobbyists are pressing leaders of both houses to push the administration bill to an early vote. But the bill must still face two Senate committees -- Human Resources and Finance -- and two in the House, Ways and Means and Commerce, where a similar measure was beaten last year.

House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) last week said he would put a "time limit" rule on the administration bill, meaning he could send it to the floor in July, even if some committees fail to act. But some House Commerce members have been pressing him for more time for hearings and debate.