Secretary of State Alexander M. Haig Jr. and Soviet Foreign Minister Andrei Gromyko agreed today on a joint statement to be issued Thursday concerning the limitation of medium-range missiles in Europe, evidently to announce new U.S.-Soviet negotiations on the subject.

The agreement was made known by the two sides following a session of more than four hours between Haig and Gromyko, the highest level meeting between the countries since the Reagan administration took office.

Neither Gromyko, in a brief exchange with reporters after the meeting, nor State Department spokesman Dean Fischer, in a more extensive exchange, would discuss the substance of the forthcoming announcement. Fischer did say the brief written statement concerns the theater nuclear force issue and the reason for delay is to facilitate a simultaneous announcement in Moscow and the United States.

Both sides have expressed repeatedly in recent days their desire for full-scale negotiations on the issue, which was discussed in preliminary form by Soviet and U.S. negotiators last October in Europe. However, the limitation of European-based nuclear systems involves complex and difficult questions, and these are closely tied to even more complicated and contentious problems of intercontinental strategic arms, on which no early negotiations are in sight.

Fischer said that in addition to the issue of theater nuclear arms, Haig and Gromyko discussed "the broad principles surrounding U.S.-Soviet relations" and "touched on a number of international issues," which he declined to specify. Fischer said the Haig-Gromyko talks will continue, as previously scheduled, next Monday.

Haig and Gromyko met alone, with only the interpreters present, for two hours and 50 minutes in the office of U.S. Ambassador to the United Nations Jeane J. Kirkpatrick, at the U.S. Mission. Later the two were joined by their advisers for one hour and 15 minutes more.

Fischer said the meeting was "frank and businesslike." Gromyko did not characterize it. Haig declined to speak to reporters as he left the mission.

Although no details were given today, all indications from U.S. diplomats were that Haig's presentation of the basic concerns of the U.S. administration followed the lines of President Reagan's letter to Soviet President Leonid Brezhnev, which was dispatched from Washington Monday and presented through the U.S. Embassy in Moscow yesterday. According to the rendition of the letter made public here, Reagan attached "particular concern" to two aspects of recent Soviet conduct:

* An "unremitting and comprehensive military buildup over the past 15 years" that exceeds Soviet defensive needs and "carries disturbing implications of a search on the part of the Soviet Union for military superiority."

* The "pursuit of unilateral advantage in various parts of the world--through direct and indirect use of force in regional conflicts," especially involving Cuban forces and arms.

Gromyko, in his U.N. General Assembly speech yesterday, turned both of these concerns back against Washington. He charged that the United States rather than the Soviet Union is staking a claim to "military superiorty" and that Washington rather than Moscow is intervening in its own interest in nearly every corner of the world.

In the view of U.S. diplomats, Gromyko's harsh, unyielding and unusually combative speech was his way of staking out a tough public position in advance of today's talks. Just about the only conciliatory note was Gromyko's declaration, repeated for emphasis, that the Soviet Union is seeking "normal businesslike relations" rather than confrontation with the United States.

Similarly, the U.S. summary of the letter to Brezhnev said Reagan described "his desire for a constructive relationship with the Soviet Union that will lead to a free and more peaceful world community."

Gromyko, in separate talks last night and early today with Britain's foreign minister, Lord Carrington, and Japanese Foreign Minister Sunao Sonoda, reportedly displayed no inclination to make concessions on international issues. But at the same time he expressed a willingness to engage in negotiations.

"Gromyko complained that the United States is so slow in getting ready, and he seemed eager to get on with serious negotiations," said a British diplomat familiar with the Carrington-Gromyko meeting.

To the Japanese minister, Gromyko expressed willingness to respond in serious fashion to U.S. proposals to limit medium-range nuclear weapons in Europe and in arms control generally. The Soviet minister authorized Sonoda to pass this word to Haig, which the Japanese did shortly before today's U.S.-Soviet session.

Carrington spent more than half of his 100-minute meeting with Gromyko probing for some give in the Soviet position on Afghanistan, which, according to a British official, the British minister called "the principal obstacle to East-West relations" at the present time. The official said: "There were no histrionics, there was not a quarrel, but there also was no movement."

Sonoda, who was seeking some sign of Soviet willingness to consider Japan's claim to the Soviet-occupied "northern islands" just north of the Japanese island of Hokkaido, received no encouragement on this central issue, reporters were told.

At the same time, Gromyko was described as quickly agreeing "with pleasure" to reopen high-level Soviet-Japanese talks on overall political issues, which had been suspended since the Soviet invasion of Afghanistan. Sonoda told Japanese reporters that the talks probably will resume in Moscow at the vice-minister level before the end of the year.

Gromyko, who has been foreign minister since early 1957 and has dealt with every secretary of state since John Foster Dulles, received recognition of his longevity from Haig at the start of today's meeting.

Within earshot of reporters and photographers admitted to record the first minutes of the meeting, Haig said, "I read your bibliography last night," evidently meaning Gromyko's biography. "I hadn't realized you'd started in 1943." He added that "they"--evidently referring to other foreign ministers--"have not got the experience you've got."

Gromyko quickly corrected Haig, noting that his career began in 1939--when he joined the American affairs section of the Foreign Ministry. In 1941 he was assigned to Washington as counselor of the Soviet Embassy. In 1943, the year cited by Haig, Gromyko became ambassador to the United States. Haig was then a Notre Dame student.