Defense Secretary Harold Brown discussed the coming pullout of U.S. ground forces with South Korean President Park Chung Hee and other senior officials today in talks that revealed anxieties on both sides.

A high-level U.S. official said Brown informed the South Koreans of concern that the U.S. Congress might not appropriate the $2.3 billion needed to upgrade Seoul's armed forces as most of the 33,000 U.S. combat soldiers stationed here are withdrawn over the next five years.

"We are worried about Congress," he said.

The Defense Secretary gave President Park a letter from President Carter, assuring him that the pullout does not signify "any change whatsover in our commitment" to South Korea's security.

"The mutual defense treaty between our two countries remains fully in force, and our determination to provide prompt support to help the Republic of Korea defend against armed attack, in accordance with the treaty, remains firm and undiminished." Carter's letter said, adding that the withdrawal would be done "gradually and carefully, in a manner which will preserve peace on the peninsula." The text of the letter was released by Park's office.

Informed sources said Defense Minister Suh Jyong Chul opened his meeting with Brown by reading a statement reiterating South Korea's desire to have the U.S. troops remain, but agreeing to accept the withdrawal if compensation were made.

"Our desire," he said, "is to realize the compensatory measures first, then withdrawal," the sources reported. The American position is that the two programs should go ahead simultaneously.

The South Koreans are also concerned that the 1954 Washington-Seoul mutual defense treaty, by saying that each country "would act to meet the common danger in accordance with its constitutional processes," opens the door for Congress to block any U.S. action to help Seoul. Rather, the Koreans say, they want a NATO-type of guarantee, with an unconditional promise of intervention in case of attack.

These and other issues related to the withdrawal were discussed today at the opening sessions in Seoul's National Defense Ministry.

A U.S. official said the two sides have agreed on the removal of 6,000 GIs in late 1978, formation of a joint command and strengthening the U.S. Air Force units President Carter has said will stay here indefinitely.

While the U.S. Army in Korea is gradually cut down to fewer than 6,000 communications, intelligence and logistics soldiers, the Air Force strenght of 7,000 men will rise as a modest number of tactical fighters are added, the official said.

As now projected, the pullout will be completed in three states. A Pentagon study put an $8 billion - later reduced to $7 billion - price tag on the measures South Korea must take to replace the departing Americans.

The Seoul Government has already embarked on a self-financed $5 billion force-improvement plan, which began last year.

The only direct cost to the U.S. tax-payer will be the $500 million needed to re-equip the 2d Infantry Division, since most of its weaponry and equipment will be handed over to South Korea.In addition, Congress will be asked to appropriate $1.8 billion in foreign military sales credits - long-term loans at concessional rates allowing the South Koreans to buy fighter aircraft, anti-tank missiles, guns, helicopters, fighters an destroyers. As planned, the loans will be appropriated at the rate of $275 million a year for five years with a single-additional payment of $300 million.

The heart of the two-day conference was a private meeting today between Brown and President Park at the Blue House, about which the high U.S. official would disclose almost nothing.

He specifically denied that Park questioned Brown on congressional attitutes about the withdrawal plan. Brown is known to have told other South Korean officials, however, about an unusual tripartite congres-drawal.

The three disparate groups are those who oppose the withdrawal as militarily inadvisable, those who oppose the Park regime over human rights and will not vote funds for it, and those influenced by the continuing investigation of alleged South Korean bribery activities on Capitol Hill and who wish to avoid any appearance of being involved.

Brown is said to feel that the funding bills will go through Congress, but only after an educational process has prepared the way.

The high U.S. official said he does not believe that Suh's insistence on measures is a major problem, although the South Koreans are well aware of congressional veto rights on funding. A $1.5 billion modernization program undertaken with U.S. help is two years late still incomplete due to congressional action in limiting funds.

Today's meetings, according to the U.S. official, were businesslike and cordial, but did not always result in agreement.

The South Koreans requested M-60 tanks. They U.S. side reacted negatively, feeling that a mixed tank force is undersirable. They said the South Korean request for F-16 advanced tactical fighters was received favorably, but the American officials predicted that it would be several years before production facilities could meet any South Korean order.

The change in the command structure will place most of the South Korean air-defense and U.S. Air Force units under a joint command with an American commander and a Korean deputy. The joint commander would be the four-star general already charged with the commander of U.S. and U.N. forces in South Korea.

The calm public exterior of the talks seems to represent a determined effort by both governments to control the rapid decline of bilateral relations. American public opinion has been offended by the alleged bribery of U.S. politicians by entrepreneur Tongsun Park, and by the deprivation of important human rights in South Korea. The South Koreans, on the other hand, are resentful of the U.S. decision to withdraw.

In addition to a mutual wish to restore relations, there is an understanding that any disagreement may be interpreted by North Korean President Kim II Sung as a sign of weakness.

Still, the opening statements by Brown and his South Korean counterpart illustrated the two governments differing objectives and attitudes. According to sources close to the conference, Suh stated that South Korea would prefer that American troops stay, but has no choice except to accept the withdrawal plan.

Brown said that a careful, gradual phase-out over a five-year period was preferable to a sudden mass withdrawal, which could occur if the political pressures against a U.S. presence in South Korea suddenly mounted.