The Democratic Party's "open" convention wars moved beyond theatrics and into the trenches yesterday as both sides settled down to personal lobbying of the delegates who will decide the issue in New York nine days from now.

President Carter and Vice President Mondale, entertaining about 400 Carter-pledged delegates at the White House, urged their supporters to stand firm behind a proposed convention rule that would require all delegates to vote in accord with their original commitments. Approval of that rule at the start of the convention would guarantee Carter the nomination.

But campaign aides at the Kennedy-for-president headquarters and the staff of a Capitol Hill committee trying to open the convention began working telephone banks to convince delegates to vote against that rule, so that the convention could choose Sen. Edward M. Kennedy (D-Mass.) or anyone else to head the ticket.

If there was any division in the ranks of Carter delegates, none was evident at the White House. A crowd of several hundred greeted the president in the East Room chanting. "We want Jimmy. We want Jimmy. We want Jimmy."

Carter looking tan and rested, replied with a broad grin, "Well, you got him."

He then launched into an emotional pep talk boosting the Carter-Mondale team, warning that the days before a national political convention are always filled with "tension, debate and some degree of trepidation."

When he suggested at one point "maybe there is some division among you" about the convention rules, Carter was greeted with loud cries of "no, no."

"It is almost inconceivable to me how a brokered, smoke-filled convention could be called an open convention," he said, turning Kennedy's rhetoric on itself. He stressed that 20 million Democrats participated in the caucuses and primaries that have given him at least a 300-vote lead over Kennedy.

Then, he added: "Let's not let the party bosses control our party.

"I have absolutely no doubt no matter what rule is adopted Fritz Mondale and I will be nominated in New York."

As a sign of the Carter's camp confidence, Mondale told visiting delegates yesterday that he would be nominated for vice president at the convention by Douglas Fraser, president of the United Auto Workers.

Fraser personally endorsed Kennedy early in the Democratic race and members of his union have worked hard to oust the president. His agreement to place Mondale's name before the convention sets the stage for a rapprochement between the auto workers and Carter in the fall campaign.

Yesterday was the first dull work day for the "Committee to Continue the Open Convention," organized by a collection of mostly junior House Democrats who are worried that renominating the presently unpopular president would hurt Democratic candidates at all levels in the November election.

"As of today, it appears that the Democratic Party in 1980 is headed for a historic disaster," Rep. Michael Barnes (Md.), a leader of the open convention drive, told reporters. Barnes said he is concerned with the nominating process and not with who is chosen nominee. But he went on to say that "our party is bigger than Jimmy Carter . . . the delegates ought to consider the party."

Barnes said his committee would lobby Carter-pledged delegates directly and would also be contacting "people who can persuade delegates," such as prominent officeholders, interest-group leaders, and union officials. He said the committee intends to spend several thousand dollars on newspaper ads to greet the delegates upon their arrival in New York City next weekend.

The move toward direct lobbying of delegates martked a new stage for the open-convention campaign.Until yesterday, it existed mainly as a media phenomenon, feeding itself through daily press conferences and news leaks.

Yesterday the press conferences continued, including one at which several Democratic former members of Congress announced their support for an open-convention rule. They spoke briefly, then accepted questions from the press. One of the questions: "What are your names?"

Still, the open-convention drive has gained support from some prominent Democrats, and they in turn may be having some influence on Carter-pledged delegates.