House Democrats, who control the only segment of the federal government left in Democratic hands, begin President Reagan's second term with sharp divisions that could hamper their efforts to restore the party's national image and strength.

The most pervasive division is the upheaval between older members, who run the House, and younger members, who would like to run it.

Nearly two-thirds of the 252 House Democrats were elected for the first time after 1974. They have demanded more control over how the House operates and this month orchestrated the overthrow of the 80-year-old chairman of the Armed Services Committee.

At the same time, House Democrats acknowledge that they must find a way to bridge the differences among their ideological factions -- New Deal liberals; younger, less doctrinaire moderates, and southern conservatives -- or watch Republicans make further inroads.

Finally, the announcement by House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) that he plans to retire in two years has set off a succession struggle that lawmakers said has begun to make the House more chaotic than usual.

"Right now the Democrats are going through an identity crisis. Do we represent the New Deal? Do we represent a minority party? Do we represent alternatives?" said Rep. Leon E. Panetta (D-Calif.).

"The question for everyone is: What is the Democratic Party all about? Reaching that conclusion will depend on not only what happens on the budget, but what happens on arms control, tax reform, Central America and immigration," he said.

Nowhere are these divisions more likely to appear than in the Democrats' efforts to reduce the federal budget deficits.

For now, Democrats are content to keep "their powder dry and mouths shut" on the subject, according to one Democrat, until Reagan sends his budget and deficit proposals to Congress.

But they are split on how to move after that, with some Democrats inclined to play the loyal opposition by letting Reagan and Senate Republicans take the lead -- and, they hope, the political heat -- and others determined to get the party off the defensive by producing a clear alternative.

"They won 49 states and 60 percent of the vote, and they have the responsiblity to lead. We're not going to bail them out," said Rep. Mike Synar (D-Okla.). "We're going to let the American people see these people for what they are. At least for a while."

But, Rep. Richard A. Gephardt (D-Mo.), the new chairman of the Democratic Caucus, said, "We control this branch of the legislature . . . we have to produce a legislative product. We have to say what we are."

Some of this is familiar ground for Democrats who survived the Reagan landslide of 1980, which placed the Senate in Republican hands. But what distinguishes House Democrats this year is that they feel compelled to do something different.

"A lot of us think if we don't do some things, make some changes, we're going to be the minority party," said Rep. Dave McCurdy (D-Okla.), a conservative.

Majority Leader James C. Wright Jr. (D-Tex.) said Democrats saw a "grotesque distortion" of the party during the election as "having grown old and fat and out shape and bereft of new ideas." He added that members of the party from across the political spectrum feel the need to do something about that image.

Initially, at least, that means that various factions of the party will try to work together on the budget, unlike in 1981, when a group of conservative Democrats, the "Boll Weevils," bolted the party and helped pass Reagan's budget and tax programs.

This year conservatives say they think that they will feel more welcome with the Democrats.

"I believe we are going to have the best opportunity in six years to be a positive part of the effort," said Rep. Charles W. Stenholm (D-Tex.), the leader of the conservative Democratic caucus who in December backed away from a symbolic challenge of O'Neill, after the speaker promised to give conservatives a greater voice in party councils.

House Majority Whip Thomas S. Foley (D-Wash.) said he senses "a merging" of interests that will allow Democrats to get together on the budget in new ways.

But wide differences of opinion exist on the key elements of a deficit-reduction package.

For instance, most Democrats say that, politically, they cannot go along with an effort to raise taxes unless Reagan asks for it. However, some Democrats, such as Ways and Means Committee Chairman Dan Rostenkowski (D-Ill.), say they think that spending cuts will not be sufficient to reduce the deficit substantially. A number of southern Democrats say that they would oppose a tax hike even if Reagan were to ask for it.

In addition, there is sharp disagreement over whether they should push a tax-simplification plan, either the leading Democratic version or one expected from the administration.

Democrats also are divided over whether to include Social Security in any deficit-reduction plan. Senate Republicans have indicated that Social Security would be frozen under the $50 billion deficit package they are considering.

Some House Democrats, including the new Budget Committee chairman, William H. Gray III (D-Pa.), have said they think that a deficit package should treat everyone equally, rather than exempting one group, such as Social Security recipients. But others have ruled out touching the program.

Defense spending is equally troublesome. Liberals and moderates say they think that defense cuts must make up a major portion of any deficit-reduction package. Many southern conservatives, who have long backed the Pentagon, have come to think that military spending has gotten out of hand, but they are unlikely to accept major cuts.

One issue on which there is wide agreement is that to craft a Democratic alternative and a new image, the party needs more effective direction.

"We need leadership," said Rep. Buddy Roemer (D-La.). "We have more talent than we have leadership."

But that also is the subject that has proven the most divisive, bringing to the fore the generational split in the House that mostly had simmered below the surface.

O'Neill's decision to retire in 1986, many Democrats said, has left a leadership vacuum at the top at the time when strong leadership is necessary. O'Neill still controls the inner workings of the House, but people are looking elsewhere for the future of the House Democratic Party.

"I think Tip made a gigantic mistake by announcing it his retirement so early," said one of his closest friends in the House. "People are looking for other fields to harvest. He's a lame duck."

Wright, a Texas moderate, is viewed as the one to beat in the speaker's race. And he carefully has tried to work with all the Democratic factions. But a half-dozen others are mentioned as possible contenders, and many younger Democrats have said they will fight to make one of their own speaker.

"Jim Wright's got a very, very large field of eggs to cross. Not that he's not capable of doing it," Rostenkowski said.

Another Democrat said everyone expects the 1986 contests, from speaker on down, to wash over into nearly everything the House does for the next two years.

In the meantime, the generational rift has begun to tear.

Nerves on both sides were rubbed raw by votes in the House Democratic Caucus that pitted much of the leadership and many longtime members against many of the more recent arrivals. Generally, the older crowd won.

But newer members succeeded in tossing out the frail and conservative Armed Services Committee chairman, Melvin Price (D-Ill.), and replacing him with a relatively junior member, Rep. Les Aspin (D-Wis.), 46, an aggressive moderate.

Younger Democrats had argued that Price was too frail, the committee was too easy on the Pentagon and that the Democrats needed a better, more aggressive image in the defense area.

In addition, some of them said, if the Democrats were going to change their image they needed to shake up the House status quo and seniority system -- an obvious challenge to the older, more entrenched members of the House.

O'Neill and other longtime members strongly opposed the challenge to the House's seniority traditions. The speaker is said to have felt that the move was an unfeeling power grab by the younger members, and many harsh words were exchanged between the groups after the vote.

O'Neill for years was supportive of the younger members and is personally well-liked. But several younger Democrats said they took his decision not to reappoint Rep. Tony Coelho (D-Calif.) to the Democratic National Committee executive board as a sign that relations would not improve soon.

"The Price thing was only the tip of the iceberg," said a younger Democrat. "It showed the leadership's lack of understanding about what is happening, the need for change." He predicted that if accommodation is not made to younger members when similar issues arise there will be "all-out war."

One loyal O'Neill supporter said, "We were on the wrong side of that Price issue."

Another Democrat said the real concern is that internal struggles will overtake everything else, preventing House Democrats from becoming the organized, unified and focused group that may be necessary to survive as the House's majority party.

"All of us want to be in the majority," he said. "We don't want to be in the minority . . . . That's no fun. We realize that."