About 4,500 persons -- including many college students and government employes but very few of the poor and unemployed who had been expected -- marched from the White House to the Capitol yesterday in the Rev. Jesse Jackson's national youth pilgrimage for "jobs, peace and justice." At the Capitol, the ranks swelled to between 6,000 and 8,000.

Jackson, the director of the Chicago-based Operation PUSH, had said he expected 25,000 and has promised that, if the federal government does not respond, the Democratic Convention this summer in New York will be the scene of his next protest.

Mayor Marion Barry, who has proposed cutting thousands of jobs from the D.C. government because of a growing fiscal crisis, was booed when he began to speak yesterday.

Barry in turn blamed Congress for the city's money problems and offered full home rule as the answer.

Speaker after speaker spoke of a sense of desperation sweeping the nation's poor and warned that this desperation may explode at any time, as it did in the 1960s over civil rights.

"We will be holding job and hunger hearings in targeted districts . . . we will be holding voter registrations . . We will engage in selective boycotts . . . We will prevail in this jungle," Jackson told his enthusiastic supporters.

Jackson said this time, as in the fight for civil rights, blacks have as their weapons "our votes, our consumer dollars, our marching feet and our made-up minds."

Unlike Barry, Mayor Richard Hatcher of Gary, Ind., and chairman of PUSH, the black self-help organization, received a wildly enthusiastic reaction from the crowd.

Recalling the words to a poem by Langston Hughes, Hatcher said too many black Americans have suffered their "dream deferred" . . they are out of hope, out of work and out of cash . . . We are poorer, sicker, hungrier than we were 10 years ago. It is a nightmare now. Not a dream," Hatcher said to wild applause.

Hatcher accused the federal government of paying more attention to housing missiles than providing housing for people, and swelling oil company profits, rather than providing work for unemployed Americans.

Jackson's PUSH (People United to Save Humanity) organization had spent about $125,000 for the rally. And over the last month, Jackson spent a lot of time here drumming up support for the march. During an interview earlier in the week, he acknowledged that apathy would be a problem.

"There is a lot of frustration and cynicism in the black community," he said. "There is tremendous agony. There have been implosions rather than explosions in the black community: implosions of liquor and drugs. Our bodies are being bursted, our minds are being blown. But inevitably, implosions turn to explosions."

In the comments of the marchers -- some of whom had come from as far as Illinois and Connecticut -- there was the sense of this anxiety and frustration.

"I came out for the teen-agers," said Saraleta Thomas, who had her two young daughters with her. "I'm a mother of two and I'm in the job market, but I want to make sure there's some security for them."

Although many of the speakers expressed support for the Haitian emigrants seeking asylum in the U.S., many in the crowd expressed concern over the effect that the influx of Cubans and other foreigners to this country may have on the job market.

"I think the Cubans, just like anyone else who's being mistreated in their own countries should have the right to come . . . but I think there should be some kind of quota," said Wyatt Vailes, a student at the University of the District of Columbia, who said he is concerned about finding a job in the advertising and design field once he graduates.

Other speakers yesterday included the Rev. Ben Chavis of the Wilmington 10, who urged the marchers to "carry on the struggle once you go home."

Rep. Parren Mitchell (D-Md.) told the crowd that Congress was to blame for black joblessness because members "don't believe that blacks will rise up against racism."

Ben Boddie, a youth activist from Southeast Washington declared, "This is a message to Marion Barry and the Congress . . . If we don't get summer jobs, they're gonna hear from us," alluding to predictions that cutbacks in the mayor's summer jobs program this year will lead to increased crime, drug use and violence among the city's youth.

"You've heard of the bridge over troubled waters?" asked activist Charles Massey. "Well we got troubled waters in the District of Columbia -- the Anacostia River." Massey warned that the city's young people need jobs this summer which will provide them with training "instead of being errand boys and errand girls. We don't need that," he said.

Participants in the march began gathering on the Ellipse behind the White House as early as 9 a.m. yesterday. Some brought breakfasts and lunches and spread out on blankets until the processional began an hour later.

The group started out numbering around 4,000, according to U.S. Park Police, and picked up about 2,000 persons along the way. The group stopped briefly in front of the White House for a prayer vigil led by Jackson then headed east on Pennsylvania Avenue for the Capitol.

By noon city officials said about 6,000 to 8,000 persons had assembled at the Capital to hear speeches from national and local politicians, representatives from various youth groups and the United Auto Workers. After more than three hours of speeches and singing, the rally ended at 4 p.m. a

"I came from the Atlanta University Center and I came basically because food stamps are being cut and grants to black colleges are being cut. We want to show black students are concerned about poor people," said Marquis Walker, a divinity student at the university center.

There was a contingent from the Wisconsin Steel Mill of Chicago where 3,400 employees have recently lost their jobs.

"Here was 3,400 men willing and able to go to work. They went to work one day and the next day they were reading in the paper their mill was shut down," said Debra Dilworth, the wife of one of the men who is now out of work.

"I have a 13-year-old daughter very much interested in going to Catholic high school, and now her dreams are shattered," added Dilworth, whose husband worked 13 years for Wisconsin Steel. "Where's he going to go now?"