There is general agreement that the Hispanics of this country are on the verge of becoming the second largest minority bloc in American society -- next to the approximately 36 million who are handicapped or disabled.

So where was the air of euphoria at last night's party observing National Hispanic Heritage Week at Meridian House? The answer is that numbers mean very little in America "unless they are translated into political clout," said Graciela Olivariz, who as director of the Community Services Administration is the highest-ranking Hispanic appointee of the Carter administration. The CSA and related agencies hosted the event.

The party was one of several in the week of gatherings here that will lead to the banquet tommorrow night of the Hispanic congressional caucus, which President Carter is expected to attend. Last night the 200 or so guests came mostly from the middle echelons of government agencies and from local Hispanic groups. Most drank white wine and ate from the refreshment table, where the heated Cuban black bean dip was entirely consumed before the party was half over.

It was the political issue, though, that most concerned Olivariz. In 1970 the Hispanic population was about 8 million, compared with 22.5 million blacks. "In the census we've got to pick up at least another 10 million" if the true position of Hispanics is to be documented, Olivariz said. "I would say that if the 1980 census were done correctly, the Latin population would come to about 22 million. It is not just in the southwest and Miami and New York. It is all over," she explained.

"But the great problem is to get to the undocumented workers" who are illegal residents here for periods as short as a few weeks or as long as the rest of their lives. "They are afraid to cooperate with the census, because they are afraid that its confidentiality cannot be trusted," Olivariz said.

Thus, Olivariz doubts that Hispanics will reach their proper position in the census until 1990. Even then, political clout will be hard coming, she said. By comparison with the black minority influence, for instance, Olivariz predicts that for some time Hispanic power in presidential elections will be "minimal." The best she expects is that Hispanics will "pick up some more congressional districts."

For this reason, such events as the annual Hispanic Heritage observances are seen by activists as something more than tokenism; they are efforts to enhance the visibility of a population bloc that has been somewhat lost despite its meteoric growth. "If we did not do this," said actress Carmen Zapata, "the result would be retrogression -- the opposite of progress."