An unusual venture in cooperation between the Washington public school system and a small private school is in serious jeopardy because the school. Hawthorne Senior High, is in deep financial trouble and the school system has refused to help it out.

For the past five years Hawthorne, which regards itself as innovative but academically tough, has occupied a public school building, the 106-year-old Summer School at 17th and M street NW. Hawthorne has been allowed to use the building rent-free. In return it has enrolled 41 D.C. public school students each year without charging tuition, normally $2.175.

However, this year Hawthorne has lost foundation grants that used to total about $30.000 annually. Its fee-paying enrollment has dipped to 70, and many of them do not pay full tuition.

Since last summer, the directors of the school Alexander and Eleanor Orr, have sought unsuccessfully to get money and supplies from the D.C. public school system. Recently, the Orrs threatened to drop the 41 D.C. scholarship students at the end of the first semester on Friday unless the Washington School Board promises to give them something more than the old Summer building.

But D.C. school officials say that if the scholarship students are required to leave Hawthorne, then the school will be required to leave the building.

"That's our agreement," said Washington School Board President Conrad P. Smith. "If Hawthorne doesn't allow our students to remain in their program, then we won't allow them to remain in our building. If they're having financial difficultes, there's no reason for them to single out the 41 D.C. students."

But the Orrs contend that the free rent for the Summer building doesn't cover the cost of the scholarship students. In addition they would like Hawthorne to serve as a model for a string of "public independent schools," partly financed by the school board but each having full-control of its own curriculum and teachers.

In any event, the Orrs say, Hawthorne needs $30.000 to keep running until June. Unless it gets the money or cuts back on students and faculty, the school will run out of funds by mid-March.

"We've looked upon this program as a wonderful chance to experiment in public school-private school cooperation," said Eleanor Orr. "But now the foundations have told us that we can't get any more money from them unless there's evidence from the public schools that they too value the program. Without that, the foundations say they won't keep pouring money down the drain."

However, D.C. school Superintendent Vincent Reed declared, "Legally, we can't give them public school money or public school supplies to operate a private school, and I don't think we ought to do it anyway. I don't like the precedent. I don't think we should pick up any private school that's going broke . . . Besides, we don't have enough for our own schools this year.

"Look, I don't want to get into a fight with the Orrs," Reed added. "But if they break the agreement and drop our students, then they'll have to leave our building. They want to hold 41 kids hostage to force us to give them something, and I have real problems with that."

The Orrs, now in their early 50s, call themselves "dumb idealists." They have been operating a private school in Washington since 1956.

As its peak in the late 1960s Hawthorne had 170 students in a modern gray concrete building at 501 I St. SW. Included were the children of such notables as Rep. Don Edwards (D-Calif.), poet Howard Nemerov, television newscasters John Cancellor and Martin Agronsky, Jerome Wiesner, science adviser to President Kennedy who now is president of the Massachusetts Institute of Technology, and Mary Finch Hoyt, who now is press secretary to President Carter's wife Rosalynn.

The Orrs own four children, however, did not attend. Instead, they have gone to public schools in Fairfax County, where the family lives.

Throughout the years Hawthorne's program has stressed art, poetry and literary classics. The school has a reputation for political liberalism and tough final examinations. Classes are small and informal with students calling the 12 teachers by their first names.

"We started to become unpopular around 1968," Alexander Orr said, "because we became involved in the wrong things at the wrong time."

For several turbulent years Hawthorne's building was used as a staging area for demonstrations against the Vietnam War. It housed hundreds of partipants in Martin Luther King's Poor People's Campaign in 1968 and for the counterinauguration against President Nixon in 1969.

By 1970 enrollment had fallen sharply, Orr said, with most fee-paying blacks being among those who left. Debts piled up. At one point the school had its tuition money seized by the Internal Revenue Service because it failed to pay taxes withheld for employees. Later, its mortgage was almost foreclosed.In early June 1972 Hawthorne's board of directors sold the Southwest building despite strong objections from the Orrs.

A few weeks later, the D.C. School Board, then headed by Council member Marion Barry, agreed to let Hawthorne use the Summer building and, in return, Hawthorne would accept the 41 scholarship students, to be selected by lottery.

Mrs. Orr said last week that several school board members and former superintendent Hugh Scott agreed informally that the school system would eventually contribute more the Hawthorne than just the building. But no record of any such understanding was ever made.

The 1974 renewal agreement between hawthorne and the school system, which is still in effect, was strongly opposed by former superintendent Barbara Sizemore, in part because she felt the small classes, which average about 10 students, gave these students an unfair advantage over those in public schools.

Actually, the scholarship students have had a mixed record at Hawthorne. According to the Orrs, about half of them have left or flunked out without graduating. Most of those who survived, though, have gone on to college. Their average college board scores have been about 150 points higher than the average for D.C. public school seniors.

"When we started," Mrs. Orr said, "our assumption was that there would be no great difficulty in assimilating these students (from the D.C. public schools). What we found was that in terms of what these kids had learned in D.C. their education was abominable. I can't find the words to tell you how awful it was. The difference between what they know and what their transcripts say they know is so great. When you deal with it, it gets to your gut."

To try to find out what their students know, the Orrs developed basic skill tests in mathematics and grammar, pegged to average achievement levels for seventh grade, Mrs. Orr said that almost all the incoming D.C. students fail the tests, although all those who graduate from Hawthorne are able to pass it.

In a report to the Meyer Foundation, from which the school unsuccessfully sought funds for this year, the Orrs wrote:

"These kids are not just behind. They are handicapped . . . The newspapers tell us these kids cannot read, but it is more complicated than that. We have heard them read out loud almost perfectly, with all the intonations characteristic of understanding, and not have the slightest idea what the words are saying . . . They relate to words as if the words are knowledge itself, as if the words referred to nothing but themselves. Tho these kids language is a tool of imitation: it has never been experienced as a tool with which one can think."

"Everyday personal interchange with (these students) convinced us that they were not stupid," the Orrs added. "We are convinced that this limitation is correctable; that it is caused by their previous education." But even after three years at Hawthorne, the former public school students failed 71 percent of their math and science courses. "To us this is partial evidence of how long it takes," they continued, "to unlock the natural intelligence of these students."

Many of the parents whose children attend Hawthorne after being in public school say they don't mind the high failure rate or the relatively low grades their children receive.

"Initially, it embarrassed me," said Gail Wright whose daughter Theresa had been an honor student at Rabaut Junior High in Northeast Washington. "Then I realized, 'My God, she hadn't been educated.' The D.C. schools had given her all sorts of awards, but the child wasn't functioning. Now I'm pleased because I know she's being challenged."

The students seem to like Hawthorne too.

"The classes are more interesting," said Debra Webb, who comes from the Deanwood section of Northeast. "We don't want to leave and go back to public school . . . We're one big family here."

For the past two weeks the parents of the former public school students have been lobbying Reed and members of the school board, and have held a press conference, demanding money and supplies for Hawthorne.

In a long statement, they charge that until Jan. 5, Reed had led the Orrs to believe that there would be "no problem" in getting public school supplies. They say their children will be hurt severely if they are forced to switch to public school in midyear.