PORTSMOUTH, N.H. -- At 10:30 in the morning, the question came from a young woman student at Portsmouth High School. Nine hours later, in a slightly more belligerent form, the same question came from a shopper at the Mall of New Hampshire outside Manchester.

''How are you going to balance the budget,'' he asked Gary Hart, ''when you still haven't paid off the New Hampshire businessmen you owe for your 1984 campaign debt?''

It was the only personal question Hart was asked in his first full day of 1988 campaigning in the state where his presidential hopes were born in 1984 and seemingly ended last May when he ''suspended'' his efforts. All the other questions were on issues in the news: the homeless, the West Bank unrest, the Seabrook nuclear plant, arms control, the federal deficit.

Hart, who has always been a quick study, had his answer down pat when the campaign-debt question came up the second time. ''I've paid off 75 percent of what I owe,'' he began. ''If Ronald Reagan had done as well on the national debt, it would be $35 billion or $40 billion this year, instead of $150 billion.''

But when the student first raised the question, Hart was clearly bothered. He gave a rambling answer, including the remark that ''I'm not the first or only candidate for president . . . who has ended with a debt. For some reason, mine has attracted more attention. I'm not quite sure why.''

If Hart really does not understand why, then he is missing a vital point about the questions his revived candidacy poses for people. They want to know if they can trust a man who has a habit of walking away from things or whether as president he will continue to pick and choose which commitments he will keep.

With his wife, Lee, convoying him through his schedule, a constant presence at his side, few voters are bold enough or insensitive enough to ask Hart about the womanizing charges that caused him to end his candidacy last spring.

William Gray, a Republican retiree, listened in on a short news conference Hart held after the high school class here. He came up to me afterward and asked, ''Why won't you people in the press ask him if he couldn't be compromised or blackmailed as president by that Florida gal or one of his other women?'' At that moment, Hart was walking right behind Gray toward the exit, and I said, ''Why don't you turn around and ask him yourself?''

He looked at Gary and Lee Hart, hesitated, and they were gone.

My guess is that the voters who are asking Hart about his campaign debts are using that question as a surrogate for the question they are embarrassed to ask: How can we trust you after what you did in the Donna Rice affair?

When Marvin Kalb pressed Hart on the issues of trust and judgment in his excellent hour-long interview Jan. 3, Hart asked that the affair be placed in context. ''It's not as if I just parachuted down onto Earth,'' he said. ''I've been around in public life for more than 15 years.''

The problem is that as long as voters have known Hart, and for years before that, he has exhibited a pattern of ''walking away.'' He left behind in Kansas the family's name and church affiliation. He left divinity school for law school. He left his marriage twice and twice returned. He left the Senate to seek the presidency. And he left the presidential campaign, only to return to it again.

Each of these decisions had its own rationale. Hart can rightly claim that every one of those turnabouts, except the last, is a decision many others have made without eyebrows being raised.

But the pattern gives pause, even to those who find ways to forgive. The first Hart-for-president buttons I had seen since his reentry were being worn by Ron and Jean Natoli, who accompanied their candidate to the high school appearance here. Partners in a land-surveying business in Lee, N.H., they had backed Hart as a long shot in 1984 and happily signed up with him again last year.

When he dropped out, Jean Natoli said, ''I had no doubt he would come back, and I just waited for him.'' Her husband was less certain and searched unavailingly for another Democrat to support. When Hart came back into the race again last month, Ron Natoli ''really wrestled'' with the decision. ''I was upset,'' he said, ''not because of Donna Rice so much as his decision to cut and run last May, rather than face down the criticism. But I finally decided, better late than never.''

Others with less of a personal commitment to Hart have an even harder time than Ron Natoli in coming to terms with his candidacy. Hart is giving them little help. He talks constantly -- and well -- about issues and programs. But he answers only rarely and reluctantly the questions that go to his consistency and his character.

He leaves it to the voters to judge whether all that is past -- or whether he is, once again, just walking away from himself.