Moshe Dayan is a prowler.

Sometimes at night, trailed by the ubiquitous and burly security squad that stays with him always here, the Israeli defense minister wanders through the lobby of the Madison hotel where both Israeli and Egyptian peace-talk delegates are staying.

The Madison, that home away from home for stars and other household words, is used to the whims of high visibility clientele. But even regular Madison lunchers will gawk at Moshe Dayan lunching in the coffee shop, at Ezer Weizman at another table (always), at the wall of security between either and the rest of the world.

Several thought the eye-patched Israeli hero looked frail and old, in person much less glamorous than either his image or his current newspaper pictures would indicate.

In any case, anyone, even the most innocent well-wisher, who moves in the direction of any of the dining leaders suddenly finds his way blocked by the almost magical interposition of a brawny guard.

"You get the impression," one lunch-time observer joked, "that half the people in the room are packing a rod."

According to one doorman, the Egyptians and Israelis never dine together. "I don't think they've even been in the same dining room together," he said.

Comings and goings outside the Madison are controlled chaos. There are the usual complement of regular-issue light-flashing police cars along with some mysterious-looking vehicles with triple-flashing lights and heaven knows what-all inside.

Americans and foreigners leap into or out of limousines through gantlets of guards into (or out of) the hotel.

A State Department agent in tinted glasses with an ear radio receiver that looks like a hearing aid, talks down into his tie which conceals a pencil sharpener-sized radio. "What about the cars? What about the cars?" he asks tersely.

Within minutes the cars have driven off, and the agent is left alone with a cheerful doorman, with whom he chats.

Both the Egyptian and Israeli delegations of about 10 each, including the defense and foreign ministers of each country, are staying at the Madison. In fact, some of the still see-sawing talks have been held at the hotel, although Blair House is the official site.

Do the hotel staff members mind the security that has descended upon the hotel, the agent is asked. "They're used to this." he says confidently. "They're used to this kind of security and important dignitaries."

A waitress insists the presence of the delegations from the Middle East makes no difference to her. "I've served room service to Jimmy Carter," she says with a smile.

They're normal people, is what the employes say about the delegations. True, but they come to the Madison with an abnormal amount of flurry, security, and problems, even for the Madison.

"At the beginning everyone was tense," says one employe. "There's too much security. First of all, before you can work with them, you have to be cleared by the FBI. I don't think that's right. I don't think they should pry into your private life before you can work here."

Things have relaxed a little now, she says. There are other guests allowed on the floor where the Egyptians are staying now, and the floor above, where the Israelis are staying, she said. "It's not that there are less security people, but we seem to be used to each other now."

Still, the security personnel let you know they are there. "When Dayan is ready to leave, they don't want anybody on the floor," the employe said. "I think it's an honor to see them. They ought to at least let people get a chance to see them. Then they can say they've seen them."

And then one must get used to working with both the Israelis and the Egyptians. "The Egyptians are better to work for," pronounces one employe. "The Israelis are more demanding. Every time they want something, they want it right then and there. They just seem like they can't wait. They want it immediately - boxes, strings. Have you ever heard of a glass table with a light underneath? They wanted something like that. They described it as a table that you could put a map on and read. They want so many different things I've nnever heard of."

"The Israeli love chocolate candy," said one employe, grinning. "We give it to them everyday. They say candy was scared to them." These, she noted, are not plain old candy bars they're getting either - they're boxed Godiva chocolates.

"But I think," she continued, "we're all getting to the place where they're trusting us and we're used to them."

One waiter grumbled about the negotiators' tipping practices. "They're cheap people," he said of both sides. "They don't know how to tip."

And one window-washer said, "Usually we're all on the rush, trying to get things done. Now we're more quiet and more cautious. I try to do my job in a very clean way."

One limousine driver who has been assigned to two junior members of the Egyptian delegation said he takes "his people" shopping most evenings to Tysons Corner or White Flint. "They don't have any perception of time," he said. "They say they'll be down [from the hotel rooms] in five minutes and they'll come down an hour, maybe two hours later."

"But they're very nice," he said. "They sometimes invite me to have dinner when they are the guests of embassy people. They treat me like a friend." Even the Madison, which usually relegates drivers to the side door, "ley me come in through the front when my people call me."

Security remains ever-present. According to one State Department agent, security knows who is on the floors where the delegations are at all times. Anyone walking into the rooms of the suites is accompanied by an agent who stays with the person. The only exception is the private guest. Chambermaids making up the beds are accompanied by an agent. The food in a room-service order brought to a delegate's room is checked for incineratives and explosives. The tray is checked as well. Then the waiter with the room-service order is accompanied into the room by an agent who stays until the waiter leaves.

When the limousines pull up, "it's like mass confusion," said one doorman. "The negotiators are perfectly willing to talk to people. They're gentlemen, these guys. But the security agents just them right into the hotel. They won't even let them stop," he said, holding up his white-gloved hand. "But that's their job."

Marshall Coyne, owner of the Madison, sailed through the front doorway during a relatively quiet moment. Dapper in black pinstripe suit, he was affable, but not talkative. When asked about his high-powered guests he raised his hand in a majestic gesture and said, "I can't talk. Please. Not now."

Still, through the security guards, there are chances to glimpse the negotiators, which the staff of the Madison relishes as much as passersby on the street.

"I try to look good," said one neatly dressed doorman. "I'm the first person they see. And then, I want to look good for the cameras," he said with a grin, pointing to the photographers waiting outside the Madison for the limousines.

This doorman has watched both sides of the negotiating teams carefully. "Moshe Dayan seems to be concentrating on something when he comes in," the doorman said. "He seems to be in a hurry."

To the Egyptians, the doorman says "good morning" in Arabic and they heartily respond likewise.

One night last week, a member of the Egyptian, delegation, characterized as "very nice" by Madison staffers, walked across the street from the Madison with security agents to admire the chrysanthemums being sold by flower vendor John Lauferbach across 15th Street. "He bought a bunch for $1, then got in the back of a limousine," and speed away, recalls Lauterbach.