There was no love lost on First Lady Rosalynn Carter by some of the volunteers at yesterday's annual "Affair of Heart" benefit luncheon-fashion show at the Sheraton Park Hotel.As far back as anyone could remember, the First Lady or the wife of the Vice President had been a guest of honor, but yesterday neither was able to attend.

"It's unfortunate that the White House couldn't show support for the work of the heart volunteers in this city," said Dorothy Stewart, a member of the women's board of the American Heart Association's local chapter. In the crush at the door before the luncheon, Mrs. Stewart had given up her ticket to a guest seeking a place for the sellout affair. an aide to Mrs. Carter. The First Lady has been deluged with invitations at the rate of 50 or more per day, and there are two cartons full now in her office. Yesterday she opted for lunch with the President and "very old friends" from Georgia, said the aide. Mrs. Mondale had "other plans," according to an assistant.

But there were other administration representatives at the head table. LaBelle Lance, wife of OMB director Bert Lance, said she was "sure Mrs. Carter would love to watch fashion shows as much as I do." Mrs Lance showed up in the same red Ultrasuede dress as Mrs. Warren Burger, wife of the Supreme Court Chief Justice. Neither had a clue about who had designed the dress. But then Kasper, who was a guest at the luncheon, hadn't noticed that it was one his many synthetic suede styles either.

Each year there are fewer of these huge fashion show-luncheons, because increasing costs eat into the profits.But yesterday's was a full blown affair, complete with an almost three-hour program. Estee Lauder favors, a chicken luncheon and a choreographed show of the spring clothes of six Seventh Avenue designers.

The designers themselves were required to show up, a rule that caused only one, Geoffrey Beene, to turn down the invitation. Those who came were John Anthony, Gil Aimbez (who designs for Genre), Donald Brooks, Kasper, Richard Assatly (whose firm) is Gino Snow) and Bill Tice, who does loungewear for Swirl.

Washington fashion show-luncheon guests are rarely effusive, and yesterday's audience seemed to appreciate the designers about equally. Said one guest, Lydia Preston, "I certainly couldn't see myself in the minidress (by Richard Assatly in white crepe for evening), but I'm sure my daughter would love them." She said she would prefer a suit by Kasper of John Anthony.

Woodward & Lothrop picked up most of the tab for the show, including $150 for each of 30 models, $1,500 to bring in choreographer David Carter from New York, $750 for a harlequin chandelier and an untallied sum to build a double triangle-shaped runway created by the store's display department plus the expense of a new rug to cover it.

It totalled about $20,000, figured a Woodies executive, but was well worth it, not for any measureable return, he said, but to show support for a local charity.

According to Dorothy Stewart, the show should net at least $15,000, part from the luncheon and the rest from the sale of advertising space in the luncheon program.

One unexpected expense: a Gil Aimbez $1,000 sample dress brought down just for the show was stolen behind the backs of two security guards prior to the luncheon.