COMING UPON David Letterman's TV show on the other side of midnight is like discovering a homey little diner on the outskirts of town, the kind where the food is pleasantly greasy and the waitress knows all the customers by name. It's like going back to your college hangout and finding out that that nothing has changed, that the old gang is still up Cuba Libres and sobbing over schmaltzrock on the jukebox.

David Letterman is what television is all about, for better and for worse, and four nights a week on NBC, on "Late Night with David Letterman," it's almost invariably for better. Sometimes this program is so funny, and so winningly funny, that you're tempted to call up friends and tell them to tune in (the way people did with the original "Saturday Night Live" and, more recently, "Police Squad!"). Except they might not stay your friends if you rang them up between 12:30 and 1:30 a.m., a time of night when urban animals, shady characters and David Letterman reign supreme.

"You folks are easily entertained, and boy, did you come to the right place," Letterman told a raucous audience in NBC's studio 6-A one night. Last Tuesday he looked into the camera, flashed the gap-toothed grin that stops just short of "what-me-worry?" and said to viewers, "Keep reminding yourself, 'Yes, this is actually network television.' "

Letterman may have to keep reminding himself of that as well. In fact, there is within this 34-year-old lad from Indianapolis still a little living spark from the 10-watt radio station he used to work at when attending Ball State University--an Essence du Broadcasting at its most direct and intimate. Letterman's television hangout, such a comfortable place to put your feet up and have a few good har-hars, is the successor to Studs (Terkel's) Place, Garroway's at Large, Jack Paar's rubber room, Steve Allen's crazy attic of the soul and the public workshop once manned by the magnificent Ernie Kovacs.

Even the way announcer Bill Wendell introduces him each night is pretty ingratiating:

"And now, a man who is frightened by the slightest change in air temperature--David Letterman!"

"And now, a man who once saw Maurice Evans in a restaurant--David Letterman!"

And now, a man who suffered a humiliating defeat on NBC daytime in 1980 but lived to tell the tale--lankish, prankish, boyish and goyish; comedian, sports fan, aficionado of "Stupid Pet Tricks," and one-time weekend weatherman at WRTV in Indianapolis . . . David Letterman! He's sitting with his big feet on the round table he uses as a desk in a Rockefeller Center office that also contains: a small fridge filled with yogurt and Perrier; a giant economysize bottle of Extra Strength Tylenol; a pair of very large running shoes (those would be Dave's); a cheap clock radio tuned to a station that seems only to play the Doors; an apple, and, tossed onto a couch, a pair of pants, which he apologetically picks up and throws into a corner, by way of cleaning the place up.

Something about Letterman suggests a sandwich wrapped in foil that has "Don't Litter" written on it and was brought in from a deli a block away. His face is as open and rumpled as a well-worn baseball glove. He's not the boy next door; he's the broadcasting major next dorm. He's just a big tall funny guy who's a great gift to television--the kind of gift you buy at novelty shops, maybe, but a gift just the same.

He always knew he'd end up on TV some day.

"Yeah, I had the feeling," he says. "For some reason, when I was a kid, I was aware of Arthur Godfrey's daytime show and Garry Moore, and I found it fascinating to see these people sit at little desks and have these microphones in front of them and talk, and I thought, 'This is amazing,' and I sort of would pretend to be on TV or radio and thought, 'This would be great,' and then I forgot all about it until I got into high school and took a speech class, the first class I didn't have to work too hard to get good grades in. And I thought, 'Now the trick is, how to find out how to get paid to do this.'

"The first real impression of television I have is that I used to come home from school and watch Johnny Carson on 'Who Do You Trust?' a comedy game show on ABC daytime in the '50s , and it would be interesting to see that show now, but at the time it seemed like here was a guy in a suit who made fun of morons and did it without hurting the feelings of the morons. And I thought, 'Gee, that's a wonderful talent to have.'

"And then the next thing I really liked was the Steve Allen Westinghouse show--my parents would only let me stay up on Friday nights to watch it--and it just used to make me laugh till I had tears in my eyes. Again, it was a guy in a suit doing silly things--Steve yelling 'shmock shmock' and going out and raiding the Hollywood Ranch Market across the street. When we were putting together the daytime show, we got a bunch of tapes of that, and as stupid as you may remember that being, looking at it again, it's much more stupid, but it was great to see.

"To me, that's what television oughta be." DAVID LETTERMAN has a deep feeling for the superficialities of TV. His show celebrates and parodies them. It's full of invention and an '80s version of whimsy. One night, Letterman announced that he'd left his "artificial tooth" at home and so a housewife from New York State was recruited from the audience to host the show for a few minutes. Another night's opening was played entirely from Dave's Point of View; a hand-held camera enabled viewers to see everything from his perspective but not see him. On a cue card it said that one of the guests was to be "Burt, the Human Caboose," but then up popped another cue card imploring "Dump the Caboose!" and Letterman could be heard to say, "Darn the luck, Burt, we're out of time."

Once a month, Letterman's Monday-through-Thursday show also appears on Fridays, for 90 minutes instead of an hour, when "SCTV" gets a week off. The first of these pseudo-specials was "Dave," a parody of a pseudo-special built around a pseudo-star. In July, Letterman and company offered their Christmas special six months early, to beat the rush. The next 90-minute show will be seen this Friday.

Of course the Letterman show that has made the noisiest splash aired in late July, when conceptual comedian (or whatever he is) Andy Kaufman got into a brawl on the air with professional wrestler Jerry Lawler; the fight ended with Kaufman throwing coffee from Letterman's cup in Lawler's face. Producer Barry Sand says the incident was not staged, at least not by anyone connected with the show. "If I had known about it, I wouldn't have let him pull that stunt," says Sand.

But far more memorable to the show's cult of fans was the night Larry "Bud" Melman took over as guest host while Letterman took a much-needed rest at home, where a camera caught him eating a TV dinner in front of the set and going through a flyer from a hardware store. Larry "Bud" Melman (Not His Real Name) has become something of a mascot for the program. Looking to be approximately in his later fifties, Melman is the very soul of unassumingness, definitively ill-at-ease yet wonderfully funny on camera (each of his appearances says, essentially, "There isn't anybody who doesn't deserve to be on television").

Usually he pops up at the end of the show, materializing in a cut-out circle the way Groucho used to do at the end of "You Bet Your Life" and saying, "This has been a Melman production." Except that one night he said, "This has been a Melnor--er, Melman, production." He also does commercials for Melman's Bus Lines, the bus company that encourages children to run up and down the aisles and sit on the driver's lap.

On the night he guest-hosted, Melman dutifully delivered a monologue and then interviewed the first guests, physical fitness nuts Jack and Elaine LaLanne. When Jack LaLanne tried to persuade Melman to get down on the floor and put his legs up on a chair for some exercises, Melman cried out, "Oh, God!" with the terror of the truly unfit. Things were going very well--too well, apparently, for later in the show the picture suddenly went dead, "Please Stand By" was seen on the screen and, after a few seconds of "Love Is a Many-Splendored Thing," Letterman was back in his chair.

Sand says the real identity of Mr. Melmac--er, Melman--is being kept secret for now but may someday be revealed with a huge flourish of publicity. "We're trying to keep him a mystery," Sand says. HIGH JINKS like this have not earned the show spectacular ratings; it gets a 14 percent share of the audience viewing at that hour, beaten by the garbagey old reruns CBS puts on, but it is doing better than "The Tomorrow Show," previous tenant in the time slot. And, according to Sand, Letterman gets super demographics: the 18- to 34-year-olds advertisers crave.

All of them are members of the television generation--or one television generation, anyway. So is Letterman. "Before, a stand-up comic or an observational humorist had the entire horizon from which to draw comedy," Letterman says. "Now you get people who have been raised on television and their source of comedy is nothing but television. And I think that may be a little limiting. And I do that. My stand-up act was mostly, 'Didja see this on television, it was so stupid, wasn't it?' And it was funny because there are millions and millions of people who went through the same process. But I don't know that it's the best approach in the long run."

Maybe not, but partly because of it, Letterman's show has a shaggy, mom-and-pop feel that spoofs the pretentiousness of big-time TV. Among the uncountable refreshing things about it is that it never mushes over with show-biz sentimentality. You don't get all that smooching and hugging and actors telling each other how marvelous they are. When Letterman appears at 12:30, he's exorcising phoniness--years and years of cloying bull. Like "SCTV," he addresses the young viewing audience's new awareness about TV and its fakery.

At interviewing, however, Letterman is a bumbling flop. He's okay with comedians who can rattle on from one commercial break to the next (or who, as Robert Klein did hilariously last week, bring along and narrate home movies of their bar mitzvahs, which turned out to be a high point in the television year). And Letterman's love of sports comes through when he talks with sports figures. But he's a TV kid, so he can't relate to a raconteur like Peter Ustinov, who launched into what could have been an amusing digression one night only to have Letterman shoot it down with, "Oh, you're imitating an English-speaking American, right?" In Letterman's defense, Sand says this isn't an interview show in the first place; "it's a comedy show with interviews."

Letterman says he's made a conscious decision not to do much topical material, partly because he doesn't want to try to compete with Johnny Carson's monologue, which airs an hour earlier. But he's never been one to leap into frays. At Ball State, even in the turbulent '60s, he was rigorously passive and apolitical.

"Quite honestly, the only protest that I ever was involved in was, we thought we maybe could get the cafeteria cooks to wear hairnets," Letterman recalls. "And that was about it. Meanwhile, all over the United States, protest. Kent State was not that far away, and even Bloomington, the campus down there, was a hotbed of protest. But not Ball State. We were pretty well protected."

How do you feel about that, Dave?

"I feel stupid. I feel silly for not having at least the curiosity to find out what was going on elsewhere. But it just completely went by me. Or I let it go by me."

Nevertheless, this may have something to do with what appears to be Letterman's Very Even Keel. He doesn't seem like the kind of guy to look for the nearest window ledge should his late-night show not make it. Still, he does say, "I feel like I have to avenge myself for the morning show," the 1980 daytime program that had a run of 18 weeks until it was replaced by "Las Vegas Gambit" and "Blockbusters."

"In the beginning, it was the worst experience of my life," he says of that program. "It was like the fight where 'Benny the Kid' Paret was killed, and it was just, you were on your feet and they wouldn't stop it. You know, you could hear people scream, 'It should be stopped, they should stop the fight.'

"Once we knew we were in trouble and got canceled, then we adopted this kind of attitude like, 'Oh yeah? We're canceled? Fine.' You know, that kind of defensive, high-school-punk attitude about it. And then it got to be fun, because someone would say, 'Oh, we can't do this' and we'd say, 'What are they gonna do--fire us?' " By this time, Letterman says, his show had lost "most major markets in North America," as NBC affiliates dropped it and picked up syndicated fluff to fill its time slot.

There is, clearly, some residual bitterness."People I worked with on the old show at NBC probably think I have some kind of an attitude problem, 'cause I could throw little fits when they'd say, 'We want you to have a psychic on,' and I'd say, 'No, I don't want to have a psychic on.' And they'd say, 'Well, try it,' and we'd try it, and it would just be awful and embarrassing. I didn't know really how you had a discussion on a topic like that with a network executive. My forte was not personnel. I didn't realize that politically what you have to do is say, 'Fine, we'll be happy to,' and then you forget it. You forget all about it. And then six months later, they say, 'Did we have a psychic on?' and you say, 'Yeah, well, the plane got stuck in Alabama, they couldn't get here.' "

Although NBC failed with an intelligent daytime talk show like Letterman's, it was thought earlier this year that a dumb daytime talk show with Regis Philbin might work. Letterman was asked at that time how he'd feel if Regis were a smash hit. "I thought, what if this son of a bitch--I know him and have a lot of respect for him, but what if he's successful? It was just a fleeting moment, 'cause I hope he is. After you've been through this crap once, it's impossible to wish anybody else in television ill."

A little while later, Letterman said, "I'm sorry I called Regis a son of a bitch. Please don't print that." But Philbin's show was no more a smash hit than Letterman's had been, and it too was canceled.

Letterman has sometimes found it hard to put the shame of the daytime debacle behind him. "I keep seeing in print, 'Letterman, whose morning show was A DISMAL RATINGS FAILURE,' " he says. "I can remember when it was announced that Fred Silverman was leaving NBC and we turned on the Nightly News and John Chancellor introduced a story about the faltering network and they mentioned 'Pink Lady and Jeff' and 'Supertrain' 'AND THEN THERE WAS THAT ILL-FATED MORNING SHOW,' and I saw my goofy face there on the Nightly News."

However, some of the talent from that show accompanied Letterman into "Late Night." It really is a family. The executive producer, Jack Rollins, is Letterman's agent. Producer Sand came aboard during the run of the daytime show. And the head writer, once again, is Merrill Markoe, with whom Letterman is living. He says this arrangement was "awful" during the run of the daytime show because all the agonies of the day came home with the couple at night. But now it's working fine. They live with their two dogs in Manhattan. However, Letterman says, "Her dog is a pathological maniac." Photos of the two animals sit on a tacky end table in his office.

Markoe bursts into that office suddenly, looking very serious and obliging at length when asked about the real people who sometimes appear during the show's remotes from the streets of New York (Letterman says, "It's very convenient for us to do the show here; you just take the camera outside and before it's stolen you get some great videotape"). After a long explanation Markoe concludes, "You get, like, pure quintessential human behavior to look at. Pause, glance at Letterman. I'll be leaving now."

Letterman has often been mentioned as the logical successor to Johnny Carson for that dark, dark, unimaginably dark day when Johnny retires. To be called another Carson is "high flattery for me," Letterman says. But then it's pointed out to him that the inventiveness of the Letterman show, as a television production, makes Carson's program look even stodgier than it already did. Letterman, whose program is "coproduced" by Carson Productions, says, "The truth of the matter is, the reason for 'The Tonight Show' is to see Johnny Carson. I mean, do you circle in your TV Guide when you see Angie Dickinson is going to be on? You tune in to see Carson, and I think that, at this point, is the function of that show.

"America has gotten accustomed to see Johnny Carson. They like him and they know him and they know what to expect. And he's smart and he's hip and he's funny--and he can be dirty, but it's okay, because he's Johnny Carson."

Some things about "Late Night with David Letterman" don't work. There's this jumpy, jerky little band on the set, with leader Paul Schaefer (formerly of the "Saturday Night Live" band) saying things to Letterman that are supposed to spoof inanity but usually are just inane--although the band's commercial lead-in performances of rock goldies like "This Old Heart of Mine" are right in keeping with the funky tone. Often the show is overbooked, and a still-productive guest will be thrown off just to keep up a pointlessly hyped pace, but at least this helps keep it largely unpredictable.

"It's really a souffle'," says producer Sand. "If you want to find out what's going on in the world, don't tune us in. We have a lot of fun. And we take a lot of chances. Whatever goes, goes, and we let it rip. We hope to reach a level where it's always good and sometimes great."

That level, for all intents and purposes, or for most intents and a great many purposes, has been reached. When Larry "Bud" Melman was guest-hosting, he was made to say, as a jape at show-biz bombast, "This show is the reason television was invented." Yes, it was just a joke. But there is some truth in it.