TO BE KNOWN as one of the world's greatest lovers is only an insult to the rest of us who aren't; and the fact that Charles Boyer was so known, for 20 years, testifies to the endurance of his charm. Charm is an empty virtue in most people. In Mr. Boyer it seemed to be a way of moderating passions that one sensed to be intelligent as well as fierce. The quality Mr. Boyer displayed as an actor was devotion - whether to Jean Arthur, whom he pursued from Paris to New York in "History Is Made at Night," or to the discovery of the jewels in "Gaslight." Any man who can devotedly torment Ingrid Bergman, and not have an audience up in hisses, has to be more than a great lover.

Mr. Boyer was in fact a first-rate actor, whose various abilities - like those of so many actors in the '30s and '40s - were rarely seen because they were cast in parts that matched their faces. Mr. Boyer's face looked brave - brave worried or brave happy. So it was natural for him to play in such movies as "Arch of Triumph," where one could read the whole suffering of Europe in those heavy-lidded eyes. He was one of the few men who could play Napoleon ("Conquest") without looking nutty. And he was one of the fewer men who could almost go nutty, as he did in "Private Worlds," without looking like Napoleon.

The movies he made after the '40s - "Around the World in Eighty Days," "Is Paris Burning?," "Barefoot in the Park," "Casino Royale," "Fanny" and others - were a mixed bag, to say the least. Mr. Boyer seemed cut out for black and white, not color; and in his more recent pictures he often appeared like a studio snapshot of himself, pasted onto someone else's world. His world - which died long before last Saturday - consisted of cafes and violins. Even his suicide was of that world: and overodse of sleeping pills two days after the death of his wife of 44 years, and two days before his 79th brithday. Yet his brand of elegance wasn't fragile. It will survive him.