TEN YEARS AGO this fall, a convention of Marylanders wisely voted to keep the office of sheriff out of the new constitution it was drafting for the state. During the next few months, most - if not all - of the state's 24 sheriffs campaigned against ratification of that constitution, and their opposition had much to do with its unwise rejection by the voters. We bring up that piece of history in connection with the indictment the other day of Prince George's County Sheriff Edward Ansell and three others and the accompanying grand jury report on conditions in Mr. Ansell's office. Leaving aside the criminal charges listed in the indictment, which are now before the court, what the grand jury had to say about goings-on in the sheriff's office becomes perhaps a bit more understandable when you consider that the office has no real reason for existence. As the constitutional convention suggested 10 years ago, the office of sheriff as it exists in Maryland is superfluous. And this may help explain the grand jury's findings of "untrained" deputies, "favoritism, cronyism and nepotism . . . pilfering, lying, cheating and misuse of [jail] trusties for personal gain."

Whether or not these statements are true, there is a certain irony in the fact that they are being seriously leveled at a sheriff who was first elected in 1970 as a "reform" candidate on a promise to clean up the mess his predecessor was leaving behind. You may even recall some of that mess, the most notorious part of it having been the formation of a 500-man-reserve deputy "posse" and the creation of a part-time "air force" of deputies who owned airplanes and helicopters. These were remarkable additions to an office that had precious little to do with law enforcement. As a result, Sheriff Ansell arrived in office with a mandate to straighten things out. To a large extent, he appeared to do so, but the grand jury now says the office has slipped back to its old ways.

That, of course, has something to do with the image and the reality of sheriffs. Sheriffs were once as important in Maryland as they were elsewhere in this country. But in urban counties like Prince George's, law enforcement has been taken over by police departments; sheriffs remain only to run the jail, guard the courthouse, transport prisoners and serve court papers. Still, the image lingers on and the idea of being a Matt Dillon - of Chester - on the streets of Dodge City dies hard.

Therein lies the problem of the sheriff's office - in Prince George's and elsewhere. The services it provides are neither demanding nor exciting. They call for little experience or expertise. (That will be particularly true in Prince George's come January when the county jail will be turned over to the new Corrections Department.) That, particularly when combined with the old Dodge City image of action - or waiting for action - is not conducive to an efficient operation. Instead, it seems to us to be conducive to precisely the conditions decribed by the grand jury.

Clearly, the tasks to be done by the Prince George's County sheriff's office after Jan. 1 would be done more appropriately by court-appointed employees than by the hired cronies of an elected official who can build a political organization, if he wants to, on the jobs he can parcel out. As a matter of logic, that Maryland convention was right 10 years ago. The sheriff's office in a place like Prince George's was - and is - an anachronism that ought to be abolished. But as a matter of practical politics, abolishing it may be something else again.