A HIGHLY experienced gunman named Horace Graydon was sentenced this week to 35 years in prison. He belongs to the small minority of offenders -- the most violent, and the most incorrigible -- who are responsible for a large majority of serious crimes. After four previous convictions for armed robbery, he was most recently found guilty of taking part in the holdup of an armored car and the ensuing sidewalk shoot- out. Mr. Graydon was the first defendant sentenced under the new District of Columbia law aimed at precisely this kind of dangerous and chronic offender.

This sentencing law owes much to Sen. Arlen Specter (R-Pa.), who argues that, by the time a person has committed repeated and serious felonies, there is little hope that he will be rehabilitated in prison. It follows, he believes, that these offenders must be incarcerated for long periods and that the federal government must help the states with these cases. In 1984 Congress enacted major crime-control legislation that contained his proposal authorizing federal prosecution of those charged with armed offenses who already had three prior convictions for robbery or burglary. For them the law established 15-year prison terms with no possibility of parole. There are now hundreds of cases similar to Mr. Graydon's in the federal courts here and around the country.

In some jurisdictions, local prosecutors and federal officials have moved slowly to cooperate in these cases. Paper-work problems have cropped up, and it is not always easy to track down the details of prior convictions quickly. There is another problem here: some dangerous, violent offenders are not covered by the new law because they have been convicted of felonies other than burglary or robbery. The statute should be broadened to include multiple offenders whose crimes are murder, aggravated assault, rape and felony drug offenses.

The law is tough: a mandatory 15-year prison sentence is severe. But it is meant to deter or to incapacitate only those whose violence is serious and continuing. This concerted effort to involve the federal government in prosecuting hard-core offenders is warranted, and it is beginning to pay off.