THERE COMES a time when a veteran performer must decide what direction the rest of his or her career should take. When acts first begin, they can experiment to find new forms of expression and new audiences, but once they're "established," once the public perception has become fixed, it's very difficult to change styles. So it's about this time that the performer says "this is it" and then expands on the image or cruises along on the formula that's worked from the beginning.

James Taylor is a cruiser. His new album, "Flag," is so lackluster that it may make even the most diehard Taylor fan admit that his hero is a bore. What Taylor has done is recycle every cliche he's used for years and try to ignite them with some fluid production and fancy instrumentation; but the rock riffs he uses sound forced, and only during "Up on the Roof" does his heart sound in it.

Taylor, of course, is not without his charms. His most obvious is his voice, which never ceases to be pleasing and soothing. Also, up until "Flag" you usually got the idea that Taylor really enjoyed what he was doing. Here, it sounds like he'd just as soon be off somewhere else.

Taylor's voice can't compensate for most of the material on "Flag." The same man who evoked such warm images on "Sweet Baby James" now seems to write songs that are so directly spoken that they leave no room for the listener to add his own dimension. They come at you now like bricks. In "Johnnie Comes Back," he sings "Where'd she get that tear in her eye / whoever said she could leave me / where'd she learn to say goodbye / she's been watching too much TV." In "I Will Not Lie For You" the title is a good percentage of the entire song.

The only tune that fully avoids this trap is "Up on the Roof" - which was written by Gerry Goffin and Carole King, not Taylor.

The most telling factor of "Flag's" lethargy is the apparent lack of musical energy that went into each song. "Company Man" is one of those "isn't rock'n'roll a dirty business" songs. Not only has that type of song been done before, but Taylor himself did a better one than this when he sang "Money Machine" on the "In the Pocket" album. "Day Tripper," which some people have referred to as disco, not only isn't disco but it has the exact same musical arrangement as "Suite in 20G" from "Sweet Baby James." "Brother Trucker" is a CB/trucker song, no better or worse than the thousands of other CB/trucker songs. "Rainy Day Man" was first recorded on Taylor's debut album on Apple, and he's been doing "Up on the Roof" in concert for years. "Chanson Francaise" is totally in French, so it sounds pretty but conveys no message to the unilingual. (The lyric sheet does not have an English translation.)

The only really interesting track on "Flag" is "Millworker," a melancholy lament in the best Taylor tradition. Inexplicably, though, Taylor has written it and sings it from the woman's perspective, and no matter how hard you try it's impossible to project Taylor's richly toned pleas onto a bitter woman. Like most of "Flag," "Millworker" seems ill-conceived.

UNLIKE TAYLOR, Peter Allen has decided to build on what he's got and see how far it will go. Allen's "I Could Have Been A Sailor" album opens with the lines "I'm a happy man / I've made my choice in life," and indeed he has.

When Peter Allen hooked up with Dee Anthony's management organization, Anthony vowed to do the same thing with Allen that he'd done with Peter Frampton. By carefully orchestrating every move, Anthony was determined to make Allen a superstar. The only problem was, in what area? It was obvious that Allen was not drawing a rock'n'roll audience to see his hellzapoppin' cabaret shows, but the prevailing attitude seemed to be that it was only a matter of time before he broke through. Allen marched back and forth across the country, sometimes appearing in the same locale three times in six months.

Some of his songs got play. "I Go to Rio" was a minor hit and later was covered by Pablo Cruise.

"Don't Cry Out Loud" became Melissa Manchester's biggest hit in years, and he got a Grammy for "I Honestly Love You." Still, none of this was exactly what Anthony had in mind.

Like Frampton, Allen released several albums that sold moderately well and then a two-record live set that stands as his best work. Ironically, though, "It Is Time For Peter Allen" proved once and for all that he was not going to attract a rock crowd. The album's appeal lay in the stage patter and humor mixed with romantic ballads and zippy dance tunes. There wasn't a rock song on it.

So now comes "I Could Have Been a Sailor," more subdued, less hindered by an obligation to an audience that never really existed for him, an album far more suitable as a prelude to the one-man Broadway show he's mounting. For all the people that ask when Peter Allen is going to become a superstar, there's now a rhetorical question for an answer: How many non-superstars get their own one-man shows on Broadway for 29 performances at a $15 top?

Whether Allen's notoriously large New York following will be enough to make this venture a financial success is questionable. However, the fact that his "Up In One" could come about at all is a tribute to his realization that his act belongs there and not in the rock arenas of America.

"I Could Have Been a Sailor" is not Allen's best work, but it reflects his change to the middle of the road. His own version of "Don't Cry Out Loud" as well as other songs show that Allen - having made his career decision - plans to stick to it for better or for worse. CAPTION: Picture, JAMES TAYLOR: JUST CRUISING ALONG.