It was last November when Antonio Ayala Jr. awoke in his hotel room in Atlantic City and saw his father, known as "Big Tony," pacing the floor. There was a look of concern on his face about that night's bout against Carlos Herrera, the WBA's No. 1 junior middlweight contender.

Through half-shut eyes, the 20-year-old son watched his old man move restlessly about the room a while before letting him know he was awake. "Don't worry, Fat Man," he said. "This guy (Herrera) is all mine."

That night, Tony Jr.--whose ring nickname is "Torito," Spanish for "baby bull"--knocked out Herrera at 2:43 of round three to become the mandatory challenger for the WBA junior middleweight championship. A Davey Moore-Ayala title bout was to have been held this spring but, instead, Ayala, the best fighter of Big Tony's four boxing sons, spent the spring in the Passaic County Jail awaiting sentencing on a burglary and rape conviction.

Ayala was arrested on New Year's Day after a West Paterson, N.J., woman told police that he had broken into her apartment and sexually assaulted her. In court proceedings, Ayala claimed he had been invited into the woman's apartment.

Ayala's account, according to his attorney, William DeMarco, was undermined by evidence introduced in court, over the lawyer's objection, of the fighter's arrest last August in his native San Antonio. In that incident, Ayala was found in a stranger's home not far from the $90,000 residence into which he had moved only weeks before.

At the time, Ayala's Texas attorney contended that his client had merely wandered into the wrong house while drunk. The issue became moot when the complainant chose not to press charges against Ayala.

According to the evidence in the New Jersey trial, at the time of his arrest in Texas, Ayala was carrying an ID tag belonging to and bearing the name of the young woman, and he had entered the house through a window. On April 13, Ayala was convicted of burglary and rape. His sentencing is scheduled for June 21.

When he was 15, Ayala assaulted a young woman--a stranger to him--in the ladies room of a drive-in movie theater in San Antonio. The woman suffered internal injuries. Ayala was tried as an adult and received a 10-year prison sentence, but he didn't do any time. In connection with the sentencing, the victim asked that he be granted probation. He ended up on 10 years probation.

The tidy resolution of the matter follows a pattern in Ayala's life: from speeding tickets that somehow got dismissed when he was driving at the precocious age of 12 to the obviously more troubling incidents later. For Torito, there was alway an "out" that seemed to go with being a fighter and the best little boy in Big Tony's household stable of boxers.

Passaic County Jail, a balmy day in June.

In the office of a deputy warden, Ayala sits in olive-green fatigues and rubber sandals, staring through the cloud of cigarette smoke he blows. Ayala's eyes are puffy from sleep. His expression is wistful as he speaks.

"I've cried a lotta nights thinking about what could have been. You know, after the jury's verdict, I felt I didn't want to go on. I lost a lotta weight. I told Bill DeMarco to forget about the appeals. But that passed. 'Cause I decided I'm not a bad person. I don't give a damn what they say about me. I know I'm not like a lot of these other guys (inmates) I see in here. They joke about the crime they've done. They're proud to be here. They can't wait to be out on the street so they can boast: 'Yeah, man. I did time.'

"Me, I made mistakes. And I'm man enough and intelligent enough to correct them myself. Meantime, I spend a lot of hours writing. About my life. Not to sell. Just to resolve what went wrong. From days going way back . . . "

Boxing is an Ayala family obsession, with Big Tony intending the sport to be his son's ticket out of the San Antonio barrio, where the Ayalas and many other poor Mexican-Americans lived. "Browntown," it was derisively called. And it was no place for the meek. "In our neighborhood," Ayala Sr., said, "either you roared like a tiger or you'd be a pet. Some kids were abused. I didn't believe in turn-the-other-cheek, and I made damn sure my sons didn't, either."

Like the American pit bull terriers the Ayalas bred, Big Tony's four boys were raised for battle. "Dad," said Ayala last winter, "might squeeze my thigh or Sammy's or Mike's and say, 'will power.' And see how much pain we could take."

For Torito, the discipline was part of a relentlessly thorough boxing regimen. The Ayala boys (three are professional boxers now and one is an amateur) were educated in the art of slipping punches at an age when most youngsters are learning hickory-dickory-dock. By the time they were in grade school, Torito's brothers Mike and Sammy were in pee-wee competition throughout Texas, and their father had built a backyard gym, paying quarters to neighorhood youngsters to spar with his boys so he could gauge their progress.

At age 5, the younger Torito--who, until then, had declined to matriculate in Big Tony's fight nursery--had a change of heart. Tagging along to Robstown, Tex., where Mike and Sammy were to compete, he demanded a chance to box.

"I told him, 'But you haven't trained, Torito,' " his father recalled. "He wasn't impressed. Tells me, like kids do, 'I wanna box.' What could I say? I let him box. He fought barefoot that night--all he had were cowboy boots--and got whipped. Cried afterward, too, because he'd lost."

Three days a week Ayala sees a psychiatrist. "It helped me understand myself, the fears I have, and had," he says. "I was always scared that I was not good enough. Not good enough as a person, or a fighter."

The Ayala War Wagon, as their vehicles through the years were known, transported the boys to bouts throughout Texas, Mexico and beyond. At some sites, fans would throw money into the ring to express their pleasure, as much as $20 or $30 for the pee-wee fighters to split.

When not in a competition, Torito and his brothers trained regularly under Big Tony, whose moods fluctuated according to how his boys perfomed. As Caesar Cano, a San Antonio fight trainer, recalled, "I saw the father as a man who loved those kids. His love was so great that he wanted them to be perfect. One day he'd be so hard, the next he'd kiss them on the forehead and neck."

Tony's skills were so far beyond his years that when then-WBA welterweight champion Pipino Cuevas came to San Antonio to train in 1977, Ayala, then 14, was convinced he could hold his own with Cuevas. And he did.

San Antonio fight promoter Tony Padilla was there the afternoon Cuevas and young Tony went at it. He remembers Lupe Sanchez, Cuevas' manager, saying to his fighter afterward, "Aren't you ashamed, a 14-year-old boy did that to you?"

Padilla continued, "And Pipino was muttering, 'increible, increible'--which is 'incredible' in English."

Though Torito went on to win two Junior Olympic titles and later a national Golden Gloves title, he did not emerge without psychological scars. It was a tough life, with tough companions.

Removed from the streets of San Antonio or its gym, Ayala was less sure of himself. "I always wanted people to like me," he said, "but I lacked self-confidence in the way I carried myself, or in the way I spoke." Even conversing with his mother Pauline or later with his common-law wife Lisa, he often cut short his thoughts, as if what he had to say was not worth pursuing.

It was only through boxing he could express himself and keep the approval of Big Tony. In the Ayala household, the father was boss, a strong figure whose imminent arrival would set his wife and sons scrambling to tidy up, and whose word had the force of law.

The father's authority extended beyond boxing, into his sons' lives, and in time was resented by Mike, leading to a rift between father and oldest son. (In June 1979, Mike Ayala fought for the featherweight title against champion Danny Lopez and was knocked out in the 15th round in what The Ring Magazine rated as "the most exciting fight of the year." Mike is currently a high-ranking super-bantamweight and is also on 10 years probation for an incident in San Antonio in which he shot a man.)

When Mike switched managers, leaving his father for Dennis Rappaport, comanager of heavyweight Gerry Cooney, it was viewed by the old man as a betrayal, and the ensuing events--a reconciliation and another falling out--are spoken of in charged language, with Ayala Sr., swearing he'd "sell my soul to the devil" before resuming civilized dialogue with his son. For his part, Mike insists that he only needed some slack in the relationship.

By contrast, Torito always marched in step with his father. In an interview only weeks before his last arrest, he added this postscript in a final session with a reporter: "Make sure to put in, 'I love you, dad,' in your story." Ayala then watched as the words were scribbled across a notebook page.

Tony never surmounted the limits of the macho response to life. One afternoon in a West Paterson gym he bumbled through a training session and became so annoyed with himself that he finally stood with his hands at his sides and let his sparring partner punch him at will. Comanager Lou Duva was startled enough to phone Big Tony, who said, "Oh yeah, he does that all the time."

"There was," says Duva's daughter-in-law Kathy, who is a boxing publicist, "a sense of--'If someone else won't punish me, then I will.' "

"In my life," Ayala says now, "I was always pushed with the boxing. No questions asked. Don't get me wrong. I'm not pointing a finger. Like, as far as my dad . . . nobody could ever take the place of my dad. It should have come out of myself to change things. I was the one hurting. I should have spoken up. Because, you know, a lotta time I hated boxing."