On Thursday, a sturdy horse clip-clopped down C Street NW in Washington, carrying new Interior Secretary Ryan Zinke to work for his first day on the job. Zinke, flanked by U.S. Park Police officers, wore a cowboy hat and boots. Tonto, his forelock tossing in the breeze, was a steady mount.

The image was captivating — in part because it was so incongruous with the 21st century urban surroundings. But that commute once was routine. For Theodore Roosevelt, the horse-besotted president whom Zinke has said he admires, riding down Washington streets was an everyday affair.

The 26th president once canceled a Cabinet meeting on a warm spring day so that he and his horse could be photographed jumping. Most days, he galloped through Rock Creek Park, mounted on Bleistein or Renown, sometimes rocketing over barriers he’d erected: a stone wall, a bank with a ditch, a 5-foot-7-inch hurdle. Roosevelt didn’t change clothes for his daily ride. He just walked out of the office, put on spurs, a hat and riding gloves, and mounted up. He fretted over how Washington was expanding, because urban growth meant fewer places to ride.

And Roosevelt had lots of animals to ride: He kept the largest string of horses at the White House since Chester Arthur in the 1880s.

As is well known, Roosevelt had a lifelong love for horses, from days on his childhood pony to life with his Rough Riders. And in his stable in Washington, in addition to Bleistein and Renown, were his old polo pony Black Diamond, carriage horses Judge and General, two saddle horses and young son Archie’s pony, Algonquin.

For a while, the Roosevelts also had a horse named Wyoming, which had been a gift from the people of Cheyenne and Douglas, two cities in that state. Wyoming knew tricks, like dropping to his knees to salute the president. The stables for all of these were in a vine-covered brick building that stood just southwest of the White House, beside the Army and Navy Building and opposite the Corcoran Gallery. The Roosevelt horses ate hay grown on the grounds of the Washington Monument.

Roosevelt got Washington onto horseback. “Thanks To President Roosevelt, Riding Has Become Very Popular,” read one 1905 headline. His predecessor, William McKinley, had been more of a driver than a rider, a reporter wrote, but now that the president was galloping around town, so were dignitaries and diplomats. (Roosevelt actually took jumping lessons from the German ambassador.) Some liked to ride on a “speedway” that went through public land in southwest Washington; others cantered around the then-open country surrounding Fort Myer in Arlington.

Roosevelt was a great protector of horses, too. He rode Bleistein only at a walk when they were on paved surfaces to protect the horse’s hoofs. He worried about the dampness of the White House stable; too many of his horses had developed an equine respiratory ailment called heaves, and in 1904, Roosevelt asked Congress for $90,000 to refurbish it. Organizers planned a cowboy endurance race from Deadwood, S.D., to Omaha, but Roosevelt — who usually adored all things cowboy — disapproved, because he thought it would be too hard on the horses. The event was canceled.

Roosevelt also fought the docking of horses’ tails. Before 1900, he didn’t particularly care whether his horses had docked tails. Many did; it was the style. But by the time he arrived in Washington, Roosevelt was against the practice, and he vowed to let Bleistein’s shortish tail grow out. According to one report, he refused to keep two dock-tailed carriage horses that he’d used in Long Island and bought new, natural-tailed steeds in the District.

After that, fashion changed, and horses’ tails grew. Then-Sen. Jacob Gallinger (R-N.H.) introduced a bill forbidding docked tails in the District of Columbia, and it passed 200-8. The Society for the Prevention of Cruelty to Animals took up the issue on a nationwide scale.

In 1907, Roosevelt decided that his military officers needed to be in better shape, which he expressed in horseman’s terms: He wanted them able to ride 90 miles in three days. Many found this a ludicrous order. Who could ride that much?

So in January 1909, his last year in office, Roosevelt set out to prove a point. He stayed on horseback from 3:40 a.m. to 8:30 p.m., covering 98 miles of Virginia roads. He stopped in Warrenton for lunch and changed horses twice. In one day, he’d done what he’d asked the officers to do in three.

Coated in mud, his Rough Riders’s hat glazed with ice, Roosevelt rode right up to the White House’s portico. His horse stood, steaming. The president jumped off, handed the reins to a groom and headed inside. Before the door closed behind him, a reporter wrote, “the President flung back three hearty words — this with a grin that showed all his teeth and wrinkled his weather-beaten face into a hundred little furrows: ‘It was bully,’ he said.”

Eliza McGraw is a Washington writer. She is the author of “Here Comes Exterminator!” a book about the 1918 Kentucky Derby winner.

Read more: