But last month, the return to familiar ground for the 52-year-old was the first as a woman, competing in a charity race at British racecourse Warwick.
As Vince Smith, she boasted 250 races as a jockey before hanging up the whip but has made a return to the saddle as she transitions to becoming a woman making her the first jockey to ride as both a male and female.
"The reaction of the racing community has been fabulous," said Smith, who finished second on her return on board the gelding Roy Rocket.
"I couldn't have hoped for a better reaction. I've still got a long way to go, there will be rocky times and it can't all by rosy."
Smith always knew she was different from the age of four or five but, as she puts it, "I couldn't quite put my finger on it."
The idea she wanted to be a girl gnawed away at her but, "I think it took longer to accept it as I was attracted to females."
Smith had a series of long-term relationships with women, relationships that "put it to the back of my mind, suppressed it really."
But increasingly, Smith would be attracted by women as much by their looks as enviously eying their clothes, jewelry and make-up.
'If I don't do this, I'll die unhappy'
Yet in the often macho world of horse racing, she felt too nervous to make her secret public until last year.
Partially inspired by Caitlyn Jenner and Kellie Maloney
, who both transitioned from the "male-dominated sports worlds" of the decathlon and boxing respectively, helped instil a bit more confidence in Smith.
"I'd also lost a couple of friends along the way and I thought to myself 'if I don't do this, I'll die unhappy,'" she added. "There's still a long way to go but I'm as happy as hell."
The first and hardest conversation was with her parents.
"That's the main backbone for help and acceptance," she said. "If you haven't got your family behind you -- and lots of my transgender friends don't -- it's a big kick in the teeth.
"I knew my mother would be fine really and she was just upset I'd kept it to myself for so long. My dad was great too, telling my mother 'you'd always wanted a daughter!'
"If your family accept it, it makes it easier for other people too and it's sort of snowballed from there."
His former jockey friends, among them Graham Bradley, Richard Dunwoody and Steve Smith Eccles, have all been accepting.
"Those guys are hard as nails but they don't have an issue at all as they're comfortable in their bodies," she said. "They'll walk down the street with me and call me to go for a drink."
There have been some negative reactions from what she calls fringe friends, who cannot even look at or talk to her since going public. But, as she puts it "two or three out of 1,500 people isn't bad."
She added: "People that are transitioning are not sick, they're just normal people. Once people speak to me, they realize I'm just the same person."
'I guess that makes me a lesbian'
The name Victoria was an easy pick, one she'd potentially earmarked for a daughter, and she has been taking hormone medication for the past year.
Having initially been reticent about it, she is committed to full gender reassignment surgery in the years to come although her transitioning is still in its early days.
"It used to be confusing as why did I feel female but was in a male body and fancied females," she said. "It's a little bit difficult to understand myself so I can understand it's tricky for others. Now, I guess that makes me a lesbian!"
Smith believes that her former employment as a jump jockey in part prepared her for the potential knocks she faces.
"Being a jump jockey is hard with those broken bones and bumps," she said. "But that's helped toughen me up. I know other people transitioning have found it harder so I feel lucky."
Having been inspired by the likes of Jenner and Maloney, Smith is hopeful of having a similar impact of others reading about what has been a positive experience for her. As she puts it, "if I help just one person then great."
It still takes time for people to get used to Victoria. Her parents still call her Vince although her mother has taken to referring to her as Victoria on text messages.
"People do still call me Vince and that's understandable," she said. "It's a big change and it takes some getting used to."