Richard Williams has played a hugely influential role in the success of his daughters, U.S. tennis stars Serena and Venus.

Photos: The man who made Serena and Venus 'tough'

PHOTO: LEON NEAL/AFP/GettyImages)

Richard Williams has played a hugely influential role in the success of his daughters, U.S. tennis stars Serena and Venus.

Venus and Serena have won 28 grand slam singles titles between them and shared 13 grand slam doubles titles, both achieving the world No. 1 ranking.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Paul Gilham/Getty Images Europe/Getty Images

Venus and Serena have won 28 grand slam singles titles between them and shared 13 grand slam doubles titles, both achieving the world No. 1 ranking.

From a young age, Venus and Serena were given tennis lessons in the city of Compton in Los Angeles County by their father, who was determined to produce two stars.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Ken Levine/Getty Images North America/Getty Images

From a young age, Venus and Serena were given tennis lessons in the city of Compton in Los Angeles County by their father, who was determined to produce two stars.

Richard is said to have been watching TV when he saw a Romanian tennis player win $40,000, before deciding to learn a sport he was unfamiliar with and teach his daughters to play.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Paul Harris/Getty Images North America/Getty Images

Richard is said to have been watching TV when he saw a Romanian tennis player win $40,000, before deciding to learn a sport he was unfamiliar with and teach his daughters to play.

Richard, seen here with Venus, devised a 78-page training regime for his daughters.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Ken Levine/Getty Images North America/Getty Images

Richard, seen here with Venus, devised a 78-page training regime for his daughters.

He made them keep journals about their tennis goals and how best to achieve them.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Ken Levine/Getty Images North America/Getty Images

He made them keep journals about their tennis goals and how best to achieve them.

Richard, seen here with Venus during a practice session, instilled the motto purportedly first said by Benjamin Franklin: "If you fail to plan, you plan to fail."

Photos: The man who made Serena and Venus 'tough'

PHOTO: Ken Levine/Getty Images North America/Getty Images

Richard, seen here with Venus during a practice session, instilled the motto purportedly first said by Benjamin Franklin: "If you fail to plan, you plan to fail."

Photos: The man who made Serena and Venus 'tough'

PHOTO: Ken Levine/Getty Images Europe/Getty Images

In his autobiography, ''Black and White: The Way I See It,'' Richard describes how he brought ''busloads of kids from the local schools into Compton to surround the courts while Venus (pictured) and Serena practiced" to toughen them up. "I had the kids call them every curse word in the English language," he wrote.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Jamie Squire/Getty Images North America/Getty Images

Richard's work came to fruition in 1999, when Serena first won the U.S. Open, before Venus landed the following year's Wimbledon and U.S. Open titles. Here, Serena celebrates her maiden grand slam triumph.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Clive Brunskill/Getty Images Europe/Getty Images

Upon Venus' 2000 Wimbledon victory, Richard celebrated by shouting "Straight Outta Compton" (in reference to the song by controversial rap band N.W.A.) before then jumping up onto the commentary booth and embracing his daughter.

As Venus and Serena began to earn more success, Richard took a step back from their careers. Patrick Mouratoglou now coaches Serena, while Venus works with David Witt.

Photos: The man who made Serena and Venus 'tough'

PHOTO: TIMOTHY A. CLARY/AFP/AFP/Getty Images

As Venus and Serena began to earn more success, Richard took a step back from their careers. Patrick Mouratoglou now coaches Serena, while Venus works with David Witt.

In 2002, Richard divorced from Oracene Price -- who also helped coach the young sisters -- having married in 1980. They are still often seated together during their daughters

Photos: The man who made Serena and Venus 'tough'

PHOTO: Al Bello/Getty Images North America/Getty Images

In 2002, Richard divorced from Oracene Price -- who also helped coach the young sisters -- having married in 1980. They are still often seated together during their daughters' matches.

In 2010, Richard married Lakeisha Graham, having met the previous year. Their son Dylan was born in 2012. Graham is 37 years younger than her husband, and one year older than Venus.

Photos: The man who made Serena and Venus 'tough'

PHOTO: Clive Brunskill/Getty Images Europe/Getty Images

In 2010, Richard married Lakeisha Graham, having met the previous year. Their son Dylan was born in 2012. Graham is 37 years younger than her husband, and one year older than Venus.

Story highlights

Serena Williams' father reflects on Blake arrest

He says U.S. race relations "may have got worse"

Success of his daughters "brings tears to my eyes"

(CNN) —

Was it racism or a case of mistaken identity?

When former tennis pro James Blake, who is biracial, was tackled to the ground by police in New York this week, it came as no shock to the father of the game’s first black female No. 1 player.

“We’ve had so many things happen to people of my kind and my color, it’s not surprising, not at all,” the 73-year-old Richard Williams told CNN.

That view is not shared by NYPD Commissioner Bill Bratton, who insists the Blake incident was a case of mistaken identity, and a case of excessive force.

“I don’t believe that race was a factor,” Bratton said. “This rush to put a race tag on it, I’m sorry, that’s not involved in this at all.”

Williams’ comments mirror those of his daughter Serena, who told CNN ahead of her bid to make tennis history at this month’s U.S. Open that “a lot of people in America and outside America are frustrated and concerned and really scared” about recent high-profile incidents involving police.

For Richard Williams, who moved his young family from Michigan to the city of Compton in the early 1980s, the problem of racism in society has been a constant factor as his daughters Serena and Venus have grown into tennis champions.

“I think it hasn’t changed that much at all (since I was a kid) – matter of fact it may have gotten worse, I don’t know … but I don’t think it’s changed that much,” he told CNN.

Williams says his daughters’ upbringing in the tough urban environment of Compton, where he often brought in local kids to shout insults at the girls while they practiced, helped them on the pro circuit.

“In order to be successful, you must prepare for the unexpected,” he said from his home in West Palm Beach, Florida, where the family moved to continue the sisters’ tennis education.

The Williams family famously took a stand against racism when Serena was abused during the final at Indian Wells in 2001. Neither sister again played at the Californian tournament until this year, when Serena made her return.

“The whole crowd turned against her,” he said of the Indian Wells incident, which was fomented after Venus pulled out of her semifinal against Serena at late notice.

“All she had to do was remember the training that she had been through.”

A successful businessman, Williams didn’t need to move the family from Saginaw but he says he felt Compton “would be the best place for them.”

“The ghetto will make you rough, will make you tough and will make you strong,” he said.

PHOTO: JEWEL SAMAD/AFP/AFP/Getty Images

Serena Williams of the U.S. reacts as she takes on her sister Venus during their 2015 U.S. Open women's singles quarterfinals match. Serena won in three sets to move on to the semifinals.

Despite having played such a key role in molding the careers of his two daughters, Williams admitted he could not watch this week when Serena beat Venus to advance to the semifinals at Flushing Meadows.

“I’ve never watched Venus play with Serena. It’s like watching your kids fight. They’re just too close. I can never watch, I never have,” said Williams, who divorced from their mother Oracene Price in 2002 and remarried in late 2010.

While Serena is on the brink of tennis history, hoping to become the first player since Steffi Graf in 1988 to win all four grand slams in a calendar year, Williams paid tribute to older sister Venus – who in 2002 became the first black woman of the professional Open Era to be ranked No. 1.

“Venus changed tennis altogether, period. Venus was not accepted when she first came onto the scene. Not at all. She changed this sport. Venus and Serena made things so different, they raised the bar.”

Venus, 35, won the last of her seven grand slam titles in 2008 but has since had to come to terms with having Sjogren’s Syndrome – an autoimmune disease which has had debilitating effects on her energy levels.

Serena, 33, has looked unbeatable after her own serious health scare, winning eight grand slam titles since a freak foot injury in 2010 led to a near-fatal blood clot in her lungs.

However, she remains one short of Graf’s modern record of 22 slams and three behind Margaret Court’s all-time mark after suffering a surprise semifinal defeat against Italy’s Roberta Vinci on Friday at the U.S. Open.

372178 11: FILE PHOTO: Sisters Serena, left, and Venus Williams shake hands after a game 1991 in Compton, CA. Serena and Venus Williams will be playing against each other for the first time July 6, 2000 in the tennis semifinals at Wimbledon. (Photo by Paul Harris/Online USA)

Before that match, her father had said the achievement of a possible calendar grand slam would be “unbelievable.”

“It’s hard for me to talk about. It brings tears to your eyes to think where they came from,” he said.

“I couldn’t live the way I live if it wasn’t for them. Serena often likes to let me know that if it wasn’t for me, she wouldn’t be where she is. That’s enough to make you cry.

“I try to get them to believe that they pushed themselves, not me. Too many parents push their kids the wrong way, and that could’ve been me at one time.

“But I hope the two of them, whenever they retire, that they can accept what they have.”

Both Venus and Serena also have extensive off-court interests, and this drive to succeed in all areas can be attributed to their father, says CNN tennis correspondent Ravi Ubha.

“Everything they have become has been a result of Richard putting them into tennis, and not only in tennis,” Ubha said.

“Venus just recently graduated from university. She’s long had a very successful interior design company; Serena has her own fashion line. It really stems from what Richard did.”

Can anyone stop Serena? Tell us on CNN Sport’s Facebook page