(CNN)In the days since House Speaker Paul Ryan announced his retirement, the jockeying for the speaker's gavel has centered on three names.
Republican women wonder when they'll get a female speaker of the House
They are all prominent and respected lawmakers, and so far they are all white men.
At this point, no women and no minorities are in contention for the post, with most of the debate focusing largely on how House Majority Leader Kevin McCarthy of California, Majority Whip Steve Scalise of Louisiana and House Freedom Caucus co-founder Jim Jordan of Ohio have positioned themselves for such a race.
Nearly two dozen interviews with current and former lawmakers and aides revealed a Republican conference where the belief is the lack of female or minority House leaders has more to do with sheer numbers and the top positions already being occupied, but there is still a desire to expand diversity in GOP ranks. The interviews also revealed that the subject of gender and racial diversity is still taboo to discuss even among women in Congress, where the reputation of a back-slapping boys club persists.
Ryan acknowledged last week in an interview with CBS that his party needed to work harder to be inclusive, and he committed that would continue to be a focus in his post-Congress career: "We need more minorities, more women in our party. And I've been focusing on that kind of recruitment."
Many women who spoke with CNN noted that any race for leadership is hypothetical at this point. Ryan has been clear that he won't leave until January, and House Republicans must maintain the majority for there to be a speaker's race at all. But Republican women, both current and former members, who spoke with CNN confided they'd like to see the focus now be on adding more women and minorities to their ranks so when another top job opens up, there are more members to fill it.
In 2013 the House Democratic Caucus made history when it crossed the threshold of having more female and minority members than white men, but there are just 22 Republican women serving in the House, compared with 61 Democrats. The total number of minority members also lags behind in the GOP conference.
The most obvious candidate to step up for Republicans would be Cathy McMorris Rodgers, the party's conference chairwoman, who is facing her own rigorous re-election for her Washington state congressional seat, but has long shown interest in climbing the leadership ranks.
"I hope that Cathy will step up and run for speaker," former GOP Rep. Cynthia Lummis of Wyoming told CNN in a phone interview. "I hope she'll do it. I hope she runs. I say that not because I am opposed to one of the fellas being the speaker, but I think Cathy has earned the right to be among the people who are seriously discussed."
While Lummis was candid at the time of the interview that she had not spoken with McMorris Rodgers yet or encouraged her personally, her point is that the Republican conference could use more women in leadership and McMorris Rodgers, who has spent the last several years connecting with the conference, deserves to at the very least be floated.
"Sure, there aren't as many women," said Debbie Walsh, the director of the Center for American Women and Politics at Rutgers University. "There aren't as many in leadership positions, but she is the chair of the Republican conference and you would like to think in at least a list of potential speakers."
At some level, GOP members argue it's a numbers game.
Unlike the Democratic caucus, where more than 30% -- almost a third -- of the House members are women, fewer than 10% are women in the GOP conference. That's not a large pool of applicants to draw from when it comes time to fill committee chairmanships and the leadership ranks.
Currently two women chair House committees, North Carolina's Rep. Virginia Foxx and Indiana's Rep. Susan Brooks. Tennessee's Rep. Diane Black led the House Budget Committee last year but has stepped down from her chairmanship to prepare for her run for governor in her home state.
"There aren't enough of us and we've tried to do some recruiting, but again, when you look at the farm team, whether it is women in the statehouse or city council or women mayors across the nation, many of them have to be talked into it, as opposed to just feeling like they have something to offer," said Rep.