paul ryan GOP presser
CNN
paul ryan GOP presser
Now playing
01:52
Ryan: My kids will know me as a weekend dad
paul ryan 07262018
CNN
paul ryan 07262018
Now playing
00:47
Ryan does not support impeaching Rosenstein
ABC
Now playing
00:54
Paul Ryan in 2017: Not going anywhere
House GOP Leadership Presser with Speaker Ryan
CNN
House GOP Leadership Presser with Speaker Ryan
Now playing
00:33
Ryan: I've been assured Mueller won't be fired
Location HC-8, the Capitol    Speaker Ryan, GOP leaders hold post-Conference meeting media availability.    Sulen's chase crew covers this event, Sunlen will attend
CNN
Location HC-8, the Capitol Speaker Ryan, GOP leaders hold post-Conference meeting media availability. Sulen's chase crew covers this event, Sunlen will attend
Now playing
00:38
Ryan urges Trump to be 'surgical' on tariffs
CNN
Now playing
00:57
Paul Ryan sees gaps in background check system
NS Slug: FOX: REP JIM JORDAN-PAUL RYAN HAS GOT PROBLEMS  Synopsis: Rep. Jordan, a key Freedom Caucus member, says Speaker Ryan may face "problems"  Keywords: CONGRESS SPENDING SPEAKER RYAN JORDAN BUDGET
FOX NEWS
NS Slug: FOX: REP JIM JORDAN-PAUL RYAN HAS GOT PROBLEMS Synopsis: Rep. Jordan, a key Freedom Caucus member, says Speaker Ryan may face "problems" Keywords: CONGRESS SPENDING SPEAKER RYAN JORDAN BUDGET
Now playing
00:41
GOP Congressman: Paul Ryan has problems
Now playing
01:00
Paul Ryan jokes about Trump, Clinton
Members of Leadership will preview President Trump's first State of the Union address and discuss House Republicans' continued focus on the American people's priorities, including economic growth thanks to tax reform, national security, and protecting victims of sexual abuse.       PARTICIPANTS:   Speaker Paul Ryan (R-WI)   Majority Leader Kevin McCarthy (R-CA)   Majority Whip Steve Scalise (R-LA)   Conference Chair Cathy McMorris Rodgers (R-WA)
CNN
Members of Leadership will preview President Trump's first State of the Union address and discuss House Republicans' continued focus on the American people's priorities, including economic growth thanks to tax reform, national security, and protecting victims of sexual abuse. PARTICIPANTS: Speaker Paul Ryan (R-WI) Majority Leader Kevin McCarthy (R-CA) Majority Whip Steve Scalise (R-LA) Conference Chair Cathy McMorris Rodgers (R-WA)
Now playing
02:07
Ryan: There may have been malfeasance at FBI
paul ryan town hall in 90 seconds origwx allee_00000413.jpg
paul ryan town hall in 90 seconds origwx allee_00000413.jpg
Now playing
01:30
Paul Ryan's CNN town hall in 90 seconds
paul ryan presser 01182018
CNN
paul ryan presser 01182018
Now playing
01:34
Paul Ryan: Dems only thing standing in the way
Paul Ryan wispolitics
CNN
Paul Ryan wispolitics
Now playing
01:23
Ryan: Trump's comments unfortunate, unhelpful
paul ryan mattingly 01
CNN
paul ryan mattingly 01
Now playing
01:47
Paul Ryan: Russia will not distract from taxes
Speaker of the House Paul Ryan (R-WI) holds a news conference about the American Health Care Act outside his office in the U.S. Capitol March 7, 2017 in Washington, DC.
Chip Somodevilla/Getty Images
Speaker of the House Paul Ryan (R-WI) holds a news conference about the American Health Care Act outside his office in the U.S. Capitol March 7, 2017 in Washington, DC.
Now playing
01:59
Breitbart releases Paul Ryan audio on Trump
WASHINGTON, DC - JANUARY 12:  Speaker of the House Paul Ryan (R-WI) talks with reporters during his weekly news conference in the Capitol Visitors Center at the U.S. Capitol January 12, 2017 in Washington, DC. Ryan said that Congressional Republicans are on a "rescue" mission to repeal and replace the Affordable Care Act and that he and President-elect Donald Trump are in perfect sync with the process or replacing Obamacare.  (Photo by Chip Somodevilla/Getty Images)
Chip Somodevilla/Getty Images North America/Getty Images
WASHINGTON, DC - JANUARY 12: Speaker of the House Paul Ryan (R-WI) talks with reporters during his weekly news conference in the Capitol Visitors Center at the U.S. Capitol January 12, 2017 in Washington, DC. Ryan said that Congressional Republicans are on a "rescue" mission to repeal and replace the Affordable Care Act and that he and President-elect Donald Trump are in perfect sync with the process or replacing Obamacare. (Photo by Chip Somodevilla/Getty Images)
Now playing
02:14
Ryan is not giving up on entitlement reform
(CNN) —  

House Speaker Paul Ryan described the message delivered to him by an octogenarian businessman a few months ago as akin to getting hit in the head “with a two-by-four, more or less.”

He was in Springfield, Missouri, raising money for two House Republican colleagues when, as he recounts it, a “really successful guy” approached him to talk about family.

“Let me give you a piece of advice that I wish I’d listened to,” the man told him, according to Ryan. “They used to say it’s not the quantity of time you spend with your kids, it’s the quality of time. That’s a bunch of bull. It’s both. And don’t ever forget that. “

RELATED: The Paul Ryan legacy that matters the most

The idea that a public official would leave his position “to spend more time with family” and have that actually be the reason is an anomaly in Washington, one that draws knowing chuckles that imply a deeper, more dramatic rationale. But in a 30-minute interview with a small group of reporters in his Capitol Hill office, Ryan insisted that family – to be specific, his three kids in or entering their teenage years – really was the reason for the looming end of his 20-year congressional career, a decision made Sunday night at a family dinner in his hometown of Janesville, Wisconsin, and announced Wednesday.

The headwinds facing the Republican Party that have led most on Capitol Hill – Republicans included – to consider a future in the House minority the only likely end game? He’s aware of them, but says that’s not the reason he decided not to run again.

Frustration with President Donald Trump? He could do with fewer tweets, but their relationship remains in a good place.

Concern that he’d lose his race to a well-funded Democrat? Ryan is confident his seat will remain in Republican hands with or without him.

In truth, all of the above factored into the decision-making process (save for the last one – Ryan and his team remained confident he was fine back home throughout). But the clincher that would put an end, for now at least, to the political career of a man who had become one of the faces of the Republican Party, was, in fact, his family.

The top House Republican appeared at ease as he reclined in a chair in his second-floor office overlooking the National Mall, hands clasped behind his head, reflecting on a career spent almost entirely pushing for his policy visions in federal government.

The successes in his mind were clear, most notably the passage of the 2017 tax law and the significant increase in military spending locked in by the recent $1.3 trillion federal spending bill. While he fell short in his goal of overhauling US entitlement programs, his ability to, in his words, “normalize” the discussion for significant changes – something that in his earlier years would be considered radical and is now the mainstream of House Republican orthodoxy – wasn’t a small thing, either.

His biggest regret? The failure to repeal and replace Obamacare.

Does he have ideas about who should replace him atop the House Republican conference? Yes. But that will have to wait until after the election, he said.

His biggest concern? The rise of identity politics.

“It used to be a Saul Alinsky thing. The left was really good at it. Now the right does it,” Ryan said, pointing to technology and social media as an accelerant. “That makes it hard just to have political goodwill in this country, because of all this polarization.”

Asked how that concern squared with a President who has used a strategy that relies heavily on those same identity politics more effectively than anyone in recent political history – a President who Ryan has publicly stood by throughout his time in the White House, the speaker exhaled audibly.

“Look, he won the election. He’s the President of the United States. We’re making really good progress on a lot of signature issues and we’re making a positive difference in people’s lives. I think if you can deny the oxygen of identity politics the best way to do that is to have a faster growing economy, more upward mobility, higher wages, getting people from poverty into the workforce. And those are some of the best tools to fight the seed corn of identity politics, the oxygen that feeds identity politics.”

As far as Ryan’s views on Trump – from his initial refusal to back the candidate, which turned to hesitant support, which turned into at times strong public defense of the President – they have been at the center of one of the more speculated about relationships on Capitol Hill.

In truth, according not just to Ryan but also to aides on both sides of Pennsylvania Avenue, the relationship between the two men has been solid. They talk constantly by phone – often candid conversations, the speaker says.

One of the sharpest criticisms of Ryan – not just from Democrats, but also from some Republicans – is his perceived unwillingness to publicly attack Trump, even when he clearly disagrees on an issue or action, or, as is often the case, a tweet.

“It works better to have private conversations than to have public disputes. It’s more effective,” Ryan says. “He appreciates that. He appreciates that when we have disputes or differences of opinion that we air them privately, and what I’ve learned is he learns. And he responds and he respects.”

Asked if he could candidly advise against the idea of, say, firing Deputy Attorney General Rod Rosenstein or special counsel Robert Mueller – something Trump has appeared to be teetering toward in recent days, Ryan starts to speak and then stops.