House Speaker Paul Ryan has had soul searching conversations about his future with friends, some of his close friends tell CNN. Those people close to Ryan told CNN they believe it is possible that he could leave Congress after the 2018 midterm elections, if he can achieve his goal of passing GOP backed overhaul of the US tax system. Some say his departure could possibly happen even sooner. Some friends indicate that Ryan may be suffering from a bout of “Trump-haustion,” but others believe there is serious contemplation of leaving Congress in 2018. According to people close to Ryan, the idea that he would resign immediately after tax reform, because it’s all he’s ever wanted, is not accurate. Ryan particularly dislikes the toll the job takes on his family, according to multiple sources. Politico published a report Thursday attributed to unidentified sources that Ryan, a Wisconsin Republican, has made clear to those around him that he “would like to serve through Election Day 2018 and retire ahead of the next Congress.” Ryan vehemently denied the report, telling reporters that he is here to stay. “I ain’t goin’ anywhere,” Ryan told reporters while exiting the Capitol when asked about the retirement rumors. Ryan’s office expanded on his denial of the Politico report. “This is pure speculation,” Ryan spokeswoman AshLee Strong said in a statement. “As the speaker himself said today, he’s not going anywhere any time soon.” President Donald Trump called Ryan following the release of the Politico story, a Ryan aide told CNN. “The speaker reassured the President ‘I’m not going anywhere!’ the aide said. House Majority Leader Kevin McCarthy completely dismissed the Politico report as “not true.” Ryan has been speaker of the House since October 2015.