Senate Judiciary Committee Chairman Charles Grassley (R-IA) (C) talks with ranking member Sen. Dianne Feinstein after the committee voted out Supreme Court nominee Judge Brett Kavanaugh at the conclusion of a mark-up hearing in the Dirksen Senate Office Building on Capitol Hill September 28, 2018 in Washington.
Washington CNN  — 

Senate Judiciary Chairman Chuck Grassley on Friday suggested the lack of women on the committee was due to its heavy workload, but later said the committee is a lot of work for male senators as well.

When asked why there aren’t more female senators on the committee, the Iowa Republican said: “Well, it’s a lot of work. Don’t forget, compared to a lot of committee meetings, we have an executive every Thursday. … So it’s a lot of work. Maybe they don’t want to do it.”

When asked if he wants to see a woman added to the committee, Grassley said, “Well, we can’t do anything about that.”

“First of all, the people of the states elect women or men. So you’ve gotta go by that. And then you’ve gotta have a desire to serve. And my chief of staff of maybe three years tells me that we’ve tried to recruit women and we couldn’t get the job done.”

The makeup of the committee has been particularly relevant during Supreme Court nominee Brett Kavanugh’s nomination process. Kavanaugh was accused by Christine Blasey Ford of committing sexual and physical assault more than 30 years ago when they were at a party during their high school years.

Both appeared before the committee to testify about the allegation, but Republicans hired a special sex-crimes prosecutor to conduct the questioning on their behalf. The move was, in part, to curb concerns over the optics of an all-male Republican panel questioning Ford about her report of sexual assault.

Kavanaugh has continued to deny all the allegations that have been made against him.

Grassley was asked to clarify his comments about women on the committee, and he followed up by saying it’s difficult to recruit anyone to the panel regardless of gender.

“You ask me to get in the minds of everybody, whether they’re men or women,” Grassley told reporters. “We have a hard time getting men on the committee. Do you know we’ve got four people that are on the committee because the leader asked them to be there because they couldn’t fill the seats up? They said we need you. It’s just a lot of work whether you’re a man or a woman. It doesn’t matter.”

Grassley also said he expects more women to join the committee come January.

“On average, any woman in the United States Senate, whether they’re on Judiciary or any other committee, probably work harder than the average man,” he said.

Tensions on Capitol Hill have been high this week, particularly on Friday, as the Senate voted to advance Kavanaugh’s nomination.

On Friday afternoon, Sen. Susan Collins, R-Maine, announced she was a “yes” vote for Kavanaugh, essentially securing his confirmation when the final vote takes place, which is expected on Saturday.

CNN’s Jeremy Herb contributed to this report.