Perez defeated Ellison in a 235-200 vote among the Democratic National Committee's 435 members who cast ballots Saturday.
It took two rounds of voting to select a winner after Perez found himself just one vote shy in the first ballot. The five other candidates, who had amassed 13.5 total votes, all exited the race before the second round began.
In a victory speech, Perez vowed that Democrats would wage an all-out battle to deny Trump a second term.
"We are at a 'where were you?' moment in American history: Where were you in 2017 when we had the worst President in US history?" he said.
Immediately after winning, Perez made a quick move to bring Ellison's supporters into the fold -- appointing Ellison as the deputy chair of the DNC, to cheers and unanimous approval from the crowd.
Ellison called on Democrats to support Perez.
"It's my honor to serve this party under the chairmanship of Tom Perez," he said.
The results brought a chaotic close to a race that stretched on for three months. In the waning days, former Vice President Joe Biden and several aides to former President Barack Obama -- including Valerie Jarrett and David Simas -- made calls to undecided members in an effort to sway them toward Perez.
In Ellison's camp, Vermont Sen. Bernie Sanders, New York City Mayor Bill de Blasio and supportive members of Congress applied pressure on DNC members with phone calls.
The behind-the-scenes calls underscored why the race was broadly seen as a proxy battle between the Sanders-aligned progressive wing of the party, which supported Ellison, and those more closely connected with Hillary Clinton and Obama, who largely backed Perez.
By selecting Perez -- and again spurning Sanders supporters -- Democrats risk the backlash that could come with leaving the left wing of the party disappointed a second time in the past year.
In a statement congratulating Perez, Sanders said "it is imperative that Tom understands that the same-old, same-old is not working and that we must open the doors of the party to working people and young people in a way that has never been done before.
"Now, more than ever, the Democratic Party must make it clear that it is prepared to stand up to the 1% and lead this country forward in the fight for social, racial, economic and environmental justice," he said.
While Obama and Clinton are bigger names, Sanders' reaction is probably what matters most in terms of whether the election result polarizes party members.
Meanwhile, Perez was praised by the most recent Democratic president and Democratic presidential nominee.
Obama touted what he said was the party's track record of expanding the economy, creating new jobs, keeping Americans safe through "tough, smart" foreign policy, and expanding the right to health insurance.
"That's a legacy the Democratic Party will always carry forward," Obama said. "I know that Tom Perez will unite us under that banner of opportunity, and lay the groundwork for a new generation of Democratic leadership for this big, bold, inclusive, dynamic America we love so much."
In a tweet, Hillary Clinton congratulated Perez and Ellison, writing: "Excited for strong, unified party standing for best of our country into the future."
President Donald Trump also tweeted his kudos to the victor, but not without getting in a jab.
"Congratulations to Thomas Perez, who has just been named Chairman of the DNC," he wrote. "I could not be happier for him, or for the Republican Party!"
That prompted Perez to tweet back: "@keithellison and I, and Democrats united across the country, will be your worst nightmare."
Perez, a Maryland Democrat who worked in Obama's Justice Department before serving as labor secretary and making Clinton's short list for the vice presidential nomination, made overtures to Ellison's backers in his nomination speech before the votes were cast.
"Good leaders are great listeners. You will always have my ear, and I will always have your back," Perez told the crowd, promising to "plan strategy together, lift each other up together."
"You will not be underutilized," he said. "You are of import in everything we do. The most important question you will hear from me is, 'What do you think?'"
In a joint news conference late Saturday afternoon, Ellison said he sees his job as serving as a "bridge" for Democratic institutions, adding that he he did not plan to give up his House seat.
Ellison also attempted to rally his supporters behind Perez. "I trust Tom Perez," Ellison said. "If you trust me, then you've got to come on board with Tom Perez."
Perez said the two had discussed naming the race's loser the DNC's deputy chair "for some time."
Democrats had long anticipated the election could require several ballots to settle the contest and might drag on for much of the day Saturday.
But the pool of votes that the candidates aside from Ellison and Perez would amass shrank significantly when New Hampshire Democratic Party Chairman Ray Buckley dropped out a week ago to support Ellison and then South Carolina Democratic Party Chairman Jaime Harrison exited to back Perez.
On Saturday, Pete Buttigieg, the 35-year-old mayor of South Bend, Indiana, dropped out minutes before the vote without endorsing another candidate.
The move allowed Buttigieg, who is seen as a rising star within the Democratic ranks, to reap the benefits of the bigger stage of the DNC race without suffering the embarrassment of a low vote total or angering either camp by making a last-second endorsement.
"We saw the potential of doing well on multiple ballots, but we can do the math," he said in what was supposed to be his nomination speech. "It is time for this process to move toward a solution that we can all get on board with."
Buttigieg's exit left Idaho Democratic Party executive director Sally Boynton Brown as the third-place contender, drawing 12 votes in the first round before dropping out.
Television analyst Jehmu Greene, Air Force veteran Sam Ronan and attorney Peter Peckarsky also ran. Greene received half a vote in the first round of balloting before dropping out. Ronan and Peckarsky received none before doing the same. And one voter backed Buttigieg in the first round despite his having already dropped out of the race.