Solskjaer's team had the perfect start though, with England sensation Mason Greenwood rifling United in front after 18 minutes.
However, on an afternoon where United struggled, Leicester came roaring back, as goals from Youri Tielemans and Caglar Soyuncu turned the game on its head.
Soyuncu's goal sparked a crazy end to the game, with Marcus Rashford equalizing just minutes later, only for Jamie Vardy to pull Leicester back ahead 54 seconds later.
Patson Daka's first goal for Leicester wrapped up the victory in the dying moments, and meant United has failed to win any of its last three Premier League games and only once in its last five games in all competitions.
The defeat ends United's 29-game unbeaten run away from home -- an English league record.
When asked about his side's performance, the Norwegian called it a "bad day."
"It's always tough when you come here and then, you need to perform to get results," he told the Manchester United website.
"And of course, it had to happen one day that we lost away from home. We didn't deserve more because we didn't perform up to the standards to take points in the Premier League. So we have to look at it, learn from it, which, of course, we've had time now over the international break to look at things we need to do better. Today, we just didn't do things well enough."
Despite the vast resources given to Solskjaer as he attempts to rebuild United, there have been very few rewards for those investments.
The club finished sixth in the Premier League in his first season at the helm after taking charge in December after Jose Mourinho was sacked.
In his two full seasons since, United have finished third and second but have failed to win any silverware, finishing runner-up in last season's Europa League.
And although the talent level could have been queried under previous managers, it's harder to do so under Solskjaer.
The acquisitions of Bruno Fernandes, Raphael Varane and Jadon Sancho have transformed the squad; the return of Cristiano Ronaldo seemingly the final piece of the jigsaw.
But on Saturday at the King Power Stadium, Solskjaer's side looked completely unprepared, with Leicester looking vastly the superior side.
The team appeared disorganized at times, while a side full of attacking prowess lacked penetration.
And this was against a Leicester side which hadn't won since August 28 in the league.
It is the first time United has conceded four goals in an away league game since April 2019.
Given the amount of money and the high expectations at the club, the defeat only ramps up the pressure on Solskjaer to turn results around.
Solskjaer said his side didn't deserve to get anything from the game given the performance.
"We didn't deserve a point. We got the lead, Mason showed his quality and then we give them a goal. All the goals we conceded were disappointing in the way we conceded. It wasn't just the midfield, but we were too easy to play through. Maybe we gave too much space but that just the whole team. I agree we didn't defend well enough. Today, we just didn't do things well enough."