The closed-door contest follows the postponements of Monday's and Tuesday's games against the White Sox until a doubleheader scheduled for May 28 following unrest in Baltimore.
A source within Major League Baseball told CNN the league is not aware of any prior closed-door games in major league history.
The game at Oriole Park at Camden Yards is scheduled to begin at 2:05 p.m. ET. The game will be televised on the MASN regional cable network.
"After conferring with local officials, it was determined that Wednesday afternoon's game should be played without fan admittance in order to minimize safety concerns," Major League Baseball said.
The office of new Commissioner Rob Manfred said the league and Orioles will keep an eye on the situation in Baltimore.
"Our thoughts are with all those who have been affected by violence in Baltimore, and everyone in our game hopes for peace and the safety of a great American city," Manfred said.
Hall of Famer Frank Thomas tweeted that the series should be moved to a later date. The former White Sox star said, "Playing in front of a empty house makes no sense!!"
The game changes come as the situation in the city remains tense following Monday's riots, with more protests and arrests Tuesday.
Mayor Stephanie Rawlings-Blake said the closed game will mean "another sad day in our city."
"We're a sports town. We love our O's," Rawlings-Blake told CNN's Wolf Blitzer.
The Orioles said tickets from Monday's postponed game can be used for admittance on May 28. Fans with tickets for Tuesday's game must exchange them for tickets for any remaining home game, including May 28.
The team also announced it would move three games (May 1-3) against the Tampa Bay Rays to Florida. The Orioles will still be the home team, the team said.
Fans with tickets for those games and Wednesday's contest can swap their tickets for future games, based on the dollar amount of the ticket.
Exchanges must be completed by June 30, the team said.
The Orioles have averaged more than 33,000 fans in their first nine games at Camden Yards this season, about 73% of capacity.
According to MLB.com
, games have been postponed in the past because of security concerns. In 1992, four Los Angeles Dodgers games were pushed back because of riots after the acquittals of police officers in the Rodney King case.
In 1967, riots in Detroit prompted baseball officials to move games between the Tigers and the Orioles to Baltimore. Opening Day of 1968 was postponed for two days after the killing of the Rev. Martin Luther King Jr.
After the September 11, 2001, terrorist attacks, all professional baseball was called off for one week.
ESPN anchor and baseball expert Keith Olbermann tweeted that a September 1882 game hosted by Worcester had six fans come, the lowest spectator turnout for a major league game.