London (CNN) -- UK authorities faced growing questions Saturday after passengers arriving at London's Heathrow Airport faced long lines at passport control.
Waits of up to three hours were reported on Thursday night for non-European Union arrivals.
More delays at immigration control were seen Friday night, according to UK media reports.
The British Airports Authority (BAA), which runs Heathrow, said recent delays had been unacceptable but that it had no control over passport control.
"Immigration is a matter for the Home Office. Immigration waiting times during peak periods at Heathrow recently have been unacceptable and we have called on the Home Office to address the problem as a matter of urgency," a spokeswoman said.
"There isn't a trade-off between strong border security and a good passenger experience -- the Home Office should be delivering both."
There were queues of around an hour on Saturday morning, a BAA spokeswoman said, but lines appeared to be moving faster midday Saturday.
The head of the UK Border Force, Brian Moore, defended its performance.
"We will not compromise border security but we always aim to keep disruption to a minimum by using our staff flexibly to meet demand," he said in a statement.
"The vast majority of passengers pass through immigration control quickly. Queues are caused by a number of factors, including incorrect flight manifests or early or late planes which result in bunching.
"The important factor is to have staff that are flexibly deployed in the right numbers at the right times and this is what we always try to do."
Moore said the agency was working with BAA to monitor flight arrivals and passenger movements for the rest of the weekend.
The delays have caused heightened concern because London is preparing to receive large numbers of additional visitors in July and August when it hosts the Olympic Games.
Leo Lourdes, from London, told CNN he fears this is a recurring problem at Heathrow after experiencing long delays on two journeys through Terminal 5 this month.
Returning from a business trip to Germany Friday evening, he was greeted by "massive, massive queues that stretched all along the corridors" and scenes of confusion as bewildered passengers were herded into line.
Three hours later, he finally made it to the front of the queue, where he saw only a few desks manned, he said.
Many people around him seemed to be angry or disappointed by their welcome to the United Kingdom, especially with so many signs on display heralding the London 2012 Olympic Games, he said.
On April 10 he was met by a similar scene when he returned from a holiday in Costa Rica, Lourdes said.
"It was ridiculous, so slow," he said. "I know these days they have to be very stringent on security but there also seems to be a slight lack of customer responsiveness."
Passenger Chris Windebank also experienced long delays Thursday when he arrived back at Heathrow's Terminal 5 from a business trip to Berlin.
He told CNN was yelled at by Heathrow security staff to "get to the end the queue" and when he questioned what was going on was told "we don't know."
When he made it to the passport control for the EU, he found many of the desks unmanned, he said.
There was "no explanation, no apology. I felt I was treated like commodity, not a valued customer of the airport."
His experience at Heathrow compared very badly to the efficiency he saw at Berlin, Windebank said, and "will kill business for Britain unless it is fixed."
Alastair Campbell, the former Downing Street communications chief under Labour Prime Minister Tony Blair, tweeted Thursday: "If this is what Heathrow T5 border queue is like on an average Thursday Olympic athletes should think about coming soon."
He said European Union passport holders were waiting just under an hour to get through immigration control, while non-EU passport holders were queuing for as long as three hours.
Only three or four staff were on duty to process EU arrivals and three for non-EU arrivals, he said.
Politician Keith Vaz, who chairs the Home Affairs Committee, told the BBC that Immigration Minister Damian Green had been summoned before the committee to explain what was happening.
Moore said the agency would be ready for the extra demand expected during the Olympic Games.
"We are fully prepared to manage busy periods during the Olympics and will be implementing our well rehearsed plans."
Heathrow, one of Europe's busiest airports, is building a special terminal to be used by departing athletes and officials for the three days after the Olympic Games ends.
An extra 40,000 passengers are expected on August 13, the day after the Games finishes, compared with a normal day, BAA said in a statement.
CNN's Jo Shelley contributed to this report.