(CNN) -- Sri Lanka has rejected criticism that it is not doing enough to provide for the estimated 250,000 refugees following the end of its war with the Tamil Tigers.
Displaced Tamil civilians at Manic farm in the northern Sri Lankan district of Vavuniya.
Relief agencies say more needs to be done to help the refugees.
However, the man in charge of Sri Lanka's refugee camps said the government was in control of the situation.
"Well, we have always stated very clearly that these are our people, these are Sri Lankan citizens and that we as a government would naturally take full responsibility for their welfare and well being," said Mahinda Samarasinghe, minister of disaster management and human rights. Watch Sri Lanka dispell allegations »
A U.N. spokesman said the country's government had not adequately helped refugees and was restricting it from providing aid.
"It is critical at this stage ... that the U.N. is allowed full and unfettered access along with its humanitarian partners to this population which is incredibly fragile having sat under military siege," said Gordon Weiss. "Curiously for reasons that we don't understand the authorities have restricted access quite severely."
Samarasinghe rejected the claim.
"We are disputing that very clearly because what we have told them is that they cannot use their hundreds of U.N. vehicles going into the zones, the big flags, zones and inconveniencing the people," he said. "That is all that we have said. We have said please don't take your vehicles in."
But without their vehicles the United Nations has said it can not get food and aid to people quickly. Watch aid agencies fear for Sri Lanka »
Sri Lankan President Mahinda Rajapaksa declared victory Tuesday in the country's 25-year civil war against the Tamil Tiger rebels. The announcement brought celebrations and dancing to some parts of the country.
But in northeast Sri Lanka, where most of the fighting occurred in last few months, many are struggling for food, clean water, emergency health kits, cooking pots and school supplies, relief agencies say.
Samarasinghe said the government could handle the refugee situation.
"No one is complaining about shortages of food, according to Samarasinghe. "No one is complaining about lack of shelter anymore."
CNN's Sara Sidner contributed to this story.