(CNN) -- The U.N. refugee agency is expanding its presence in war-torn Iraq to help accommodate thousands of displaced people, many of whom believe it's getting safer and easier to return to their homes in once-perilous neighborhoods.
This Iraqi family, which left Tal Afar for Karbala in 2005, is one of the country's many internally displaced families.
Around 140,000 uprooted people trekked back to their dwellings between June and October, said the Office of the U.N. High Commissioner for Refugees.
Most of them were internally displaced -- people who fled their homes in Baghdad and places like Diyala province but took shelter somewhere else in Iraq. Just a few were refugees -- people who traveled to other countries.
Although the return is a trickle compared to the few million who remain displaced inside and outside Iraq, High Commissioner António Guterres said the homecoming is "an encouraging sign."
"It is clear that the security situation has improved," Guterres said in an interview from Geneva, Switzerland.
The UNHCR is to open four more provincial offices in Iraq next year, giving the agency bases in 14 of the country's 18 provinces, and it is doubling its Iraqi budget operations to $81 million. More than 100 staffers -- local and international -- will be involved with the Iraqi government in helping citizens return to their homes.
One key aspect aiding the return is that Iraqi security forces are evicting squatters from peoples' homes, said Guterres, who stressed that the agency isn't pushing Iraqis to return, but wants to support those uprooted people who choose to go back home.
The population flight was a symbol of the devastation wrought by the Iraqi war, and the U.N. agency regards it as the worst population displacement crisis in the Middle East since the events that led to the founding of the state of Israel in 1948.
The UNHCR estimated around 4.4 million Iraqis left their homes, with more than 2.4 million of them staying inside the country and up to 2 million living in other countries, such as Syria and Jordan. Many fled before the Iraqi war started in March 2003, but most were displaced after the conflict began.
Sunni-Shiite sectarian violence that erupted after a Shiite shrine in Samarra was bombed in 2006 sparked civil warfare that also spurred people to flee.
The eventual drop in violence is attributed to various factors, including: The U.S.-led "surge" offensive; the growth of the Awakening Councils, the Sunni militants who decided to back the United States; militant Shiite cleric Muqtada al-Sadr's call for his militia to cease fire; improvements in the Iraqi security forces; and the central government's offensives against Shiite militants in Basra, Baghdad's Sadr City and Amara, and against Sunni militants in the Mosul region.
UNHCR spokesman Ron Redmond said the number of people leaving the country is now relatively small. "We're not seeing large outflows of people going to neighboring states like we once did," he said.
"It needs to be stressed that a primary concern for refugees outside Iraq remains the security situation, but for those who are getting help from host countries and from the United Nations, they may for the time being feel safer and better off elsewhere," he added.
But Redmond said the return of some Iraqis illustrates the "increasing confidence that it is possible to go home."
"Once you get that sort of momentum going, you will see more and more refugees going back."
Guterres spent three days in Iraq last week, visiting Najaf in the Shiite region and Ramadi in the Sunni area. He met with local officials and briefed Iraq's most influential Shiite cleric, Grand Ayatollah Ali al-Sistani, on the agency's work.
His agency is working with Iraqi authorities to help people get their dwellings back or find another place to live, and with access to food, health care, education and proper housing. They hope to help citizens cut through government red tape and return to normalcy.
Guterres said the return process will be arduous. Re-integrating people into their homes will take time and will need to be accompanied by an economic recovery process, he said.
One challenge is finding "an effective mechanism of property compensation" for people who have lost their homes in the war and can't get them back.
"There are a huge amount of things that need to be done," he said. "Let's hope the security improvements will go on."