Calebasse, Haiti (CNN) -- Lelly Laurentus thought he was doing the right thing when he handed his two young children over to an American woman who promised to take them to a better place.
Laura Silsby, a member of Idaho-based New Life Children's Refuge, had given him a flyer from her charity saying that it wanted to help "children who have lost their mother and father in the earthquake or have no one to love and care for them."
She promised schooling, soccer fields and even a swimming pool. She told Laurentus that she had government permission to carry out her plan, he said.
Laurentus loved his girls. They were everything to him, he said. But he also thought of his own life -- he dropped out of school at 15 and worked two jobs to put food on the table.
In the devastating January 12 earthquake, he lost the meager home he had sitting on a hillside in Calebasse.
"I can't stand that they were suffering here," Laurentus said. "I had confidence in the Americans. I trusted them."
So last Thursday night, when Silsby came with a bus, he placed his girls, Soraya, 4, and Leila, 5, on two seats towards the front. He didn't pack any of their things, he said. Not even their teddy bear. The American woman had supplied bags filled with clothes, toys and snacks.
"Will you remember me?" he asked the young girls, who could not, by his own admission, comprehend what was happening.
Then he watched them drive off, with 18 other children from the village.
But the girls never made it out of Haiti.
Silsby and nine other American Baptist missionaries from Idaho, Kansas and Texas were arrested at the border with the Dominican Republic last Friday and detained on charges of illegally trying to take 33 children -- the 20 from Calebasse and 13 others -- out of the country.
They are being held in a jail in Port-au-Prince, and members of the group described conditions to CNN as sparse but said they are getting sufficient amounts of food and water.
U.S. officials have been given unlimited consular access to the Americans and U.S. and Haitian authorities are "working to try to ascertain what happened [and] the motive behind these people, said P.J. Crowley, a spokesman for the State Department, on Monday.
"Clearly there are questions about procedure as to whether they had the appropriate paperwork to move the children," he said.
Crowley said the group was scheduled to appear before a Haitian judge Monday afternoon, but that hearing apparently didn't happen.
A hearing for the group will be held Tuesday at 10 a.m., according to Haitian Information Minister Marie Laurence Lassegue, who said the group will be provided with a professional interpreter.
The missionaries say they were just trying to help the children leave tragedy behind and start life anew.
"We believe we've been charged very falsely with trafficking which is the furthest possible extreme," Silsby, whose group describes itself as a Christian ministry dedicated to rescuing orphaned, abandoned or impoverished Haitian and Dominican children, told CNN Monday. "We literally gave up everything and used up our own income to help these children and by no means [are we] part of that horrendous practice."
Clint Henry, senior pastor at Central Valley Baptist Church in Meridian, Idaho, a church attended by many of the group's members, reiterated that point saying the group's intentions were "upright and pure."
"The intention was simply to go down and try to be an aid in ministering to children that had been orphaned in the quake," Henry said. "It was our intention to be part of a new orphanage. The decision was made that we could house those children in the temporary sites."
But the incident has focused the spotlight on the issue of orphans in Haiti, where aid groups say the quake left tens of thousands of children without anyone to care for them.
The government has accused the American missionaries of kidnapping and is investigating the incident before releasing the children back to their families.
"The children certainly were not fully willing to go, because in some cases, from what I heard, they were asking for their parents, they wanted to return to their parents," Prime Minister Jean-Max Bellerive told CNN.
Haiti's government requires documentation stating approval for children to exit the country. Silsby acknowledged that she had none.
"They really didn't have any paperwork and this is probably a misunderstanding on my part but I did not really know they would be required," she said.
On Monday, Information Minister Lassegue appealed to Haitians to not give up their children to strangers.
"We're talking about people who are coming to countries in distress, after a hurricane, after a tsunami, after an earthquake," she said. "You will never see the children again."
The United States has already expressed concern over the fate of vulnerable orphans and the potential for child trafficking.
Laurentus said he first heard of Silsby through another Calebasse resident, Isaac Adrien, who was working at a Port-au-Prince Christian school.
Adrien said he met Silsby at the school. She asked him whether he knew of people who had suffered in the quake and needed help. Adrien thought of his own community.
Last Thursday, Adrien took her up on an hour-plus drive up to Calebasse, where clouds touch land and people live a simple farming life.
"All the schools are closed here," Adrien said. "I thought it would be a good idea to send the children away."
Silsby assured the residents of Calebasse that she would stay in touch with them, Laurentus said. And that they would be able to see the children whenever they wanted.
Silsby's group told CNN they were going to house the children in a converted hotel in the Dominican Republic.
But without proper documentation, Silsby and the other missionaries were turned away by Haitian police at the border.
"They wouldn't listen to her," Adrien said about Silsby. "She was crying."
Adrien said the 33 children were forced to sleep outside the police station Friday night before Haitian authorities placed them in the care of SOS Children's Villages in Port-au-Prince.
Laurentus said he bore no ill-will toward the Americans; that he still trusted them.
But he wants his girls back home.
As did Adrienne Paulime, who put her 10-year-old daughter Benatide on the bus last Thursday.
"We gave her to the Americans so she could have a better life," said Paulime, wiping her eyes dry.
Benatide's brother Alain Paulime said he received a phone call last Friday from a woman at SOS orphanage. She said his kid sister was crying.
"Can you come pick me up?" Benatide asked Alain.
Paulime said he was anxious to let the orphanage know that Benatide did not belong there. She had a family.
Monday, Alain Paulime and his sister Carline Paulime joined Laurentus in the long trek down the mountain into the crowded capital.
"I want them back home," Laurentus said.
But in the end, they were forced to leave empty-handed, without even seeing them.
The children were traumatized, said George Willeit, head of SOS Children's Villages.
"The emotional security of the kids is very unstable at the moment," he said.
Many of them did not think they would ever see their parents again. He said he would not allow a visit now for fear of raising the children's expectations.
They would have to wait to go home, until the investigation is over.
CNN's Karl Penhaul, Edvige Jean-Francois, Jill Dougherty and journalist Jessica Desvarieux contributed to this report.