Washington (CNN)When Warren Weinstein was taken hostage by al Qaeda in the summer of 2011, his family in Maryland was initially reluctant to take their case to Congress.
Inside the fight to free Warren Weinstein
Weinstein's wife, Elaine, and his two daughters worried drawing attention to their loved one's plight -- especially media attention -- would make Weinstein, a government contractor working with USAID in Pakistan, a more valuable hostage to his al Qaeda captors.
But the family switched course and went to their representatives in Congress in late 2013 after a video of Weinstein -- frail and apparently in declining health -- surfaced in which he said he felt "totally abandoned and forgotten" by his country.
More than a year later, that reluctance evolved into a close bond between the Weinsteins and the Maryland delegation of lawmakers and staff who pressed Weinstein's case with the Obama administration as well as Pakistan. The congressional offices helped the family navigate the maze of government agencies working to free their loved one, according to one of those lawmakers and a Senate aide.
"We don't get choked up too often at work, but this is really hard," said Algene Sajery, a foreign policy adviser to Sen. Ben Cardin, who worked closely on the case said, holding back tears.
Hours earlier, the White House announced that Weinstein had been accidentally killed in a U.S. drone strike in January. The aide's first call was to Weinstein's daughters.
"Myself and my colleague worked really closely with them, talked to them all the time ... they're just such good people," Sajery said of the family. "There's a personal connection there."
Cardin and his staff had tracked the Weinstein case since news of his abduction surfaced in August 2011, but when the family reached out to get help pressing their case, Cardin's staff joined forces with Rep. John Delaney, the Weinsteins' congressman, and began setting up meetings for the family.
Cardin's staff set up meetings for the senator and Weinstein's family with everyone from the U.S. Special Representative to Afghanistan and Pakistan to the Pakistani Ambassador in Washington.
Cardin and Mikulski wrote to President Barack Obama calling on the U.S. to beef up efforts to secure Weinstein's release and dedicate more resources to the cause.
And the staff and lawmakers helped the Weinsteins weave through the network of agencies -- from the FBI to the State Department and the White House-- working to bring Weinstein home.
"We focused on making sure that the capabilities of the government was well coordinated," Delaney told CNN on Thursday. "The government is a bureaucracy and you have to make sure that it's working."
But for the Weinsteins and the families of other American hostages held by terror groups abroad, the government hasn't always worked well enough -- a frustration that was palpable to the staffers on Capitol Hill who worked to help them.
"Unfortunately, the assistance we received from other elements of the U.S. Government was inconsistent and disappointing over the course of three and a half years," Elaine Weinstein, the hostage's wife, said in a statement Thursday. "We hope that my husband's death and the others who have faced similar tragedies in recent months will finally prompt the U.S. Government to take its responsibilities seriously and establish a coordinated and consistent approach to supporting hostages and their families."
It's a criticism that has resurfaced as the families of American hostages were killed abroad spoke up and voiced their frustrations with what they characterized as insufficient U.S. government efforts to bring their loved ones home.
Delaney was "saddened, disappointed and outraged that our government was not able to bring Warren home," he said in a statement on Thursday. And in an interview with CNN later in the day, he strayed away from blaming the Obama administration or individuals in various U.S. agencies, instead pointing the finger at a disjointed system.
The Weinstein family's frustration with that system boiled over last summer when U.S. Army Sgt. Bowe Bergdahl was released by a Taliban-affiliated group in exchange for five Guantanamo Bay prisoners.
The U.S.'s policy not to negotiate with terrorists appeared to have an exception.
"It's one of those things where it's both there's some hope, but at the same time it caused them some great frustration," Delaney recalled, saying the family asked: "Why not Warren?"
But as the State Department called the Bergdahl exchange a "unique situation," the Weinsteins' hopes were dashed.
"That's when they asked us to really step up our efforts," Sajery said. "That's when they decided to really go public."
Delaney introduced a resolution calling on the Obama administration to use all tools necessary to bring Weinstein home and make his return -- and that of other U.S. hostages abroad -- a top priority. Mikulski, Cardin and Sens. Dick Durbin and Mark Kirk of Illinois pushed a similar resolution in the Senate.
And the Weinsteins took to the airwaves, with Alisa Weinstein appearing on CNN's AC360 to make a public appeal asking for her father's release -- and for the U.S. government to do more to secure his freedom.
"My father is just as deserving of freedom as Sgt. Bergdahl, as are all of the Americans who are being held abroad," she said on CNN last June. "You cannot distinguish between these hostages ... They can't just pick and choose, decide that it works to get one person out and then leave everybody else there."
White House press secretary Josh Earnest on Tuesday said "significant resources" were dedicated to try and free Weinstein and said "as painful as it is," the U.S policy of not negotiating with terrorists would remain in place, arguing removing that policy would incentivize kidnappings abroad and put more Americans at risk.
And State Department acting spokeswoman Marie Harf said many officials at the department were in touch with the Weinsteins throughout the process. While the U.S.'s policy of not negotiating with terrorists would remain in place, the State Department is in the midst of reviewing how it works to secure the release of American hostages held by terrorists abroad, she said.
The White House announced the review last fall, which Obama ordered last summer after a spree of American hostages held by terrorists were kidnapped or killed.
But Delaney, the Weinsteins' congressman, is revving up to push for more reforms that will make government agencies more effective at locating and working to free American hostages held abroad.
Delaney on Thursday called for a government effort to streamline the efforts of various agencies and countries in the region that help the U.S. locate American hostages -- an effort he's been pushing in recent months.
The tragedy of Weinstein's death could be just the momentum needed to spur those reforms.
"I think that every single American wants any American held hostage returned," Delaney said. "There's tremendous support to do more."