(CNN) -- As Richard Smith chatted over the phone with his nephew late Tuesday, he noticed something was off.
Robert Joel Halderman pleads not guilty Friday to first-degree attempted grand larceny.
"He was slightly down," Smith said about Robert "Joe" Halderman. "He wasn't his normal, bubbly, cheerful self. He sounded very tired."
Just three days later, Smith woke up to news that his nephew was an extortion suspect -- accused of demanding $2 million from "Late Show" host David Letterman to keep quiet about the funnyman's sexual dalliances with staffers.
Halderman, 51, of Norwalk, Connecticut, is a 27-year CBS veteran, most recently a producer for the network's news magazine "48 Hours." He pleaded not guilty Friday to charges of first-degree attempted grand larceny and was suspended from his job.
The news of the alleged extortion scheme came as a shock to Halderman's family, Smith told CNN's "Situation Room."
"We don't understand anything that would drive Joe into this position," he said. "It's just not his nature."
Smith said Halderman is a "very loving individual with his family," and who "loved his job with CBS."
The uncle recalled Halderman's 2004 divorce, saying his two children left for Colorado with their mother -- a move that upset his nephew.
"I know that Joe was very close to his son and his daughter," Smith said. "His ex-wife moved outside of Denver and took the two kids with her, and Joe was quite upset about that."
Still, he added, "I never heard any hostility from him other than it hurt. So we had no reason to believe he would be depressed enough to get into this kind of situation, and that has to be proven yet."
According to court documents, Halderman was required to pay about $6,800 a month in child support and alimony.
New York prosecutors describe Halderman as a man in debt who was "desperate and willing to do anything."
However, Halderman's defense attorney boasted his client's "impeccable reputation," saying he denies the allegations.
"This story is far more complicated than what you heard this afternoon," attorney Gerald Shargel told reporters Friday, without elaborating. "It's not the open- and-shut case you just heard about."
The day after speaking to his uncle, Halderman allegedly received and deposited a $2 million check from Letterman. He was arrested Thursday outside CBS offices, Manhattan District Attorney Robert Morgenthau said.
Morgenthau said Letterman, who had reported the alleged extortion to his attorney and the district attorney's office, met with Halderman a few times before handing over the check -- and that the last two meetings were audiotaped.
While prosecutors did not reveal where Halderman may have gotten information about Letterman's private life, one of the comedian's assistants, identified by Halderman family members and court documents as Stephanie Burkitt, was once the suspect's live-in girlfriend.
Halderman, who has no criminal record, could face five to 15 years in prison if convicted.