Judges Denny Chin and Barrington D. Parker ruled in favor of the league, while Chief Judge Robert A. Katzmann ruled for Brady.
"We are pleased the United States Court of Appeals for the Second Circuit ruled today that the Commissioner properly exercised his authority under the collective bargaining agreement to act in cases involving the integrity of the game," the NFL said in a statement. "That authority has been recognized by many courts and has been expressly incorporated into every collective bargaining agreement between the NFL and (the NFL Players Assocation) for the past 40 years."
The NFLPA, which represented Brady, said in a statement it is disappointed in the decision.
"We fought Roger Goodell's suspension of Tom Brady because we know he did not serve as a fair arbitrator and that players' rights were violated under our collective bargaining agreement," the NFLPA said. "Our Union will carefully review the decision, consider all of our options and continue to fight for players' rights and for the integrity of the game."
Decision reverses earlier ruling
Monday's ruling reverses a federal judge, Richard M. Berman, who had nullified
Brady's four-game suspension in September because of "several significant legal deficiencies" in how Goodell investigated accusations that footballs were below league-mandated minimum pressure levels at the AFC Championship Game in January 2015.
In Monday's decision, the majority of the panel said it believed that Goodell "properly exercised his broad discretion under the collective bargaining agreement and that his procedural rulings were properly grounded in that agreement and did not deprive Brady of fundamental fairness."
It's not immediately clear what the NFL will do next. The league potentially could reinstate the suspension, but Goodell has not commented publicly
on the case since before Super Bowl 50, and he would not say then whether the four-game ban would be imposed if the league wins on appeal.
In February, Goodell said he was "not going to speculate what we're going to do depending on the outcome. We'll let the outcome be dictated by the appeals court. When it happens, we'll deal with it then."
'More probable than not' that Brady was in on scheme
In May 2015, the NFL imposed the suspension on Brady after an independent investigator found it "more probable than not" that the Patriots quarterback was involved with locker room attendant Jim McNally and equipment assistant John Jastremski in the AFC Championship Game in a scheme to take air out of the footballs New England would use.
The presumed advantage of an underinflated football is that it is easier to catch.
The Patriots defeated the Indianapolis Colts 45-7 and went on to win the Super Bowl that season. Brady has denied wrongdoing.
When the panel of judges grilled attorneys for Brady and the NFL in March, much of that focus was on Brady's destroyed cellphone
, which the NFL believed may have held evidence of a scheme to deflate footballs used in the AFC Championship Game.
Brady accused of destroying potential evidence
In the original investigation, the NFL had asked to see the phone's text messages but lacked subpoena power to force Brady to comply. In the report, the independent investigator, Ted Wells, hired by the NFL said that Brady, who answered questions over the course of one day, did not turn over personal information such as texts and emails.
According to the report, no one said Brady tampered with the footballs, but he was implicated in texts involving -- and interviews with -- McNally and Jastremski.
In July, when Goodell denied Brady's appeal of the suspension, the league said that the athlete's "deliberate destruction of potentially relevant evidence went beyond a mere failure to cooperate in the investigation and supported a finding that he had sought to hide evidence in his own participation in the underlying scheme to alter the footballs."
Brady had said that it was his practice to destroy his phone and SIM cards whenever he got a replacement phone.
Chief Judge Katzmann, who was the lone judge to dissent, said that Goodell "exceeded his authority, to Brady's detriment, by resting Brady's discipline on factual findings not made in the Wells Report."