(CNN) -- New York Yankees All-Star players Tuesday remembered owner George M. Steinbrenner III as a demanding father figure who had a gentle side.
"He was always good to be around, recalled pitcher Andy Pettitte. "He was tough. He was fair."
Stories about the larger-than-life Yankees owner, 80, provided a backdrop for Tuesday night's All-Star game in Anaheim, California, where the crowd remembered the baseball titan.
Yankees manager Joe Girardi and other members of the team wore black armbands in memory of Steinbrenner.
Pettitte, Alex Rodriguez and Derek Jeter spoke earlier about "The Boss," who died Tuesday morning after suffering a massive heart attack in Tampa, Florida.
They described him as a boss many folks would like to have: Demanding. Caring. Clear about what he wanted: to win.
"I once got a handwritten note from him," Rodriguez told reporters a few hours before the All-Star Game. It said, 'I AM COUNTING ON YOU!' I still hold it dear. I am playing for him."
"He expected perfection and it rubbed off on the organization," said Jeter.
Tributes rolled in all day Tuesday, hours after Steinbrenner's family said he was rushed from his Tampa home to St. Joseph's Hospital, where he died.
"It is with profound sadness that the family of George M. Steinbrenner III announces his passing," the family said in a statement.
"He was an incredible and charitable man. First and foremost he was devoted to his entire family -- his beloved wife, Joan; his sisters, Susan Norpell and Judy Kamm, his children, Hank, Jennifer, Jessica and Hal; and all of his grandchildren.
"He was a visionary and a giant in the world of sports. He took a great but struggling franchise and turned it into a champion."
Under Steinbrenner's guidance, the Yankees have earned seven World Series trophies since 1973. He was the longest-tenured owner in Major League Baseball, having served as principal owner since purchasing the club on January 3, 1973.
"Owning the Yankees," Steinbrenner once said according to MLB.com, "is like owning the Mona Lisa."
The Yankees also won 11 American League pennants under Steinbrenner's ownership.
Major League Baseball Commissioner Allan "Bud" Selig said Steinbrenner was a giant in the game.
"His devotion to baseball was surpassed only by his devotion to his family and his beloved New York Yankees, " Selig said. "He was and always will be as much of a New York Yankee as Babe Ruth, Lou Gehrig, Joe DiMaggio, Mickey Mantle, Yogi Berra, Whitey Ford and all of the other Yankee legends. He was my dear friend for nearly four decades. Although we would have disagreements over the years, they never interfered with our friendship and commitment to each other.
Steinbrenner was a pop culture figure, famously earning some attention in episodes of the TV sitcom "Seinfeld."
"I met the real George Steinbrenner on only one occasion when he actually came and played himself on an episode of 'Seinfeld,'" recalled Jason Alexander, who played George Costanza. "He seemed to really enjoy himself. I did not get to know him but the fact that he allowed himself and his beloved team to be satirized on our show is an indication to me of his true character."
Hall of Fame Yankees catcher Yogi Berra called Steinbrenner a "great friend."
"George was The Boss, make no mistake. He built the Yankees into champions and that's something nobody can ever deny," Berra said in a statement. "George and I had our differences, but who didn't? We became great friends over the last decade and I will miss him very much."
At a press conference Tuesday, Berra said Steinbrenner had long ago apologized for firing the popular manager.
"George said to me it was the 'worst mistake I made in my life' and I said 'George everyone makes mistakes,'" Berra added, choking up at the memory. Berra predicted Steinbrenner would get into the Hall of Fame.
Former Yankees Manager Joe Torre remembered the boss as tough and passionate.
"I will always remember George Steinbrenner as a passionate man, a tough boss, a true visionary, a great humanitarian, and a dear friend. I will be forever grateful that he trusted me with his Yankees for 12 years. ... It's only fitting that he went out as a world champ," he said in a statement.
In an interview Tuesday on CNN's "Rick's List," Torre said, "He did scream, he did get loud, but when you closed that door there was a human side of him too. That's one thing, when you work with George Steinbrenner....you can't pick and choose the parts you like. You have to understand that it's the whole package."
CNN sports contributor Max Kellerman said, "Steinbrenner is the most famous owner in sports, really maybe in the history of American team sports -- it's a very big deal. He was an ongoing character on 'Seinfeld.' ... Steinbrenner has made his way into popular culture, into the cultural consciousness."
Steinbrenner was born on July 4, 1930, in Rocky River, Ohio. His mother, he once said, imbued him with a compassion for the underdog.
He told The New York Times Magazine in 1978 that he "never really appreciated" his father when he was growing up but that now he "can't give enough credit" to him. "Anything I ever accomplish I owe to him," he said then.
Now a baseball icon, Steinbrenner began his major league ownership career in 1973 after he bought the Bronx Bombers from CBS for $8.7 million. Steinbrenner was chairman of the board of the American Ship Building Co., which had an estimated $180 million in sales in 1978. His other ventures including political fundraising, backing of Broadway plays, an interest in the Chicago Bulls basketball team, ownership of the thoroughbred Kinsman Stud Farm in Florida and an extensive holdings in banking operations and Florida real estate.
Steinbrenner was for decades the most colorful, controversial and domineering owner in baseball.
He demanded victory from his teams and managers, firing managers who didn't deliver. Famously, he hired and fired manager Billy Martin half a dozen times in the 1970s and 1980s when the Yankees dominated baseball.
He was willing to shatter ideas about what baseball players should be paid to stack his teams with stars, starting with pitcher Catfish Hunter and slugger Reggie Jackson in the 1970s and continuing up to the present with Derek Jeter and C.C. Sabathia.
But Steinbrenner didn't always know where to draw the line on payments, and stepped over it at least twice -- with illegal campaign contributions to Richard Nixon in 1974, and with a $40,000 payment to a gambler for damaging information about a player. He was suspended from baseball for the Nixon contribution and banned for life for the payment to gambler Howie Spira, according to Major League Baseball. He was reinstated three years later.
He often starred in commercials lampooning himself, including a commercial in which he chewed out Jeter for partying.
On his 80th birthday last week, Steinbrenner thanked fans, friends and family for their good wishes.
"I am very fortunate to have the love and support of a great family and many, many friends. The Yankees and their fans are a large part of what keeps me going. It means a lot," he said.
New York Mayor Michael Bloomberg said, "This is a sad day not only for Yankee fans, but for our entire city, as few people have had a bigger impact on New York over the past four decades than George Steinbrenner. George was a larger-than-life New York figure whose passion and drive to succeed will forever be missed."
Funeral arrangements will be private, the family said, adding there will be an additional public service with details to be announced at a later date.
Steinbrenner's death marks the second loss in so many days for Yankee veterans. Announcer Bob Sheppard died Sunday. Steinbrenner called Sheppard "a good friend and fine man whose voice set the gold standard for America's sports announcers."
CNN Producer Adam Thomas contributed to this report