He was 74 years old.
Hallyday had announced that he was being treated for cancer last March, CNN affiliate BFM reported.
"We all have a piece of Johnny Hallyday inside every one of us," a statement from President Emmanuel Macron'
s office said. "The public today is in tears, and the whole country mourns."
He is survived by his wife Laeticia and their two daughters, Jade and Joy; his son David from his marriage with famed singer and actress Sylvie Vartan and a daughter from his relationship with actress Nathalie Baye.
While the French Elvis was hardly known by many Americans, in the French-speaking world there was only one "Johnny."
Hallyday was born to a French mother and Belgian father on June 15, 1943 as Jean-Philippe Smet. He became interested in singing as a teenager after watching Elvis in a movie.
He captivated French young people in the 1960s, introducing them to motorcyles, fast cars and dance moves like the twist.
"The rock 'n' roll attitude which he brought from the US made him an idol to youth of the 60s," Macron's office said. "When the good days of rock 'n' roll were over, the public could have abandoned him but a new life for him began."
His career spanned for more than half a century, filling up stadiums and selling millions of records. An army of fans followed his tours from big cities to small towns.
As the years passed, Hallyday branched out from his rock 'n' roll origins, catching new trends as they came along. He tried practically every musical genre, from R&B and pop to country and acid rock.
Hallyday held a special place on the French music scene. When it came time to commemorate the first anniversary of the terrorist attack at Charlie Hebdo, organizers chose Hallyday to sing in tribute.
His health had been declining for years. In 2009, he underwent surgery for colon cancer and months later for a herniated disc. Just after the last surgery, he was treated for an infection in Los Angeles, where he was placed into an induced coma, BFM reported.
After learning of Hallyday's passing, actors and singers around the world expressed their condolences on Twitter.
French-Canadian singer Celine Dion called Hallyday a "giant in show business."
"I'm very sad to hear the news that Johnny Hallyday passed away. He was a giant in show business...a true icon! My thoughts go out to his family, his loved ones, and to the millions of fans who adored him for many decades. He will be sadly missed, but never forgotten," she wrote.
Lenny Kravitz tweeted, "Farewell Dear Johnny. Your friendship, sweetness and support are imprinted in my heart. It is an honor to have known you and to have spent time with you and your beautiful family. Your soul is pure Rock and Roll. Repose en paix."
"Sex and the City" actor Gilles Marini remembered him as the "best performer we ever had."
"Heart broken to learn that our French national hero Johnny Hallyday has passed away. He was our Elvis Presley. My heart goes to his family and particularly his wife. His music covered so many generations. He was the best performer we ever had. RIP and thank you," Marini wrote.