Bahamas-born Miller led going into the final straight and plunged over the line to beat Felix, the world champion, into second place, ahead of Jamaica's Shericka Jackson in third.
Miller's dive stirred some controversy on social media, though some prominent names in track and field came to her defense.
"Shaunae Miller's dive was to recover from falling. Sprinters know the quickest way across the line is a well timed lean. Trust me on that," former Olympian Michael Johnson tweeted.
Miller attended college at the University of Georgia, winning the 2013 NCAA indoor title
, and the team there was quick to celebrate her victory.
But the unusual end to the race also sparked plenty of jokes, with Olympics fans suggesting Miller should try out for other sports teams where her diving ability could prove useful.
Felix, who won the 200-meter title in London four years ago, claimed her seventh Olympic medal in total with silver to become the most decorated female track and field athlete in history.
But the title she badly craved eluded her after a thrilling battle with Miller.
"I wanted to win," Felix told reporters. "It's been a tough year. I didn't quite do enough."
Miller, who came second behind Felix at last year's world championships, went out fast from the start and dominated the opening 300m.
But 30-year-old Felix -- competing at her fourth Olympics -- began to force her way back and looked set to overhaul her younger opponent before Miller made an unorthodox move for the line.
"This is the moment I have been waiting for, I just gave it my all," the 22-year-old told reporters after winning the island nation's first medal of Rio 2016.
"I am just so happy, so grateful, such emotions I just can't say.
"I've been going through so much this year. Everyone at home will all be celebrating right now."
Going for gold
At London 2012, Felix was unstoppable -- winning three gold medals.
That came after claiming 200m silver as an 18-year-old debutant at Athens 2004 and then again at Beijing 2008 -- where she helped Team USA win the 4x400m relay.
However, she was unable to defend over the shorter distance in Rio after suffering torn ankle ligaments in April and then failing to qualify at the US trials.
While Felix had never run the individual 400m at an Olympics, her success in relays and her world title had made her one of the favorites.
Meanwhile, Kenya's David Rudisha had no such problems in following up his London 2012 success Monday.
Rudisha became the first 800-meter runner since New Zealand's Peter Snell in 1964 to retain the men's title, winning in one minute 42.15 seconds.
"It's just fantastic. I had no doubts before," he told reporters. "The feeling in my body was good.
"It is great to win such a big competition, my second gold. It's so great. I am so excited. It is the greatest moment of my career."
Algeria's Taoufik Makhloufi claimed silver, with Clayton Murphy of the US taking bronze.
To close the night's action, pole vaulter Thiago Braz da Silva won host nation Brazil's second gold of the Games, and first in track and field.
The 22-year-old beat defending champion Renaud Lavillenie of France with an Olympic-record leap of 6.03 meters.
The pole vault was one of the events delayed by torrential downpours earlier in the evening session.
The conditions were so bad that the first two men's 110-meter hurdles heats had to be re-run when the weather cleared.