The decision allows women to fill about 220,000 jobs that are now limited to men -- including infantry, armor, reconnaissance and some special operations units.
"This means that as long as they qualify and meet the standards, women will now be able to contribute to our mission in ways they could not before. They'll be able to drive tanks, give orders, lead infantry soldiers into combat," Carter said at a news conference Thursday.
His move comes despite the objections of Marine Gen. Joseph Dunford, the chairman of the Joint Chiefs of Staff, who had advocated keeping some roles limited to men.
"There will be no exceptions," Carter said.
Carter's historic announcement comes after years-long reviews, and after public push-back from the Marine Corps, which had sought exceptions to keep positions such as infantry, machine gunner, fire support and reconnaissance to men. A Marine Corps study suggests all-male squads are more effective in combat and less likely to be injured than integrated groups.
Carter acknowledged the Marines' resistance, but said he'd decided to set a policy that covers the full department.
"We are a joint force, and I've decided to make a decision that applies to the entire force," Carter said.
Thursday had originally been selected for Carter's announcement of the policy change so that Dunford, the Marine general and Joint Chiefs of Staff chairman, could join the Defense secretary.
But Dunford "was not comfortable" sharing the stage to explain his disagreement or serve as a "potted plant," so he opted out -- without Carter's objection, a senior Obama administration official told CNN.
"In the end, the chairman didn't feel comfortable having to say he disagreed with his boss," the official said.
Instead, Dunford released a tepid statement -- clearly avoiding saying he agreed with Carter's decision. He said that "in the wake of the secretary's decision, my responsibility is to ensure his decision is properly implemented."
"Moving forward," Dunford said, "my focus is to lead the full integration of women in a manner that maintains our joint warfighting capability, ensures the health and welfare of our people, and optimizes how we leverage talent across the Joint Force."
The policy move will take effect after 30 days, Carter said.
He said the decision doesn't mean there will quickly become an even gender split in most combat positions. He said there are "physical differences on average" between men and women and that "thus far, we've only seen small numbers of women qualify to meet our high physical standards" for some units.
"Going forward, we shouldn't be surprised if these small numbers are also reflected in areas like recruitment, voluntary assignment, retention," he said.
He acknowledged that "some service members, men and women, have a perception that integration would be pursued at a cost of combat effectiveness."
However, Carter said: "The military has long prided itself on being a meritocracy."
Democratic presidential front-runner Hillary Clinton, who hopes to become the first woman to win the race to the White House, praised the move.
"We've seen women in our armed forces prove their heroism and abilities, now our official policy is catching up and women who are qualified for these positions should be able to compete and win them," Clinton said at a New Hampshire event after the announcement.
Two women made history in August by becoming the first female soldiers to complete the Army's Ranger School, but they couldn't apply to join the 75th Ranger Regiment, an elite special operations force -- until now.