The group of about 30 men and women interrupted the parliament's Question Time with cries of "you shame us" and "this is a humanitarian crisis."
Parliament was suspended for 30 minutes while officers forcibly removed the protesters, some of whom had superglued their hands to leather in the public gallery, CNN-affiliate Sky News said
They were protesting against the Australian government's policy of processing asylum seekers in offshore detention centers, which has been condemned by rights groups and the United Nations.
The demonstration comes less than two weeks after the government announced a one-off transfer of refugees
from remote camps on Nauru and Manus Island to the United States.
Australian parliament divided
Speaking after Question Time resumed, senior government minister Christopher Pyne said the protests had been the most serious since 1996 and called for an investigation.
"This is obviously a very serious matter," he said, suggesting a member of Parliament may have helped them to gain access.
Opposition Labor Party Leader Bill Shorten said the protests were "the exact opposite of democracy."
"Do we reward those who would seek to stop this parliament operating by walking away from them and giving in to them?" he said.
But Australian Greens Party Leader Richard di Natale met with the protesters after the incident and thanked them.
"We are at a point in this nation's history where we have a government, supported by the Labor party, who have said that it's acceptable for innocent people seeking our help and protection to be turned away," he said in a statement.
"But not only do we turn them away, we torture them."
Human rights groups criticize camps
The protesters claimed to be speaking on behalf of Australians who are unhappy with the bipartisan support for Australia's strict refugee policy.
A number of human rights groups, including Amnesty International and Human Rights Watch, have criticized Australia over their use of offshore detention centers to house refugees who arrive by boat.
Amnesty described the remote camp on the small island nation of Nauru was like an "open-air prison," in a recent report,
adding "vulnerable" children had been put there so they could "suffer."
A United Nations report released in October
found multiple cases of "attempted suicide, self-immolation, acts of self-harm and depression" among children on Nauru.
The Australian government has repeatedly denied claims that the living conditions on Nauru and Manus Islands are unacceptable.
"(Our) commitment is compassionate and it's strong," Prime Minister Malcolm Turnbull said in October