In a joint press conference Wednesday alongside Myanmar's de facto leader Aung San Suu Kyi, Tillerson called for a full and independent investigation into the situation in Rakhine state.
"We are very concerned by reports of widespread atrocities committed by Myanmar security forces," Tillerson said.
"What we know occurred in Rakhine state ... has a number of characteristics of crimes against humanity," he added.
"Whether it meets all the criteria of ethnic cleansing we continue to determine ourselves."
More than 615,000 Rohingya refugees have fled the country since August 25, according to aid agencies, pouring into neighboring Bangladesh, which has struggled to deal with the influx.
However, Tillerson was adamant that broad-based sanctions -- as some have called for in Washington -- were not an answer to the crisis.
During his visit to the Myanmar capital Naypyidaw, his first to the country, Tillerson pledged an additional $47 million in aid for refugees, bringing the total the US has earmarked for the crisis to $87 million since August.
Refugees have fled violent clashes in the north of Rakhine state, where Myanmar's military has intensified what it calls "clearance operations" targeting "terrorists" after Rohingya militants attacked police posts
, killing 12 security officials.
Continued reports of horrific suffering and atrocities since the crisis escalated have intensified criticism of Myanmar's government and Suu Kyi in particular.
Representatives of the United Nations and the European Union have described the situation as "ethnic cleansing," an assessment echoed by British Prime Minister Theresa May this week
However, Tillerson said more information was needed for Washington to use that language, and called for an independent inquiry into the causes of the crisis and alleged atrocities.
"The United States will continue to work with our partners to ensure there are consequences for individuals confirmed to have been responsible for atrocities," he said.
Those consequences could include targeted sanctions, but Tillerson was clear he felt broad-based sanctions targeting the Myanmar government or other institutions were not appropriate.
"We want Myanmar to succeed, we want its democracy to succeed," he said. "I have a hard time seeing how (broad-based sanctions) help this crisis."
Acknowledging proponents of sanctions are "well intentioned," he called for patience in dealing with the "very complex situation" in Rakhine.
"It's difficult when you're sitting half way around the world and you see the images from the refugee camps in Bangladesh to not want to just rush to do something," he said.
'I have not been silent'
Speaking alongside Tillerson, Suu Kyi said what was "most important, is that we bring peace and stability to this country, which can only be done on the basis of the rule of law."
Since the crisis began, the Nobel Peace Prize laureate has come in for intense criticism from the international community.
In a highly publicized September speech on the situation in Rakhine, Suu Kyi said Myanmar was not "afraid of international scrutiny," but did not denounce alleged atrocities against the Rohingya community
and appeared to ignore the findings of a report commissioned by her administration.
Myanmar considers the Rohingya refugees illegal immigrants from neighboring Bangladesh, despite the fact that many Rohingya families have lived in Rakhine for decades. Bangladesh considers them Burmese.
Responding to a question about her alleged silence on the issue Wednesday, Suu Kyi said "I haven't been silent."
"What people mean is what I say is not interesting enough," she said. "My statements are not meant to be exciting, they are meant to be accurate."
"I've not been silent, (but) I have not been making very incendiary statements," Suu Kyi added, before thanking Tillerson for the "open mindedness with which you came here."