(CNN)For those standing on its northern side, near San Diego, a fence that divides the United States and Mexico is little more than a physical barrier made of welded metal and steel.
Deported vets helped paint this upside-down US flag on the border. Will they have to remove it?
But just a few yards to the south, near Tijuana, the fence there evokes emotion. For years, artists and community groups have transformed portions of the wall into vibrant visual statements about hot-button issues involving the two countries, such as immigration, human rights and foreign policy.
One mural depicts colorful butterflies swirling around an outstretched hand in a gesture of friendship and unity. In another, the top of the fence gradually disappears as it matches the color of the sky. Some have painted crosses mourning the migrants who have died while traveling through remote stretches of the US Southwest.
And then there's one painting that seems to stand out from the rest. It's an upside-down US flag, with similar crosses. In 2013, a group of deported US military veterans helped paint what they call a symbol of distress for their situation.
Now, they say they are being told by the US government -- which owns the fence and the grounds near it -- to remove or alter the painted flag. The US Border Patrol said it has received complaints about some of the border art and is trying to decide whether they are graffiti or "intentional" murals. It says the paintings were applied without permission.
While questions arise over whether there is room for compromise, the man who first conceived of the design says he is refusing to change it. He says the federal government is trying to squelch free speech -- after five years of no formal objection.
"What we want to do is raise awareness around the issue of deported veterans. It's not about us trying to take on our own government or us trying to create imagery that's hostile toward the United States of America," said Amos Gregory, a San Francisco-based visual artist and a disabled US Navy veteran. He is a US citizen, and while he does not live in Tijuana, he periodically goes there to assist the veterans on other paintings.
The colorful paintings are primarily in an area along the border known as Friendship Circle. Here, San Diego is separated from Tijuana by a pair of parallel running fences that are about 12-15 feet high and 40-50 feet apart.
Friendship Circle, which sits in the space between the structures, is supervised and periodically opened by the US Border Patrol. Family members from both sides of the border meet to have what have been described as "pinky kisses" through a metal mesh and talk.
The fence closest to and facing the Mexican side includes the artwork considered murals by their creators.
Gregory worked with a group of about 20 deported veterans to install the inverted flag painting. Since then, the group has grown to about 60 as more recent deportees began adopting and supporting the project. They periodically touch up the paint when it begins to fade because of salt and moisture from the nearby Pacific Ocean.
Through his work in San Francisco, where he founded the Veterans Mural Project, or Veterans Alley, in the Tenderloin District, the artist discovered the therapeutic effects of painting murals for himself and fellow veterans.
When he realized that another population of veterans -- those deported to other countries -- could benefit from what he says are the healing powers of art, but could not travel to the US to participate in the project, he brought the project to them in 2013.
They realized only one image made sense.
"We want to send an SOS to the American people and to other US military veterans to raise awareness about what's going on with the deportation of members of our military and the hardships that many of them who honorably served in the US armed forces face," Gregory said of the project.
Generally, the onetime immigrants share one thing in common: They were honorably discharged from the military but later convicted of crimes after returning to civilian life. Miguel Perez, Jr., a US Army vet who served in Afghanistan, had his green card revoked and was deported just last month after serving time for a drug felony.
Many of the group of nearly 70 living in Tijuana describe themselves as discharged, discarded and in urgent need of help after risking their lives to protect the United States.
They say they are separated from their families, exiled to a country they barely know and -- in many cases -- still dealing with the wounds of war, psychological and medical issues that they say can't be adequately treated in Mexico.
The flag painting is their distress signal, they say. But while they see it as a legitimate form of protest, displaying the American flag upside down is controversial. It can be interpreted as un-American and disrespectful.
When someone from the US government last week called the United US Deported Veterans support house -- a center in Tijuana that helps deported servicemen find housing, medical care and sources of income -- it was not the message that Hector Lopez, co-founder of the center, was hoping for.
"An agent from the US Border Patrol called and said that we had to change or remove the mural because they can't have anything on United States property that says America is in distress, so either we fix it or they would paint over it," Lopez said.
The call came from Daniel Kitzman, a supervisory border agent for the San Diego Sector of the US Border Patrol who confirmed to CNN that he called Lopez to express the agency's concerns.
"The group can adjust it to a regular flag or whatever they want, but just as long as it's not a distress US flag," Kitzman told CNN.
In the five years since the creation of the mural, which also includes the names of deported veterans in Tijuana, the group had not previously received this type of request, Lopez says.
And, Lopez said, the art does not signify that the United States as a whole is in distress.
"The truth is that it means veterans in distress. We mean no disrespect to the United States, a country we consider home and risked our lives for," Lopez said. "I may have been deported, but I am American, and I took an oath to honor and protect the US flag every day."
Lopez said the veterans would prefer not to get into a fight. They need to work with the US government and immigration agencies, Lopez says, to have any chance of returning to the United States. But the decision is out of their hands. The work belongs to Gregory, Lopez explained.
The US Border Patrol says the flag painting is not the only one being scrutinized.
After CNN made requests for more information regarding the inverted flag painting, the US Customs and Border Protection's San Diego Sector sent a statement saying it has received complaints from the public about graffiti on the south side of the border wall at Friendship Circle.
"As we began to investigate the complaint, it became obvious that skilled artists applied some of the 'graffiti' with the intent of creating murals to send a specific political message and/or to beautify the Border Wall," the statement said.
The chief patrol agent decided to reach out to those who "created these murals and engage them in conversation before making any final decision on remediation or preservation," the statement said, adding the paintings were applied without permission.
But Gregory, the artist, believes that the deported veterans mural is being intentionally targeted by the agency because of its perceived political message.
The agency has not answered CNN's questions asking them to clarify the nature of the complaints they say they have received, where they are coming from and whether they have reached out to any other artists or groups besides the deported veterans.
Gregory says he is perplexed by the US government's apparent sudden interest in the mural. The image has been on the border fence for a long time, it's been photographed and published extensively, and as far as he knows, no other artist or community groups that have created their own murals on the border fence has received a call.
CNN reached out to some other artists that have painted prominent murals in the same area and they said they had not been contacted.
Tijuana-based muralist Enrique Chiu says he has been working since December with hundreds of volunteers every weekend to create what he says will be the world's longest mural. The "Mural of Brotherhood" aims to spread messages of unity and peace. The goal, Chiu says, is to cover more than 600 miles along the border. More than 2,700 volunteers have joined the effort and they have been able to paint without any interruptions, Chiu said.
Mexican-born artist Ana Teresa Fernandez created her mural "Erasing the Border" in 2011 by painting sections of the fence the same color as the sky, giving the illusion that the wall is not there at all. The US Border Patrol did paint over it several months later but she went back to repaint it. Since then, the mural has gone untouched and she says she has not gotten any recent requests to alter or remove it.
Gregory's attorney, Peter Scoolidge, says his client is protected not only by several federal acts that outlaw the intentional or malicious destruction, mutilation or modification of an artist's work, but also by the First Amendment.
The fact that the US government owns the border fence may not give it the right to remove or destroy the work, says another attorney.
Eric Baum, a New York-based attorney who last year successfully litigated the well-known 5Pointz case that resulted in a $6.7 million award to graffiti artists whose works were destroyed by a private developer, says the fact that the mural has been there for five years may suggest implicit consent on the part of the government.
"It appears that the government has not had any issue with the wall being painted, did not object to the art until recently, and, therefore, acquiesced to the creation of this artwork," Baum said.
"In this instance, the government has apparently acknowledged that they have no issue with a mural being painted on the wall, but believe an upside-down flag sends the wrong message."
Gregory, who is no stranger to controversy -- a mural he created last year depicting a police officer-related killing in San Francisco was painted over because the building owners did not like it -- said he is ready to defend the integrity of all deported veterans and his work.
It's unclear how many US veterans have been deported -- officials don't track that figure -- but they are becoming an increasingly vocal group, advocating for their rights as US veterans from Mexico.