"People expect me to say that using the boys' bathroom was super magical and just the best time of my life," Grimm said in a recent interview with CNN. "But I was just using the bathroom," he said, "I went in and left."
What was important for Grimm is that using the boy's room felt natural to him.
And what the Virginia high school student can't understand is the uproar it has caused across the country. School districts that dealt with the issue for years under the radar on a case by case basis are now trying to balance rules concerning transgender bathroom access with the privacy concerns of critics.
"I'm nothing particularly threatening or extraordinary, I'm just another 17-year-old kid" Grimm said.
"I have 17-year-old fears and worries and I have 17-year-old motivations, which is just to get out of high school and have fun with my friends and family. There's just nothing about me that is predatory or dangerous, or warrants the kind of response I got from the community," he said.
Before the community got involved, Grimm and his mother visited Gloucester High School in 2014 to tell officials that although his birth certificate recorded him as a female, he had transitioned to be male and legally changed his name.
The school allowed him to use the boys' bathroom until a community member spoke up and launched two heated meetings that Grimm attended.
"I felt very small in there, because I knew the majority of individuals who were in there were not very positive towards me," he said.
Grimm believes it is based on a misunderstanding about who transgender people are, because most people don't have to think twice about their gender identity.
Transgender people, Grimm says, "have a gender identity which is not consistent with our biological reality."
It's called gender dysphoria, or as Grimm explains it, the feeling that your body does not match what your brain feels.
Grimm emphasizes that a person with the condition can't simply be trained to feel differently.
"It's intrinsic to who they are as a person," he said.
The case drew national attention of groups including the Alliance Defending Freedom, a non-profit legal organization based in Arizona that advocates for religious freedom.
Lawyers for the group sent a letter to the Gloucester County School Board, noting that only a "minuscule percentage" of individuals identify as transgender and that it was necessary for the school board to protect "other students' privacy and free exercise rights" as well as "parents' right to educate their children."
After the community meetings, the school barred Grimm from the boys' room but offered him an accommodation.
Under the new policy, Grimm could use the bathroom in the nurse's office, or newly constructed single stall bathrooms.
Grimm -- the only student mandated to use the single stall bathrooms -- thought the so called accommodation was unacceptable.
"I'm not unisex," he said.
His lawyers filed a lawsuit. A federal appeals court ruled in Grimm's favor deferring to guidance put out by the Obama administration concerning Title IX, a federal law that bans sex discrimination in schools.
That guidance interprets "sex discrimination" to include claims based on gender identity.
Grimm thought that when the school year began this fall, his senior year, he would be able to return to the boys' bathroom.
He believes that his critics conflate an argument with indecency with the issue of bathroom use for transgender people.
"You don't see other people's genitals in the bathroom unless you're looking, which is inappropriate in and of itself," he said, "If an individual was to behave incorrectly in the bathroom, their crime would be misconduct in a bathroom, it would not be existing while transgender."
"The bottom line is I'm a boy like anybody else," he said. "I'm not a freak -- my very existence is not a perversion. I'm just a person who is trying to live his life like anybody else, and that I have to think about my bathroom usage is unacceptable."
This summer, the Supreme Court dealt Grimm a setback, granting a request from Gloucester Board to put a temporary hold on the appeals court ruling while the justices consider an appeal.
If the high court takes up the case, it will be its first case concerning transgender identity. Kyl Duncan, a lawyer for the school board, argued in briefs submitted to the Supreme Court that the lower court was wrong to defer to the Obama administration's guidance.
Duncan acknowledged that some believe that transgender restroom access could be "one of the great civil-rights issues of our time," but he said the administration had overstepped its bounds.
The justices could grant the case, or wait and allow the issue to further percolate in courts across the country.
Grimm said he wasn't exactly blindsided by the Supreme Court's temporary stay, but he was deeply disappointed.
He said, for the brief period when he was able to use the boys' bathroom he never felt push back from his peers who he said respected his gender identity, his name and his preferred pronouns.
"For me and my friends, it's a non-issue. I'm Gavin and that's as far as the situation goes. We talk about video games, we talk about what we want to do that day," he said.
The bathroom controversy "has extended throughout my entire high school experience almost, and it's one that I'd just like to finish so that I can think about high school things -- I can think about being a senior and graduating and going to college," he said.
"It's unfortunate that my thoughts instead are when will I actually be able to use the bathroom at my high school."