Castile was engaged to be married to Diamond Reynolds, who on Wednesday recorded video of Castile dying after being shot by a police officer in Minnesota.
His mother, Valerie Castile, said her son was the kind of man who would have followed a police officer's orders.
"He's not a gangbanger. He's not a thug. He's very respectable," she said. "And I know he didn't antagonize that officer in any way to make him feel like his life was in danger."
The younger Castile, in his mid-30s, was a quiet man who worked hard as the cafeteria supervisor at J.J. Hill Montessorri Magnet School in the Twin Cities.
"He's a really good person," his mother told CNN's "New Day." "Everybody likes him."
Anna Garnaas, a teacher at the school, described Castile as a "gentle, laid-back sweetheart" who was beloved by elementary students and staff. He served as a role model for his young charges, she said.
She noted there would be a lot of questions from the kids when they returned to school after summer vacation, and she was glad she had time to think about how she would explain his death.
"I am grateful that this is summer, selfishly because I don't know how to talk to them at all about this, especially my little boys who look like Phil," she told CNN Friday. "How do I explain it to them? But come fall we're going to have to take them in our arms and explain that this wonderful man that they knew day to day is not coming back and why. That is the question, right? How do I tell that to a 6-year old, you know?"
At a rally at the school in St. Paul, Valerie Castile said she thought people who didn't know her son would have loved him if they had gotten the chance to meet him.
His sister, Allysza Castile, said her brother had a "wonderful soul" and a "beautiful aura."
Allysza Castile told the crowd that he loved the children at the school.
"Every child here at J.J. Hill was his child," she said. Her brother went to work every day with a big smile, she added.
St. Paul Public Schools issued a statement saying Philando Castile was not only a valued and widely loved employee, but also a product of the school district, having graduated from Central High School in 2001.
He began working for the school district the next year and was promoted to supervisor two years ago.
"Colleagues describe him as a team player who maintained great relationships with staff and students alike. He had a cheerful disposition and his colleagues enjoyed working with him. He was quick to greet former co-workers with a smile and hug," the statement said.
A co-worker said Castile was quiet, respectful and kind.
"Kids loved him. He was smart, overqualified," the unnamed co-worker said in the school district statement. "I knew him as warm and funny; he called me his 'wing man.' He wore a shirt and tie to his supervisor interview and said his goal was to one day 'sit on the other side of this table.'"
His uncle, Clarence Castile, said he saw the young man on Mother's Day when he brought his fiancée and her 4-year-old daughter to the house. Like always, their conversation turned to Philando's job.
"We ended up talking about retirement, what kind of money he's putting away for his retirement and things like that," Clarence Castile said, adding that his nephew had a really good start on saving money.
Michael Hollis said he grew up in the same neighborhood as the Castiles. He never knew Philando Castile to get in any trouble.
"He had a good head on his shoulders and was doing everything the right way," he said at the rally for his longtime friend Thursday evening.
Castile had a concealed carry permit, his fiancée and family said, and his mother said when he told her that he wanted to get a gun, she told him, "Go and get your license."
Castile's mother said Thursday that he and his sister had stopped by her house earlier Wednesday. During the visit, they had discussed the dangers of carrying weapons, even though both of them have concealed carry permits.
"I really don't even want to carry my gun because I'm afraid that they'll shoot me first and then ask questions later," Valerie Castile recalled her daughter saying.
She said she had also had conversations with her son about what to do when he was pulled over by police. Valerie Castile said she focused on one word.
"That was something we always discussed: Comply," she said.
Reynolds says he did comply with everything the officer asked, but he was shot anyway.