(CNN) -- When a film cast includes names like Ben Stiller, Robin Williams and Owen Wilson, a bit of improvisation on set is to be expected.
Ben Stiller and the gang are back in "Night at the Museum: Battle of the Smithsonian."
"I think the movie is really only 25 percent scripted, maybe even less than that," said Shawn Levy, director of the new film "Night at the Museum: Battle of the Smithsonian," which boasts those comedic actors and several others.
"It just so happens that the scenes that get some of the biggest laughs are the improvised scenes, so it feels like more than that, but we had a very good script and it's still very much there."
The sequel to 2006's "Night at the Museum" welcomes back the trio and director as well as other original cast members, including British actor Ricky Gervais, while also adding fresh faces such as Hank Azaria (as a villainous pharaoh) and Amy Adams (as Amelia Earhart). Watch the group laugh about the film »
The new romp features Stiller's character, security guard Larry Daley, in action at the Smithsonian, where some of the pieces from the Museum of Natural History -- the setting for the original movie -- are now being housed.
Story writer Robert Ben Garant said watching the actors take the script and run with it was painless.
"It's always hard to watch bad actors improv on your skit," Garant said. "But when it's Ben Stiller and Jonah Hill and Hank, it's such a pleasure because it always starts with the idea of our script and sometimes they stick to the words, but most of the time they do stuff much, much funnier and weirder. It's really great to watch."
Owen Wilson is back as Jedidiah the miniature cowboy, who is now being held hostage at the Smithsonian. Azaria turns up the mean as Kahmunrah, the bitter brother of the pharaoh from the last film.
Azaria said he thoroughly enjoyed his turn as the film's resident bad guy.
"When the villain is this ridiculous, it's fun," Azaria said. "When he actually has a sense of humor and self-awareness, and goes from frivolous to really mean bully."
"In the end I had to be forced to do the mean stuff because I always wanted to do the silly stuff," he added. "But watching it ... I enjoyed the mean stuff the most. "
Even with the good times on set, filming at the iconic Smithsonian was a monumental task, given that it wasn't always closed while the actors worked.
Stiller said the crowds who watched the movie making magic were respectful and quiet during filming, often breaking into applause after a scene.
"We weren't really doing much, it was just the best audience in the world," he said. "So it was sort of like doing a live show at the Smithsonian."
Funny man Stiller even got to monkey around on set -- literally.
In this film, there were two simians that took on the actor in a scene that found him getting the short end of the branch.
"I basically get slapped by both of them, but I am not allowed to slap them back because of animal rights, so I have to slap a dummy or a tennis ball when they're off-screen," Stiller told the British newspaper, The Mirror. "It's a weird experience."
Robin Williams, who is recovering from heart surgery, said he was excited to return as Theodore Roosevelt -- this time with a twist.
"I got to play two versions of Teddy Roosevelt," Williams said. "One, the Teddy that was in the first movie and the other was kind of a brassy, bitchy Teddy and the one that's a bust of Teddy Roosevelt."
With so many comics on board, it can't help but be a zany ride. Wilson said he believes those good times will translate to the audience.
"Shawn was really great about kind of letting people try things," he said. "So it just had that kind of spirit on it where everybody was having a good time. I think you kind of feel it when watching the movie."
Director Levy said the actors weren't the only ones who had fun.
It's not every day a crew has access to the secret life of a museum, especially the Smithsonian, and Levy immersed himself in the privilege.
"It's such a fun world to explore, it's so dynamic, and the possibilities are literally endless," he said. "I just love telling these stories."
Since filming was such a blast, might there be a "Night at the Museum 3" in the works?
"Ask me after Memorial Day," Levy said, laughing.