- by Starling
“People saw that video and they were like, this is amazing. This is entertaining. How did you do this? That's the reaction I got. So it really was a game-changer,” says Elli Raynai, co-creator of Nourish, a lens used for Snapchat’s most recent Lens Fest.
Elli discovered Flipside Studio after an assignment to create an educational piece on how to make a decision. As a creator, he wanted it to be fun, engaging and meet his creative standards. After a positive experience and amazing feedback, he jumped on the opportunity to use Flipside Studio again, this time creating a lens for Snapchat with his partner Irem Harnak.
We had a chat with the two Toronto-based creatives to talk about using mocap animation to create Nourish, overcoming challenges with new projects, and exploring how Flipside’s tools open up new possibilities. “We don't have to limit ourselves. It's really eye-opening, and it's really creative,” adds Irem Harnak, Nourish co-creator.
“I got into interaction design by creating a VR experience and interactive VR experiences in 2018. I worked on it together with Elli...It was a great experience,” says Irem.
With backgrounds in photography and filmmaking, Irem and Elli both have a passion for interactive projects.
“In 2014 I moved over into more interactive projects when I discovered VR, more specifically the Oculus Rift, and I decided to make a movie using that technology in particular,” says Elli. “It was a really, really successful project... From there, we kept making more and more VR projects.”
The duo worked together on interactive VR installation called Made This Way: Redefining Masculinity, a VR project born out of a photo series about masculinity and transgender men that premiered internationally at the 75th Venice International Film Festival in September 2018.
“From that, I started working on different mobile, web and XR projects,” Irem says.
Elli completed an AR residency at Snapchat, where the idea for Nourish, a lens to help people stay fit and motivated, sprouted. He teamed up with Irem once again to combine machine learning, motion capture, 3D and interaction design to help fulfill the vision for the lens, using Flipside Studio.
“When the opportunity came up for doing a lens at Snapchat through the AR residency, I thought, you know, can we use the tool again?” said Elli. “Would this be a great tool to use to get our animations working in Snap?”
Irem has used other motion capture tools in the past, but found them quite difficult.
“You have to make sure all the cameras are functioning. Then there are two softwares you're juggling with, the actor needs to be completely there, and you have to do the warm-up exercises,” she explains. ”Then you have to make sure all the markers are positioned properly. Sometimes it doesn't work and then they have to leave the room. So, it was a lot of back and forth.”
In building Nourish, Irem created animations with motion capture where she had to figure out a way for characters to appear, stay there, and then exit the scene. Her biggest challenge was the learning curve in using Tween Manager in Lens Studio and attaching everything with behaviour scripts.
“Your team helped us by creating a Python script for Blender,” said Elli. “Once we had exported the motion capture data and applied it to the model, it was just a matter of exporting the fbx and then bringing it into Snap Lens Studio.”
Irem explained that due to time constraints, limits with the lockdown, and their limited animation experience, Flipside Studio solved a lot of their obstacles.
“I found Flipside was just so fast,” Irem said. “Even just to get in and then just capture it and then take the stuff out. Just to look at it was quite good compared to what I've seen before.”
Irem was tasked with 3D modeling the characters, but she wasn’t sure where to start.
“Since Elli already worked with Flipside, we were like, we should really do this because we can have our own custom animations,” says Irem.
They contacted Flipside and worked with our artist to go through the rigging process. Once they had their models, they created short animations for each character.
“We got really close and personal with the characters,” says Irem. ”So we tried to imagine how they would walk...Elli jumped into VR and I was kind of guiding him, directing him, telling him, OK, go this way, move that way. We captured really short clips and then we brought them into the software to see how it's reacting.”
It was important that each custom character had their own dance, fun movements and energy.
“Animation gives life to the characters, says Irem. “And we really did it just playing in this little room.”
“I think that the speed that you can prototype, how you can get ideas across with tools like Flipside are like night and day,” says Elli.
“It opens up possibilities for us. Even though we don't know how to do things, that doesn't stop us from doing things. But feeling like there is a tool that empowers us to go ahead, and whatever we think about, whatever characters we want to animate, we can animate,” says Elli. “The sky's the limit in terms of what kind of projects we can work on.”
Elli and Irem were able to create their project in their small home studio. Flipside Studio gave them an affordable option to create unique characters that they couldn’t have accomplished themselves.
“We wanted to put our own spin, our own style, our own imprint on these characters,” says Elli. “And we would have never been able to do that without Flipside.”
The two agreed that having to use very little equipment made things a lot easier for them.
“It made us feel very empowered that we could with very little equipment, you know, a headset and computer, we were able to do something that was impossible,” said Elli. “I think that's what was impressive with Flipside. You get speed, and you get quality at a fraction of the cost.”
Elli’s thought piece on how to make a decision can be found here.
Learn more about Irem and Elli’s Snapchat lens, Nourish, on YouTube.
Want to start making your own mocap animation? Click here to learn more about Flipside Studio.
- by Starling
“When you're doing this mocap stuff live, there's just nothing like having a character seeing the other character in real time . . . picking up a prop and making a joke or throwing something because you're angry. You just can't do this stuff outside of the software that you guys built, outside this mocap concept of animation,” says Alex Albrecht, Head of Caffeine Productions. “I don't know how we go back into a studio and not bring Flipside with us.”
We’ve been following Alex since Diggnation, a weekly video podcast he hosted with Kevin Rose that ran for 7.5 years. Lately, Alex and his team at Caffeine.tv have been using Flipside Studio to produce two animated shows: Live From the 8th Dimension and The Dungeon Run. We sat down with Alex to discuss his love for all things VR, motion capture, and how his team creates animated shows remotely using Flipside Studio.
“I came out here as a computer science guy,” says Alex. His career in the entertainment industry started in the early 2000s. With a computer science degree and a passion for improv comedy, he moved to L.A. and took a job at the RAND Corporation, a nonprofit global policy think tank.
Alex spent a lot of time watching TechTV, and was particularly intrigued by an hour-long daily tech show called The Screen Savers. After meeting a producer from TechTV at a wedding and learning his favourite show was holding co-host auditions, Alex jumped at the opportunity.
“I have an improv background and a computer science degree. Where do I sign?”
Three weeks later he was co-hosting The Screen Savers with Kevin Rose, which led them to create Digg.com and the video podcast Diggnation. From there he created, produced, and hosted content for the Revision3 network, and went on to direct and produce several feature films and commercials.
When his friend started Caffeine.tv, a live social experience app, Alex joined as an advisor, and eventually came on full time to manage original productions. Now he’s the head of production at Caffeine and oversees everything the network produces in house, including the production of two shows with the help of Flipside Studio.
“I always knew that this technology could really get rid of the barrier to entry for animation and for mocap specifically,” says Alex about his first experiences with Flipside Studio. “Because it’s based on a game engine you can literally do anything you want. If you can imagine it, you can do it.”
His biggest challenge was convincing people it was possible to create an hour of content for a weekly animated show. It hadn’t been done before.
“Twenty-two minutes is basically the most anybody makes of animated content. And they take months and months and months to generate that. And here we are going—no we're gonna do an hour and we're gonna do it every week.”
They set up the VR equipment and started to experiment. One night while Alex was away, the team put on the VR headsets, chose their characters, and hit record.
“I watched it and I was like, this is amazing. This is hysterical. It clearly was improvised. It was clearly just three people punchy at work. But it was undeniably engaging.”
From there, Alex’s team at Caffeine created Live from the 8th Dimension, which had a 46-episode run. They used Flipside Studio for every aspect of the show, and became a sort of “power beta tester” putting the software through its paces and pushing it to where it is today.
“We did everything that it was designed for. At every split second of the day. There was no feature that we weren't using extensively,” says Alex.
Years before he learned about Flipside Studio, Alex had a moment while playing Star Trek Bridge Commander, a VR game where you could embody a Star Trek Character on a ship.
He was catching up with one of the other players before everyone else arrived. His friend, as a Vulcan science officer, was telling him about meeting a girl at a bar. “But I'm seeing a Vulcan science officer in a Star Trek-like conference room, and he's like hanging his head in his hands . . . It was hysterical. Like, that was a piece of content I would consume.”
Alex was also learning about Machinima, the concept of using real-time game engines to create cinematic productions, but found the process too complicated.
“But then when Flipside came along and I was like, oh, I get it. It's Machinima. It's Star Trek Bridge Commander. But it also gives us all these production tools that we can understand because we know how production works. It just all came together . . . it was the one tool that really did everything.”
One of Caffeine’s Flagship Shows is The Dungeon Run, a weekly live RPG tabletop narrative experience. The show featured five players sitting together around a table on a massive set with an animatronic puppet as the host. The production design team built elaborate Dungeons & Dragons models, and they had multiple cameras and a jib capturing the action.
When COVID hit in spring 2020, they had to find a way to create the show without having everyone in the same physical space. After some testing, they landed on a concept that has the players on screen on video chat, with the digital map on a virtual table in Flipside Studio. The miniatures are animated and can move in real time.
“That concept has evolved into some of the most amazing Dungeons & Dragons playing that has ever been put to tape . . . it's better than what we do in the studio,” Alex says. “I don't know how we go back into a studio and not bring Flipside with us.”
All 46 episodes of Live From the 8th Dimension are available on YouTube.
The Dungeon Run streams live every Wednesday at 6pm PT https://www.caffeine.tv/thedungeonrun. Watch old episodes here and see how the production has changed with the times.
To learn more about Alex and Caffeine.tv, check out their website.
Want to start making your own mocap animation? Click here to learn more about Flipside Studio.
Watch our unedited interview with Alex below: