blog

How Flipside Studio Opens Up New Possibilities for Creators

- by Starling

“People saw that video and they were like, this is amazing. This is entertaining. How did you do this? That's the reaction I got. So it really was a game-changer,” says Elli Raynai, co-creator of Nourish, a lens used for Snapchat’s most recent Lens Fest. 

Elli discovered Flipside Studio after an assignment to create an educational piece on how to make a decision. As a creator, he wanted it to be fun, engaging and meet his creative standards. After a positive experience and amazing feedback, he jumped on the opportunity to use Flipside Studio again, this time creating a lens for Snapchat with his partner Irem Harnak. 

We had a chat with the two Toronto-based creatives to talk about using mocap animation to create Nourish, overcoming challenges with new projects, and exploring how Flipside’s tools open up new possibilities. “We don't have to limit ourselves. It's really eye-opening, and it's really creative,” adds Irem Harnak, Nourish co-creator. 

1. Tell me about yourself.

“I got into interaction design by creating a VR experience and interactive VR experiences in 2018. I worked on it together with Elli...It was a great experience,” says Irem. 

With backgrounds in photography and filmmaking, Irem and Elli both have a passion for interactive projects.

“In 2014 I moved over into more interactive projects when I discovered VR, more specifically the Oculus Rift, and I decided to make a movie using that technology in particular,” says Elli. “It was a really, really successful project... From there, we kept making more and more VR projects.” 

The duo worked together on interactive VR installation called Made This Way: Redefining Masculinity, a VR project born out of a photo series about masculinity and transgender men that premiered internationally at the 75th Venice International Film Festival in September 2018.

“From that, I started working on different mobile, web and XR projects,” Irem says.

Elli completed an AR residency at Snapchat, where the idea for Nourish, a lens to help people stay fit and motivated, sprouted. He teamed up with Irem once again to combine machine learning, motion capture, 3D and interaction design to help fulfill the vision for the lens, using Flipside Studio.

“When the opportunity came up for doing a lens at Snapchat through the AR residency, I thought, you know, can we use the tool again?” said Elli. “Would this be a great tool to use to get our animations working in Snap?”

2. What are your biggest challenges with mo-cap animation? 

Irem has used other motion capture tools in the past, but found them quite difficult. 

“You have to make sure all the cameras are functioning. Then there are two softwares you're juggling with, the actor needs to be completely there, and you have to do the warm-up exercises,” she explains. ”Then you have to make sure all the markers are positioned properly. Sometimes it doesn't work and then they have to leave the room. So, it was a lot of back and forth.”

In building Nourish, Irem created animations with motion capture where she had to figure out a way for characters to appear, stay there, and then exit the scene. Her biggest challenge was the learning curve in using Tween Manager in Lens Studio and attaching everything with behaviour scripts. 

“Your team helped us by creating a Python script for Blender,” said Elli. “Once we had exported the motion capture data and applied it to the model, it was just a matter of exporting the fbx and then bringing it into Snap Lens Studio.”

Irem explained that due to time constraints, limits with the lockdown, and their limited animation experience, Flipside Studio solved a lot of their obstacles.

“I found Flipside was just so fast,” Irem said. “Even just to get in and then just capture it and then take the stuff out. Just to look at it was quite good compared to what I've seen before.”

3. How do you use Flipside Studio for VR mocap? What made you choose Flipside Studio over other mocap animation options?

Irem was tasked with 3D modeling the characters, but she wasn’t sure where to start.  

“Since Elli already worked with Flipside, we were like, we should really do this because we can have our own custom animations,” says Irem.

They contacted Flipside and worked with our artist to go through the rigging process. Once they had their models, they created short animations for each character. 

“We got really close and personal with the characters,” says Irem. ”So we tried to imagine how they would walk...Elli jumped into VR and I was kind of guiding him, directing him, telling him, OK, go this way, move that way. We captured really short clips and then we brought them into the software to see how it's reacting.” 

It was important that each custom character had their own dance, fun movements and energy.  

“Animation gives life to the characters, says Irem. “And we really did it just playing in this little room.”

“I think that the speed that you can prototype, how you can get ideas across with tools like Flipside are like night and day,” says Elli. 

4. How does Flipside Studio make mocap easier?

“It opens up possibilities for us. Even though we don't know how to do things, that doesn't stop us from doing things. But feeling like there is a tool that empowers us to go ahead, and whatever we think about, whatever characters we want to animate, we can animate,” says Elli. “The sky's the limit in terms of what kind of projects we can work on.”

Elli and Irem were able to create their project in their small home studio. Flipside Studio gave them an affordable option to create unique characters that they couldn’t have accomplished themselves. 

“We wanted to put our own spin, our own style, our own imprint on these characters,” says Elli. “And we would have never been able to do that without Flipside.”

The two agreed that having to use very little equipment made things a lot easier for them. 

“It made us feel very empowered that we could with very little equipment, you know, a headset and computer, we were able to do something that was impossible,” said Elli. “I think that's what was impressive with Flipside. You get speed, and you get quality at a fraction of the cost.” 

5. What have you made using Flipside Studio? Where can we find it online?

Elli’s thought piece on how to make a decision can be found here.

Learn more about Irem and Elli’s Snapchat lens, Nourish, on YouTube. 

Find out more about Elli and Irem.

Want to start making your own mocap animation? Click here to learn more about Flipside Studio.

 

Flipside Studio 2021.1 introduces motion capture export, video renderer, stand-ins, and more

- by Lux

Flipside Studio

We're excited to finally take the wraps off of the new version of Flipside Studio, which includes a number of major improvements and new features.

This is also the first version of Flipside Studio to offer paid subscription plans for added functionality. All of the existing functionality of Flipside Studio remains free for everyone, but the watermark removal and some of the new features below will be paid-only.

Visit our pricing page to learn more.

Motion capture data export

Flipside Studio 2021.1 adds the ability to export motion capture data, making Flipside Studio the easiest way to record multiplayer motion capture sessions for use in any animation software or video game. Motion capture data can be exported in the following ways:

  1. Raw BVH files for use anywhere
  2. Unity animations, imported via Flipside Creator Tools
  3. Blender animations, imported via a new Flipside plugin for Blender

How it works is you record your characters with just your VR headset and controllers, and Flipside Studio exports the full-body character movements. If you have Vive Trackers, you can also do full-body motion capture!

Note: Motion capture data export is a paid feature.

Video renderer

Flipside Studio 2021.1 adds a powerful new video renderer for exporting footage of your Flipside Studio recordings.


Features include:

  • Capture multiple cameras in one render pass
  • Queue multiple renders to run back-to-back
  • Resolutions: 180 (Preview), 720, 1080, 2K, 4K
  • Output formats:
    • PNG or EXR image sequence
    • MP4 video
    • WAV audio (separate tracks for each actor)
    • Depth map
  • Adjustable frame rate
  • Adjustable video encoding settings
  • Adjustable anti-aliasing settings

Note: The video renderer is a paid feature.

Improved movement retargeting

Retargeting is the process of converting an actor's movement to the shape and dimensions of the character. Flipside Studio's retargeting system was rewritten from scratch for 2021.1 and the result is a massive improvement to the way characters move.

In our old retargeting system, the larger the difference there was between the actor and the character's dimensions, the less natural their movement would feel. In the new system, actors still control characters just by moving their bodies, but the movement of the character is more true to the character's dimensions. This means the actor and character hands won't always line up, but what the cameras see will be more accurate and look better.

The new retargeting system should be applied automatically, but if your character's movements feel off, you may need to recalibrate to fix it.

Introducing stand-ins

We've added a new feature we call stand-ins which adds the ability to freeze your character poses so they can be used as stand-in references when setting up your shots, and as actor marks during your shoot. Point your teleporter at a stand-in and your teleporter will lock onto it. Teleport into it and you'll automatically teleport to your mark for the shoot and change into the right character too if you weren't already.

And just like Flipside's current actor mark props, you can move them to adjust their positions or remove them individually by grabbing them in Set Builder mode.

While this feature is experimental, you'll find it on the underside of the Characters palette under the Stand-Ins heading. Press the Pose button and you'll be given a 3-second countdown. At the end of the countdown, a stand-in will appear for each user who's in character at the time.

Other improvements

  • Flipside Studio now supports full-hand tracking using the Valve Index controllers.
  • Ambient Occlusion can now be enabled under Settings > Output for improved lighting and shadows. Note that there is a performance cost to enabling Ambient Occlusion.
  • There's a new Microphone tab in the settings which includes several new features to fine-tune Flipside's voice recording and lip-syncing, including:
    • Lip sync gain to amplify your voice for more exaggerated lip-syncing
    • Lip sync noise gate to filter out unwanted background noise
    • Compression settings including attack time, threshold, compression ratio, and compression gain
    • Option to specify whether compression applies only to the recorded voice, only to lip-sync input, or both
  • The teleprompter now supports a few additional character sets, including Greek, Cyrillic, Thai, and Tamil. More will be added in future updates.

Flipside Creator Tools

In addition to the new BVH to Unity animation converter that works alongside the new motion capture data export feature of Flipside Studio, we've also added the following new features to the Flipside Creator Tools:

FaceMirror lets you animate faces on anything

This release introduces a new FaceMirror component that lets you animate faces on non-character objects in sets, such as props or otherwise inanimate objects.

We've included an example scene under FlipsideCreatorTools/Examples/Example-FaceMirror where you can see how it works and how it's setup. Just attach a FaceMirror to any object, then attach additional FacialExpressionReference components to power the elements of the disembodied face.

FaceMirror-powered faces work over multiplayer in Flipside Studio and can be recorded and played back complete with recorded voices.

More sophisticated animated expressions

We've added a new character expression type called AnimationParameters, which lets you control Unity animation parameters to achieve more sophisticated animated expressions.

Publish directly in Unity

You can now publish your custom characters and sets to Flipside Studio directly in Unity through a unified "Build & Publish" button in the Flipside Creator Tools window. This helps reduce iteration time and eliminates the need to hunt down your bundle files or open a separate browser window, enabling one-click publishing. Just reload the character or set in Flipside Studio to see your changes instantly in VR.

Click here to download the latest version of the Flipside Creator Tools 2021.1.

 

Looking back at 2020

- by Lesley Klassen

The global pandemic shone a light on the fragility of life. When looking back at our accomplishments this year we're grateful that we have successes to share. We know many are struggling to keep their spirits up during the holiday season.  We will get through this.  

Remote Work

When the pandemic hit our province we immediately moved to remote work. This is one of our biggest successes in 2020.  We learned we can grow a company, keep on task, and create new processes all from home.  This has expanded our capacity to bring in new talent from all over the world. 

Although our move to remote work has been pretty smooth, we are missing the traditions of our old office.  The photo below is the pole where we would document our post morning meeting chants.

Prototyping an XR viewer app

We finished a prototype of an XR viewership app that we are currently developing with our partners at the Canadian Media Fund. In 2021, we will bring it to market. As part of this project we are releasing three pieces of content including a branching narrative called The Outlaw. We will share more in the new year.

Working with New Partners, Funders and Customers

We wrapped up a project with the Women's Entrepreneurship Fund that supported porting Flipside Studio to the Quest.  This year will expand our efforts and we plan on releasing it in 2021.

We engaged with Seven Oaks School Division and New Media Manitoba to teach virtual production and animation to high school students in Manitoba. During this course, the students worked on a project which was then passed off to a Red River College intern to complete. The COVID Characters is a short documentary created by high school students and made in Flipside Studio. 

We also had the joy of working with Bento Box Entertainment, the makers of Bob's Burgers.  Although we can't share what we did together we can share that working with their team was a highlight for many us at Flipside XR. 

Leaving Early Access and Rebranding 

In June we left early access.  Venture Beat wrote a great article highlighting our accomplishments. This has been two years in the making and we are excited for the future of Flipside Studio. Along with this, we launch a new website and updated our branding.  Our customers work in animation, visual effects, and the interactive industry. Now our brand reflects the professional market we are focused on. 

COVID Relief Funding

As a Canadian company that has innovation at our core, we were eligible for a number of COVID relief programs.  We are grateful that our provincial and federal governments see the value of supporting its workforce.  Thank you to NRC-IRAP, the Canadian Media Fund, and the Province of Manitoba. 

Join our Team

In spite of everything that's happening in the world, we are hiring in 2021.  We know that the world needs tools to create content safely and we will continue to build the tools that Flipside creators need and build out the future of animation.  

Thank you to all of our champions, supporters, creators, family and friends who have kept us going since we started this company in 2014.  Let's keep pushing through this challenging time and emerge from this pandemic better and more prepared for the future to come. 



 

Interview with Alex Albrecht of Caffeine.tv: How Flipside Studio Powers Remote Real-Time Mocap Animation

- by Starling

“When you're doing this mocap stuff live, there's just nothing like having a character seeing the other character in real time . . . picking up a prop and making a joke or throwing something because you're angry. You just can't do this stuff outside of the software that you guys built, outside this mocap concept of animation,” says Alex Albrecht, Head of Caffeine Productions. “I don't know how we go back into a studio and not bring Flipside with us.”

We’ve been following Alex since Diggnation, a weekly video podcast he hosted with Kevin Rose that ran for 7.5 years. Lately, Alex and his team at Caffeine.tv have been using Flipside Studio to produce two animated shows: Live From the 8th Dimension and The Dungeon Run. We sat down with Alex to discuss his love for all things VR, motion capture, and how his team creates animated shows remotely using Flipside Studio.

1. Tell me about yourself.

“I came out here as a computer science guy,” says Alex. His career in the entertainment industry started in the early 2000s. With a computer science degree and a passion for improv comedy, he moved to L.A. and took a job at the RAND Corporation, a nonprofit global policy think tank.

Alex spent a lot of time watching TechTV, and was particularly intrigued by an hour-long daily tech show called The Screen Savers. After meeting a producer from TechTV at a wedding and learning his favourite show was holding co-host auditions, Alex jumped at the opportunity.

“I have an improv background and a computer science degree. Where do I sign?”

Three weeks later he was co-hosting The Screen Savers with Kevin Rose, which led them to create Digg.com and the video podcast Diggnation. From there he created, produced, and hosted content for the Revision3 network, and went on to direct and produce several feature films and commercials.

When his friend started Caffeine.tv, a live social experience app, Alex joined as an advisor, and eventually came on full time to manage original productions. Now he’s the head of production at Caffeine and oversees everything the network produces in house, including the production of two shows with the help of Flipside Studio.

2. What are your biggest challenges with mo-cap animation? 

“I always knew that this technology could really get rid of the barrier to entry for animation and for mocap specifically,” says Alex about his first experiences with Flipside Studio. “Because it’s based on a game engine you can literally do anything you want. If you can imagine it, you can do it.”

His biggest challenge was convincing people it was possible to create an hour of content for a weekly animated show. It hadn’t been done before.

“Twenty-two minutes is basically the most anybody makes of animated content. And they take months and months and months to generate that. And here we are going—no we're gonna do an hour and we're gonna do it every week.”

They set up the VR equipment and started to experiment. One night while Alex was away, the team put on the VR headsets, chose their characters, and hit record.

“I watched it and I was like, this is amazing. This is hysterical. It clearly was improvised. It was clearly just three people punchy at work. But it was undeniably engaging.”

From there, Alex’s team at Caffeine created Live from the 8th Dimension, which had a 46-episode run. They used Flipside Studio for every aspect of the show, and became a sort of “power beta tester” putting the software through its paces and pushing it to where it is today.

“We did everything that it was designed for. At every split second of the day. There was no feature that we weren't using extensively,” says Alex.

3. How do you use Flipside Studio for VR mocap? What made you choose Flipside Studio over other mocap animation options?

Years before he learned about Flipside Studio, Alex had a moment while playing Star Trek Bridge Commander, a VR game where you could embody a Star Trek Character on a ship. 

He was catching up with one of the other players before everyone else arrived. His friend, as a Vulcan science officer, was telling him about meeting a girl at a bar. “But I'm seeing a Vulcan science officer in a Star Trek-like conference room, and he's like hanging his head in his hands . . .  It was hysterical. Like, that was a piece of content I would consume.” 

Alex was also learning about Machinima, the concept of using real-time game engines to create cinematic productions, but found the process too complicated.

“But then when Flipside came along and I was like, oh, I get it. It's Machinima. It's Star Trek Bridge Commander. But it also gives us all these production tools that we can understand because we know how production works. It just all came together . . . it was the one tool that really did everything.”

4. How does Flipside Studio make mocap easier?

One of Caffeine’s Flagship Shows is The Dungeon Run, a weekly live RPG tabletop narrative experience. The show featured five players sitting together around a table on a massive set with an animatronic puppet as the host. The production design team built elaborate Dungeons & Dragons models, and they had multiple cameras and a jib capturing the action. 

When COVID hit in spring 2020, they had to find a way to create the show without having everyone in the same physical space. After some testing, they landed on a concept that has the players on screen on video chat, with the digital map on a virtual table in Flipside Studio. The miniatures are animated and can move in real time.

“That concept has evolved into some of the most amazing Dungeons & Dragons playing that has ever been put to tape . . . it's better than what we do in the studio,” Alex says. “I don't know how we go back into a studio and not bring Flipside with us.”

5. What have you made using Flipside Studio? Where can we find it online?

All 46 episodes of Live From the 8th Dimension are available on YouTube

The Dungeon Run streams live every Wednesday at 6pm PT https://www.caffeine.tv/thedungeonrun. Watch old episodes here and see how the production has changed with the times.

To learn more about Alex and Caffeine.tv, check out their website.


Want to start making your own mocap animation? Click here to learn more about Flipside Studio.

Watch our unedited interview with Alex below:

 

Flipside Studio 2020.1 is now available

- by Lux

Today we are celebrating the launch of Flipside Studio 2020.1, along with a completely updated brand and website refresh.

With over 32,000 recordings, a 48-episode comedy show produced live by Caffeine Studios, and a kid's show streaming on Super RTL, the largest private broadcaster in Germany, all made with Flipside Studio, we couldn't be more excited to be rolling out tons of new features and seeing what our users will create with them.

Choose between a peer-to-peer or server-based multiplayer connections

Flipside Studio now supports not only one type of multiplayer, but two (is that another first in VR?!). Now you can choose the best type of connection for your productions.

Here's a handy reference to help you choose:

  • Peer-to-peer is best when all users are physically closer to each other, like in the same local area network or the same city. Peer-to-peer will give you the lowest latency multiplayer in these cases.
  • Server is best when users are farther apart from each other. Server-based connections will give you lower latency as well as more stable connections over greater distances.

Body physics on characters for more realistic interactions

We've added the ability to add physics colliders on characters so that their interactions with the environment feel more realistic. You can generate colliders for a character in the Flipside Creator Tools with a single click, and then tweak them to suit your needs.

Note that body physics can be very performance intensive, so we recommend going through the generated colliders and removing ones that aren't necessary to achieve the shots you need. Often you can get away with just a collider or two on the arms and midsection of the body, but it can scale all the way up to individual fingers if needed.

Expressions and blinking on texture-based faces

We've added a new facial expression mode that lets you add blinking and expressions to texture-based faces, which lets your characters have more expressiveness while recreating that classic cartoon style. You can also mix and match styles now too, so part of your face can be animated with textures and another can be animated with blend shapes or even Unity animations. The possibilities are pretty much endless.

Stream directly to Zoom using Flipside Studio as a virtual camera device

Flipside Studio can now appear as a virtual camera device that apps like Zoom, Skype, etc. can use as your video source. To enable it, go to Settings > Output on the desktop camera switcher interface and look for the DirectShow settings.

Redesigned auto-scrolling teleprompter with adjustable speed

The teleprompter controller has been revamped to include auto-scrolling with a slider to adjust the scroll speed.

More realistic eye movement

We've made a number of improvements to how eyes move on characters in Flipside Studio, including improved idle movement with simulated saccades and world-space points of interest, more natural blinking, and improved range of motion when eyes lock onto targets of interest.

Trigger effects from interactions with your Twitch chat

We've added a TwitchActions component to the Flipside Creator Tools that lets you define custom Twitch chat commands that users can type which then trigger actions in your custom sets. This can be used for anything from triggering sounds and visual effects to voting on the direction the show's narrative should take.

Over a dozen new types of interactive elements to build your sets

We've added a ton of new interactive functionality to the Flipside Creator Tools, letting you build sets with interactions as complex as games. There's even a scoreboard element that keeps score for everyone in the scene.

Build your own guns (or pea-shooters!), choose-your-own-adventure stories, and more with over a dozen new components. The best part is you can hit Play in Unity and test them all in VR without having to build and upload your sets, making the Flipside Creator Tools a very powerful tool for crafting interactive VR experiences.

VR camera switcher now supports unlimited camera previews

We've added support for an unlimited number of camaras in the VR camera switcher, which is now found under the Show Tools menu, and you can have as many VR camera switchers as you need too.

Improved audio quality and playback performance

We've increased the recorded and multiplayer audio quality in Flipside Studio from 16 kHz (the standard for VoIP) to 24 kHz, which noticeably improves the voice quality particularly in frequencies between 8-12 kHz. We did this while also improving our audio playback performance!

We've finished the migration from Unity 2017 to 2019

With this release, we have now completed our upgrade from Unity 2017 to 2019, meaning all assets should be made using Unity 2019 from now on. This was our first major Unity upgrade since supporting user-uploaded assets and taught us a lot about how to migrate successfully when users also have to update their assets.

If you're still making assets in Unity 2017, now's the time to head over to the Flipside Creator Tools page to find the new version of Unity to use and update those assets! We've done a lot to make it backwards compatible with 2017 assets, but some compatibility issues couldn't be solved automatically.

Bug fixes and improvements since 2020.1.0-rc2

Flipside Studio

  • DirectShow settings can be found under the Settings > Output tab to enable/disable the Flipside Studio virtual camera device.
  • We've added usernames to the front of the ghost/build mode headsets so you can more easily see who's who at a glance.
  • Fixed an issue causing object positioning in hands to be slightly off in playback.
  • Fixed an issue causing embedded camera settings to appear different in set builder mode.
  • Fixed a bug that could cause audio to go out of sync when switching between acting and ghost mode while recording.
  • Fixed not being able to switch out of ghost mode after connecting over multiplayer.
  • Fixed a bug that allowed recorded parts to interact with the handheld camera.
  • Sitting handles feet with toes that don't point forward, such as hoofs.
  • Fixes for errors in DirectShow not working for some users.
  • Fixes for some users experiencing a broken teleprompter.

Flipside Creator Tools

  • Fixed a bug in the character importer that caused eyelids, lashes, and eyewear to be configured as eyes.
  • Fixed some fields not being editable in the Unity inspector window on the AvatarModelReferences component.
  • Improvements to our Daz character auto-importing.

Getting Flipside 2020.1

Flipside Studio 2020.1 is now the current stable release on Oculus Home and SteamVR, so those of you on the Beta channel that don't want the latest experimental features can access this release on the regular Stable channel now. We will be introducing a new beta with some exciting new features in the next few weeks, so feel free to stick to the Beta channel if you're interested in testing those out.

If you make your own custom characters and sets, you will also need to upgrade to the Flipside Creator Tools 2020.1, which can be downloaded from here.

Thank you!

From the Flipside XR team, we want to say a big thanks to all of our users who have helped shape this release by testing and and finding bugs for us to fix. You've all helped make Flipside what it is today.

Sincerely,
Team Flipside

 

Flipside Studio 2020.1 Release Candidate 2

- by Lux

Please note: This update includes some big changes which we wanted to give everyone time to test out safely and adapt to their projects, so to try it out you'll need to opt into our Beta release channel. Instructions on switching between release channels can be found here.

Flipside Studio

Improvements

  • Moved photos taken with the handheld camera to Documents/Flipside Studio/Photos.
  • Increased default gain value on lip syncing to increase expressiveness.
  • DirectShow stays enabled across sessions in the app instead of needing to press Alt+D each time.

Bug fixes

  • Fixed cancelling out of adding a part not stopping playback of the first part.
  • Fixed texture-based face animations over multiplayer.
  • Fixed OBS not connecting if the "Mix/Aux" input source was removed or renamed in OBS.
  • Fixed the virtual camera device not installing if the app isn't installed on C:.
  • Fixed error handling of sets with missing SetInfo component.
  • Fixed the NDI enable/disable option being disconnected.
  • Fixed a camera inconsistency bug caused by incorrectly cloning cameras.
  • Fixed recordings in ghost mode ending as soon as the last prop interaction happens.
  • Fixed handheld camera direction not changing over multiplayer.

Flipside Creator Tools

Click here to download the release candidate of the Flipside Creator Tools (2020.1.0-rc2).

Improvements

  • Added settings to specify multiple wrist twist bones in AvatarModelReferences.
  • Added the Flipside Creator Tools version number to the Flipside window so it's easier to see which version you're using.
  • Added automatic setup for Daz 3D characters (Genesis 8, older versions to follow), however first you'll need to manually fix the spine bones on the model so that the body settings match the following:


Bug fixes

  • Fixed internal inconsistencies between AvatarModelReferences and FacialExpressionReference.

A change to our version numbering

With this update, we're also moving to a new calendar versioning-based version numbering system, which is described in more detail here.

This release of both Flipside Studio and Flipside Creator Tools now share the version 2020.1.0-rc2.

 

Update #38 - Flipside Studio Release Candidate 1

- by Lux

Please note: This update includes some big changes which we wanted to give everyone time to test out safely and adapt to their projects, so to try it out you'll need to opt into our Beta release channel. Instructions on switching between release channels can be found here.

Flipside Studio

Release Candidate 1 is here

We're excited to announce that today's release on the Flipside Studio beta release channel is our first release candidate for a new stable version of Flipside Studio.

What this means is we're holding off on any major new features to focus on fixes and usability improvements until we feel it's ready to move over to the stable channel for everyone to use. If there are no major issues with this release, we'll move it over (along with any fixes we make in the meantime) to the stable channel after a couple weeks of general use.

Once we go stable, it will also complete our migration from Unity 2017 to 2019, so now's the time to update any assets that haven't been updated to Unity 2019 yet.

Bug fixes

  • Fixed remote users not unloading a recording when it's cartridge is removed from the player.
  • Fixed an error that could occur on users rejoining a multiplayer session.
  • Fixed remote users not always seeing new users in character on first joining.
  • Fixed a bug that could cause ToggleElement components not to properly record their current state at the start of a recording.
  • Fixed the Ghost Mode button not working on the camera switchers while in Set Builder mode.
  • Fixed a bug that caused the Recordings palette to disappear and not refresh itself with newly-added recordings.

Flipside Creator Tools

Click here to download the release candidate of the Flipside Creator Tools (2020.1.0-rc1).

Improvements

  • Added wrist settings to AvatarModelReferences so you can adjust the wrists to reduce twisting. The new settings let you adjust the Twist Weight, Parent-Child Crossfade, and Twist Angle Offset values. Assigning multiple twist bones is coming soon too.
  • Improvements to our Mixamo auto-import detection and our CC3 expressions.
  • CC3 importer auto-fixes the jaw bone hierarchy on imported models.

Bug fixes

  • Fixed a bug that was causing parts of faces with multiple blend shapes mapped to one expression not to move.

A change to our version numbering

With this update, we're also moving to a new calendar versioning-based version numbering system, which is described in more detail here.

This release of both Flipside Studio and Flipside Creator Tools now share the version 2020.1.0-rc1.

 

Update #37 - Stream directly to Zoom, Skype, etc.

- by Lux

Please note: This update includes some big changes which we wanted to give everyone time to test out safely and adapt to their projects, so to try it out you'll need to opt into our Beta release channel. Instructions on switching between release channels can be found here.

Flipside Studio

Stream directly to Zoom, Skype, etc.

We've added a plugin that creates a "Flipside Studio" virtual camera device in Windows, so you can select Flipside in your camera list in any video chat software, including Zoom, Skype, Slack, Whereby, etc.

To turn the virtual camera device on, press Alt+D in Flipside Studio. The first time you do, Windows will popup up a notice requesting administrative permission to install the virtual camera device. Give it permission and you should see "Flipside Studio" in your camera options the next time you launch your video chat software of choice.

Note: If you don't see "Flipside Studio" in your camera options, you may need to restart Windows for the change to take effect.

Also note that you will need to press Alt+D to enable the virtual camera each time you run Flipside Studio while this feature is still considered experimental and not enabled by default.

Improvements

  • Added a Re-Sync button under the Recordings palette menu which does a complete re-sync of your multiplayer session should things fall out of sync for any reason.
  • Improved default microphone selection logic when multiple microphone devices are available.
  • Improved audio playback performance.

Bug fixes

  • Fixed voices going out of sync on added parts.
  • Fixed a bug that caused eye rotations to go in the wrong direction.
  • Fixed a bug causing sets to fail to load if they share the same bundle name as the previously loaded set.
  • Fixed a bug in how character bundle version numbers are compared.

Flipside Creator Tools

Click here to download the latest version of the Flipside Creator Tools beta (v0.33).

Duplicate characters and sets

Duplicating the scene file for a character or set can lead to issues building bundles because Unity gets confused when more than one scene file has the same asset bundle label.


To make this less error-prone, we've added a new Duplicate Selected Scene menu option under the Flipside Creator Tools menu which duplicates the scene and also changes the asset bundle label at the same time, making it easy now to create alternate versions of characters and sets for testing.

Bug fixes

  • Fixed an orientation issue with the VR teleporter. It now behaves more consistently like Flipside Studio's teleporter.
  • Fixed a case where the Chair component wouldn't initialize properly for some users.
  • Fixed CC3 character importing to correctly map blend shapes based on names.
 

Update #36 - Auto-scrolling teleprompter, update saved setups

- by Lux

Please note: This update includes some big changes which we wanted to give everyone time to test out safely and adapt to their projects, so to try it out you'll need to opt into our Beta release channel. Instructions on switching between release channels can be found here.

Flipside Studio

Auto-scrolling teleprompter

We've revamped our teleprompter controller to support auto-scrolling with an adjustable speed slider. You can play, pause, rewind, fast-forward, or advance one screen of text at a time.

Saved setups are now editable

Last update we added the ability to delete your saved setups. This update adds the ability to update them as well, so you can now dress your sets exactly how you want them, save your setup, then come back and make further refinements at a later time.

Other improvements

  • Improved finger positioning when grabbing props with physics hands.
  • Added Unity's constraint components to the list of supported component you can use in custom characters and sets.
  • Further improvements to the expressiveness of lip syncing.
  • Improved error handling and reporting during initial load as well as character and set loading.
  • Your controllers now stay visible at all times instead of showing only when the user interface is visible.

Bug fixes

  • Fixed lighting issues going from sets with lightprobes to sets without lightprobes.
  • Prevented getting fingers into strange hold positions grabbing props with physics characters.
  • Fixed camera switcher not scaling with you in ghost mode.
  • Eye rotation fixes on additional characters.
  • Keep the "Press Esc to exit fullscreen" overlaytext hidden if any 3rd party capturing software is open (list includes: OBS, Streamlabs OBS, Xsplit, Twitch Studio, Mixer, Caffeine, VLC, Camtasia Studio, Bandicam, LIV Client, Mixcast).

Flipside Creator Tools

Improvements

  • Added resting position threshold to the Blink All setting on AvatarModelReferences to make the natural eyelid resting position customizable.
  • Added Unity's constraint components to the list of supported component you can use in custom characters and sets.
  • Initial support for auto-setup of imported characters from Character Creator.
  • Added a Resource Usage panel in the Inspector window for AvatarModelReferences and SetInfo components, giving visibility into the resources that your assets are using. The panel also includes recommended maximums for each resource for both PC and mobile. The mobile numbers are the current numbers we're targeting for Flipside Studio on Oculus Quest (still in closed beta testing).

Bug fixes

  • Fixed FlipsideActions.OnHide events not firing correctly.
  • TargetElement now works even if there's no ScoreboardElement in the scene.
  • Calls to FlipsideActions.ChangeSky(id) now change the skybox while testing in the Unity editor.
 

Update #35 - Eye realism, camera switcher improvements, texture-based expressions

- by Lux

Please note: This update includes some big changes which we wanted to give everyone time to test out safely and adapt to their projects, so to try it out you'll need to opt into our Beta release channel. Instructions on switching between release channels can be found here.


More realistic eyes

Eye movement has been improved in the following ways:

  • We've improved the range of motion to feel more natural when looking at eye targets like other characters or the handheld camera.
  • We've added procedural animation when not looking at eye targets that mimics focusing on different points of interest. We keep the gaze on a point within a small range of head movement to add more realism, so eyes don't feel so locked to head movement.
  • We've made blinking feel more natural too.

Camera switcher now found under Set Builder > Show Tools

We've moved the VR camera switcher to the Set Builder menu under Show Tools. This also means you can have as many VR camera switchers as you want!

This means that the desktop camera switcher going fullscreen doesn't open the VR camera switcher any more. Instead, it works totally independently. Press Alt+F to go fullscreen, and press Esc to exit fullscreen.

You'll see the instructions "Press Esc to exit fullscreen" appear on your monitor when you move your mouse, but it only shows up with a deliberate move of the mouse so it won't pop up if you accidentally bump your desk while in VR.

Other Flipside Studio improvements

  • Pointing no longer sticks with trigger presses on SteamVR, instead it points while you have the index finger trigger pressed and stops pointing when you stop pressing.
  • Improved the feel of texture-based facial animations by holding each expression for a minimum number of frames.
  • When you connect Flipside Studio to OBS, your OBS "Mic/Aux" sync offset is set automatically to ensure audio is in time with lip syncing.
  • Added a Delete Setup button to be able to delete previously saved setups.
  • Added the Trails FX Unity plugin to the list of supported components.

Flipside Creator Tools improvements

  • Auto-hide overlay text in the Flipside Creator Tools unless you move your cursor over the Game window.
  • Added a button to the SetInfo component to apply the Flipside skybox to the current scene outside of play mode.
  • Added buttons to SetInfo to copy lighting settings between SetInfo and the Lighting window.
  • Added a gizmo in the Scene window that shows where your Center Eye is located.

The latest version of the Flipside Creator Tools (v0.31) can be downloaded here.

Bug fixes

  • Fixed an issue with the Combo Textures facial expressions having the open/closed mouth textures backwards.
  • Fixed an issue with Textures lip syncing being affected by facial expressions and blinking.
  • Fixed TwitchActions to trigger events for everyone in the multiplayer session, not just the Twitch streamer.
  • Fixed one cause of characters with body physics to end up in contorted positions.
  • Fixed a cause of Twitch chat messages not appearing on the panel.
  • Fixed menu never stopping if your play area is too small.
  • Fixed multiple GunElement components all sharing the same bullet pool.
  • Fixed GunElement occasionally not firing.
 

Older posts »