Week 10 Update

This past week has been very productive. I have created and placed more objects in the scene for more interaction, such as a musical keyboard that plays short sound clip as player look at it and some animated butterflies. For static objects, I created an oriental lantern for both hanging lights and standing light by the bed, some outdoor stone light to give more illumination at night, and simple star system made of quads and emissive material that glow in the dark and disappear during day time. For the laying down experience, I noticed during play test that it was hard to interact with the pets, so they needed to be propped up higher. To make it look natural, I created a simple pet tower for them to perch on.

For mechanics, I added a look-at-user script to rotate pets. It is simple but powerful, since it enhances the presence of user.

I also looked more into optimizations earlier in the week, before I added more objects into the scene. I was hitting a wall for a while, since no matter what I did, I can only get around 25-30 fps. Once in a blue moon it would shoot to 45, but it wasn’t a consistent experience. Fortunately on Thursday night, two local OLP members organized a Unity Help session night, and they invited someone from Unity. There were about 5 of us OLPer showed up that night and we talked about our progress. The Unity lady showed up for an hour and answered as many questions we have for her. I brought the latest unity build on laptop, Gear VR & Samsung phone, but I forgot to bring USB cable to connect the phone to the laptop, stupid me.

When it was my turn to ask questions, the Unity lady sat with me and looked over Unity profiler to figure out why my frame rate couldn’t go over 30. The main thing she noticed was my day/night system, which use dynamic lighting. She changed a few lighting setting, like using culling mask on specific layer so Enlighten (Unity lighting system) does not have to compute light on everything, just the few objects within that layer. She also noticed that although I baked my lighting, I have Realtime Global Illumination activate on my baking setting, so she turned it off. The performance seemed a bit better on laptop, but it was hard to tell unless we tested it on the phone.

The following morning I applied the same changes on my work PC and deployed a build into the S7, and to my surprise, the fps went up to around 45-55. Then I proceed to clean up all the geometry in the scene, by changing their lighting setting (under mesh renderer) to use appropriate probes and shadows. Also, since I use raycasting to identify and interact with certain objects, I read up on optimization on that as well. The raycasting script I use has an option to exclude certain layer, so I put all non-interactable objects in that layer. I also removed any unnecessary colliders from these objects. Doing these changes brought my fps to above 55.

After I get to the good fps, I started adding more objects into the scene, one at a time, then rebake lighting and tested the change on phone every single time. It seemed like a lot of work, but it really help me keep track of the performance. To speed up the process, I did most of the testing using GearVR developer mode, so I don’t have to put and pull the phone off the GearVR hardware all the time. I only use the HMD hardware to test for user comfort.

During the Unity help session, I asked a few people to try my latest build. The feedback was pretty uniform, that they felt pretty comfortable in the experience, but they weren’t sure what to do at first. Based on this feedback, I felt the need to create some sort of tutorial for the demo. Due to time constraint, I decided to create a floating manual with text & picture instructions. I started this module last night using Unity geometry & UI elements, until around 3am in the morning. The module worked pretty well, but unfortunately during testing on the phone, I could only get around 35-45 fps.

Since PAX Dev & PAX West are happening this week, here are my to-do list:
– Submit a stable build for demo by Monday night (8/28)
– Optimize tutorial module to bring fps back to 55+ by Monday night
– Build freeze, no more new features unless it’s emergency
– Play test latest stable build at PAX Dev & PAX West to collect feedback
– Start the documentation process (website, updated proposal, screenshots, etc)
– If time permitted, polish existing features, like pet animation and object placements to ensure user comfort.

Have a great Labor day weekend! See you next week.

Week 9 Update

This week was a bit like a roller coaster for me. There were tons of progress done, a lot of design changes, but there were also tons of crashes, weird unexplained issues, and also that big Unity emergency patch. I lost track of how many builds I had to deploy to the phone each day to test performance and comfort, something around 10-15 probably.

Following the plan, I changed the bedroom design from a closed room into a gazebo-type, to give a more open space feeling. The bed and roof were modeled last week, and I worked on its UV and materials earlier in the week. For the wall panels, I found a Chinese wall screen design that reminded me of something I saw in my grandparent’s house.

Using the same design, I replaced all the room wall with this, and I was really pleased with how it look afterward.

As the next step, I replaced pet placeholder art with real 3D assets. I found three low-poly and really inexpensive asset packages from the Unity store and use them in the scene. The pet spawning script was tweaked to show only one pet at a time, at specific place based on user’s position (laying down, reclined or sitting up).  Once the pets spawned at the right locations, I changed the script to treat these pets as pooled objects, so I don’t need to keep instancing and destroying them at run time.

Afterward, I looked up some information to create day & night cycle. I tried following two different tutorials, and the last one seem to work really well. Since this has a lot to do with lighting, I baked lighting on every single updates. The effect was pretty nice, although it tend to be really dark at night time. To compensate, I added one spotlight from the ceiling, and surrounded player’s camera with light probe. This added a bit of ambient light so the scene was not totally dark during night time.

I also adjusted user camera’s height based on position, as a feedback from the user test during game jam. Unlike Rift or Vive where we can track headset position automatically, we need to fake this on mobile VR. During the previous play test, I only had one position for the camera, which was for the reclining option. For those who test it laying down, they felt like floating, and when sitting down, they felt like sinking into the bed. With this height adjustment, it feels more realistic. The camera is higher when we sit down, a bit lower when reclining, and very close to the bed when we lay down.

Things were nice and peachy at this time. Then, as I tried to add more 3D objects onto the scene, like planes with transparency, terrain, vegetation, things started to go wrong. Unity refused to bake any lighting, while spouting nonsense error like ‘cannot generate lighting file’ and that was it. My scene turned completely dark because of it, which made it hard to test. Then, every single build that got deployed onto the phone just crashed. I noticed that before Unity stubbornly saying no to baking, my light map increased from 2 maps to 27 different maps. That day I just walked away from my work PC and went to Seattle Unity Group meetup, where I met an artist friend who was familiar with Unity lighting. I described my problem to him, he asked a few questions.. and we all agree that I might need to just restarted the PC. We also learned about the security patch update in the middle of the meetup. Then I went home, applied the patch, got an error in the middle of patching, and then Unity spout baking error every single time. Fun times. I ended up staying up til 3 am reinstalling Unity, repatching, and just rolled back my project to the time when things were peachy. Hooray for github and version control!

On Friday, after the office hour, I started learning about optimization. I also pay more attention to the fps. My scene performance was really bad. It was around 25-30 fps on Tuesday, and by Friday it was around 12-25. To check for fps, I use Unity profiler, adb catlog, and this script attached to the camera. I wasn’t sure what to do at this point. Then Gabor posted a review from Lauren’s King Crab game, where he pointed out similar fps issue as mine. I followed his suggested setting, and the fps went up a little bit. I wasn’t able to do single-pass render, since my app just stuck at Unity logo every time I tried that.

On Friday night, I joined the OLP discord channel that Tyrus set up. We talked about optimization, and he mentioned batching and watching for draw calls. Through him, I learned about the Unity statistics option during run time on the editor. Thank you, Tyrus!  I started watching the line that says SetPass calls, combined with using Unity frame debugger  (under Window -> frame debugger). I noticed that all the interactable objects in the scene had very high number, something about rendering, I guess. Then I realized that those objects contain outline script. As I researched more, it seems outline effect usually cause performance issue, as shown in a few situations below.  I removed this effect, and my fps was back to around 25-30.

I spent Saturday learning how to modify Unity shader script and managed to create a few variation of mobile shader, convert existing standard materials into mobile mats. After doing this, I was able to add more objects into the scene. I also added a custom skybox, and had to modify the existing day/light script to control skybox exposure. Then call it a day.

Next week plan:
– work more on optimization
– add more object without crashing the app
– add a little bit interaction aside from the pet
– figure out a way for pets to focus on player’s camera for presence
– learn and submit a build to Oculus store.
– build freeze by end of week, due to PAX dev & PAX west.

See you next week!

Week 8 Update

This week was one of those time when I lost track of days and suddenly it was the end of the week. If it wasn’t for persistent calendar notifications, I would totally missed the office hour on Friday too. Thank you, technology!

I spent two days resting to recover from the conferences and game jam. Then, following advice from the conference, I worked on making real assets for this project. The first room to tackle is the bedroom scene. During the game jam in the previous week, I managed to grab a couple friends to try the early unpolished prototype, and they both like the outer space scene, and dislike the bedroom scene. They like the open space feeling, and bedroom barely had any opening. So I spent a few days researching open space room design as an alternative. Initially I wanted to create a Japanese inspired room, with tatami floor, ceiling, shoji wall panels & doors. Moving ahead, I still want to have some Japanese room elements but combined with modern design for the open space feeling. I’m hoping to get this all done by the end of next week.

Here are a couple model I created for bedroom scenes:

Then after listening in the office hour Q&A, I decided to update Samsung S7 O/S from Marshmallow to Nougat. I went to At&t first, explained my situation to them, but they were not able to help. We tried with inserting their sim card onto the phone and looked for update, but nothing happened. Not only that, the phone suddenly said I couldn’t try to update for another 20 something hours. Then I messaged Caroline, another OLP participant. She mentioned to go to Best Buy, and look for Samsung representative there, since she had better luck going by this method. I followed her suggestion and went to the closest Best Buy, and voila, the Samsung guy just plugged my phone to his laptop, push a few buttons and I have Nougat installed. Thank you Caroline!

Plan for next week:
– finalize bedroom level design with real assets
– add 1-2 animal companion to the scene
– start implementing interaction for space and bedroom scenes

See you next week!

Week 7 Update

This week I did not have any time to work on my own project due to conferences and local game jam.  There was DevGAMM on Monday, Casual Connect on Tuesday til Thursday, Seattle Indies game jam on Friday til Sunday. In fact, I’m writing this from the game jam, now that I have some free time after churning art assets over the last 3 days.

I took a lot of notes from the conferences:

From DevGAMM:

  • localizations boost sales & visibility / discovery (which is a big issue for current game developers)
  • VR experience has more impact than video games. Positive – people who play more VR can overcome certain phobias, such as height. Negative – can be traumatizing & bad experience, user no longer interested in VR (not just your product). Design carefully

What I learn from Casual Connect:

From Things to learn from Raw Data:

  1. know target audience (hard core players, expressive, or creative).
  2. Community management. Target VR enthusiast, engage them. Know where to reach the community based on audience (pc gamers use reddit). Do content marketing – for example create graphic novel about in- game content for Comic Con. Run Freaky Friday event where VR devs from different companies try each other’s products, share the result with the community.
  3. Identify the influencers. Give keys at pre-alpha to streamers or influential VR enthusiasts
  4. Key relationship. Build & maintain relationships with platform companies, like Oculus, HTC, Microsoft, etc. Look for opportunities to get featured in the platform store.
  5. Analytic – use data to find information
  6. Launch. Use appropriate information to present at conferences & events. For example, highlight technical talk for GDC, while talking more about contents and features at E3.
  7. Demo & arcade. Give public opportunity to try the product.
  8. Optimizations – happen early & often; level of detail art assets need to be tested in VR; sound location matters (ex, for a gun, the cocking sound, bullet, shooting sfx need to be separated)
  9. Breaking rules- let user have control of the head; if dev need to add feature, make sure it enhances the experience, not making user uncomfortable.

From Shipping the first VR game (Archangel):

  • make no assumptions
  • VR / AR needs a lot more testing than games
  • Need testing using all elements (textured 3d assets, effects, lighting, audio) instead of just greyboxing/ temp assets
  • understand what works
  • expect delays, need more iteration in VR than games
  • expect technical challenges, especially if managing multiple platforms
  • expect process challenges. It’ s even more difficult when working with remote team members
  • When designing interactions, consider different skill levels. Consider user position. There’s a reason stand up meetings are super short. Consider players daily activities. People who work long hours standing might not want to go home & play standing up.
  • UI – text is hard to read
  • Locomotion – still the hardest problem in VR
  • Budgeting – add 100% to budget compared to game dev due to delays, not including hardware. Always add extra time.
  • Stylized art asset can affect a lot in VR. Can’t rely on existing video game assets; they will need to be optimized for VR

From Opportunities and Challenges of Building Games in VR/AR/MR:

  • Challenges: user comfort & fatigue. Unlike video games, it’s hard to play game in VR when we are not feeling well. VR can be isolating than shared experience. Locomotion & motion sickness are still big issues
  • Design tips: don’t go immediate hi-resolution with assets. If needed, reduce amount of things in the world. If it’s not interactable, maybe they need to be removed.   Keep frame rate minimum to 90 (check with device specs). Use spatial sound, cues, haptics. Get experimental to enhance presence.
  • For testing, put a lot of people through the experience, listen to them and also watch their body language. The first test is motion test, to see how the user react to motion sickness. Second test is to see if people understand the experience. The third test is design balancing.
  • Comfort & accessibility – consider designing experience that can be used with only 1 controller. Implement height adjustment for those in wheelchair. Give people alternatives for sensor (?), for example after a while player can get tired of talking, standing, etc.

From State of VR Content in 2017:

  • VR is not casual, nor it’s like everything else (mobile game, PC games, etc)
  • Don’t coast on novelty. People are no longer shocked. They want more experience.
  • Explore both PC and mobile. Mobile systems are getting closer to closer to PC.
  • Be more broad on genre. People are curious.

From Investment Opportunity for VR Gaming Studios:

  • In VR/AR, a lot of creative works come from women. However, lots of funding are not going to them.
  • Investors look at company’s portfolio before investing. They look at the leadership of the company, what they’re thinking about the future, whether they have longetivity, have enough money to survive 18-24 months, also if they’re willing to learn and grow.
  • VCs currently avoid investing on hardware, since they take a while to tweak. They like to invest in market trend, not in long-term R&D projects. They invest in tools & contents. They’re big on location-based entertainment.

For the game jam, the theme is “it’s not a scam”.  I’m working in a team of 5 people (2 programmers, one audio person, one 3D artist.)  We are working on VR experience of a buyer’s remorse, when the user bought their dream space ship, only to find that things break along the way. The development process is interesting. We have one person with Oculus Rift + touch, and two people with HTC Vive, all working on the same project at the same time.  We are using mainly SteamVR and Newton VR for physics.  If anyone is interested, you can download our project at https://github.com/chinnie/SpaceLemonVR .   Currently the main scene is integrated for HTC Vive build, but the unity project main scene can be run on both HTC Vive and Oculus Rift + Touch.

Here is video from the presentation:

Plan for next week:
– sleep in and relaxing on Monday.
– create art assets for Enliven VR

See you next week!