Project Revisit

Today marked the first day I revisited Enliven VR since the Launchpad program submission. The main reason was I wanted to upgrade to the latest Unity 2017.1.1 version, but I had been hesitating for a while, in fear of things breaking. I also wanted to downgrade Oculus Utilities SDK from 1.16.0-beta down to 1.15.0 stable build.

Why these changes? In Unity 2017.1.0, apk is signed using Schema V2, while currently Oculus store submission only accept apps signed in Schema v1. That was the main reason why I had to create keystore manually to get it signed in Unity, then to resign the build using jarsigner. In 2017.1.1, we don’t have to deal with jarsigner anymore, and looked like I can reuse the existing keystore.

For the utilities downgrade, I have been having issues with camera, especially with objects that very close by. Sometimes I feel like the world move a bit, which definitely not comfortable.  I want to check if using the latest stable build can address this issue.  Oh, also it doesn’t help to see this information below on the release note for the beta version… This note was not there when I started using this utilities in mid July! >.<

As I mentioned previously, I was participating at Seattle VR Hackathon last weekend using my laptop, and we used the latest Unity there. I figure I might as well update Unity on this working machine, and thankfully the upgrade was seamless. Downgrading Oculus Utilities was a bit problematic though.  I started by deleting the OVR folder while Unity was open. Somehow, there were other components here and there that caused the utilities to keep being reloaded even after deletion. After many many attempts of deleting several different folders, I use the handy dandy google search, and stumble over this article. The main thing I learned for there was..  do this deletion when Unity was closed!  I followed that instruction, reopened the project, and Unity stopped trying to reload the OVR files, yay! There were a bit more remnant left, so I kept closing, deleting those components that got left behind. Once things were clean, I imported Oculus Utilities 1.15.0 and everything loaded properly. I built the latest scene and it was successful. Now we are ready for more testing and building.

 

 


Week 12 – Post PAX and Oculus Launchpad Submission

This week is the final week for Oculus Launchpad program. For those who has gone through game jam and hackathons, you know what this means…  the most productive time!

On Monday, I stayed at PAX until it ended. I took one day on Tuesday to recover from PAX Dev and PAX.

Unfortunately I did not have much time to demo my latest build there, since there were too many things going on. I did sat in the AFK room for about an hour, to observe how they run things there. I noticed the room was very quiet. There were people going in and out, but most of them sat down and either take a break, or talked to each other in very soft voice. It was a peaceful place. There were several tables inside with chairs around them. On each tables, there were several children and adults coloring books, crayons and color pencils. There were also flyers about the organizers (TakeThis.org), information about mental health and where to seek help. They even provide professional helps for those who need it during the event.

I flipped through several coloring books, and spent a lot of time going through one book called “Color Me Calm” by Lacy Mucklow. This book was designed for adults who needed to relax, and I took a lot of notes while I was in the room, which I listed below:

Images used to evoke more relaxing responses in people:

  • mandalas – for concentration, meditation & calming
    • sanskrit word for circle or round, even container for secret space
    • in book: circular, repeatable geometric pattern, round or curvy in nature, not jagged
  • nature aspects and patterns – inspirational calming
    • wooded scenes – quiet scenes of forest area evoke calm feeling
    • animal patterns – fish, nautilus, etc
    • natural pattern – snowflakes, ammonites, trees, shells, leaves, flowers
  • geometric pattern – regular, symmetrical pattern can be meditative
    • math basis, symmetrical design = balanced, evoke order, calmness and focus
    • asymmetrical patterns also convey the sense of balance because of exponential and logical pattern they create
  • water themes – envisioning steady or rhythmic flows and picturesque scenery can be soothing
    • water scenes are relaxing for many people, especially photographs and artwork of beaches, ocean
    • music with ocean waves in the background has a calming effect. Some says it reminds them of heartbeat sounds
    • environments of still water such as lakes, ponds, etc give soothing effect similar to white noise
    • water scenes are often associated with vacation, relaxation and fun times

With all these information, I will have to change the other rooms design for Enliven VR.

Back to project. For the final submission, I tried to cram as many polish as I could. Changes for this week:

Per feedback I got from a couple Oculus Launchpad participant and some friends, I decided to remove the keyboard. Most people expected to be able to change the background music by interacting with the keyboard. It was also hard to see from below, and felt unnatural.

Added butterfly behavior and modify existing wing material. Previously, when user gazed at animated creatures, my script enabled and disabled their animation property in Unity, which caused them to animate or freeze in the middle of their movement. It did not feel satisfying, and one feedback I got even expressed concern about ‘harming’ the butterflies. So this week I created a new script that made the butterfly hovering around when we gaze at them. The starting positioned seemed a bit jittery, but once it moved, it looked natural. I also set the butterfly to always animate, so they either float or hovering this time. For the wing material, it was originally very transparent, with a bit black outlines. Based on optimization documents I read, transparency really affects performance, and it is generally better to have less transparent areas. So I updated the butterfly texture to reduce the transparency and made it more colorful as well. As a result, they are easier to spot this time.

Added pets behavior. Similar to the butterfly, previously each pet started with disabled animation, and as user gaze at them, they started to move. I used the generic animator controller too, so their movements were not the greatest. Those who tested my app could tell that although the red panda was very energetic, the kitten only blinked and the husky pup walked in place unnaturally. This week I spent some time to create custom animator controller for each animal. Each pet now has four to five different animation cycles, and as user gaze at it, the pet movement changed randomly. The cat starts at idle, then it can meow, jumped a bit, or do a high jump. The pup starts at idle (sitting on its butt), then laying down, scratching its head, or sleeping. The red panda also has different movement per gaze, instead of one giant set of movements like before. These behavior will need to be tweaked more in the future to give even more personality and natural feeling.

Updated terrain geometry. Previously, I have had no luck in building Gear VR application when using Unity terrain tool. My builds tend to not running on the phone during testing. I have tried smaller terrain, with like 10 x 10 and still facing the same issue. The terrain geometry also cannot be scaled using transform properties. So in the previous builds, I just use a quad as a terrain and colored it green. This time around, I created a bumpy 10×10 planes in 3ds Max, brought the mesh to Unity, applied material and it worked like a charm. No issue in building, no performance issue during run time, and it looked a lot better than the flat quad.

Added temporary art for pillows and blanket. One of my tester was confused about the bed, since it is round, and not a standard rectangle-shaped bed. For a clear visual, I added pillows and blanket, but since time was limited, I did not have much time to implement any textures. Originally I wanted to create a Turkish comforter and bedding set, after my conversation with a fellow Launchpadder with Turkish background who mentioned about the dying tradition of handmade comforter. This will definitely be implemented in the future.

Fixed spotting and weird shadows on light-baking. After I started playing around with light baking in Unity, some assets, like the gazebo floor, pet tower, and the new added blanket had mold-like spotting or hatched lines. I was not sure what causing it and had been ignoring it for a while. Near the end of the week, one of my client started getting his hands wet lighting for his game as well, and he saw similar issue. He asked me about it, and I told him some lighting setting I have tried, like turning off shadows, etc. Also, out of curiosity, I looked up this behavior on google, and found out that it was caused by overlapping UV in the model. I updated the blanket model, fixed its UV overlap, re-baked the lighting again and voila, the spots were gone!  I shared my finding with my client, and proceed in updating the rest of problematic geometry. My environment looked a lot cleaner afterward.

Due to the natural disaster happening in Florida and Texas, Oculus Launchpad submission got extended from Sunday night to Tuesday at noon. As much as I tried to get much done, on Saturday I started to feel sore throat, and by Sunday it proceed to sinus infection. My husband had been down with flu since the end of PAX, and I was miraculously unaffected. However, the bug in the air finally hit me. So on Saturday, I finished writing the proposal, budget list. I spent half a day on Sunday to set up a wordpress site for Enliven VR main page, but I couldn’t get any style to work to my liking, so I scrapped it and just built it from scratch, using handy dandy notepad++ and my outdated knowledge of HTML and CSS. I spent Sunday night building the latest version of the app, and as previously having issue with jarsigner.

After taking some medicines this morning, I proceed to rebuild the app, resigned, and after a couple failed attempts, I noticed typo on the application path during jarsigning session. Once I use the correct path, it worked!  Again, based on my experience from game jam and hackathon, when you have a working build very close to submission, might as well submit it before proceeding into making a broken build in the next few minutes. I did the submission for both builds and documentations, and that’s it for this week.

Time to fight the PAX pox now. Next weekend will be the anticipated Seattle VR Hackathon, so I will try my best to rest before then. I might also take a bit of break from Enliven VR development to get a clear vision and reconsider design changes. So, see you in a week or two~


Week 10 Update

This past week has been very productive. I have created and placed more objects in the scene for more interaction, such as a musical keyboard that plays short sound clip as player look at it and some animated butterflies. For static objects, I created an oriental lantern for both hanging lights and standing light by the bed, some outdoor stone light to give more illumination at night, and simple star system made of quads and emissive material that glow in the dark and disappear during day time. For the laying down experience, I noticed during play test that it was hard to interact with the pets, so they needed to be propped up higher. To make it look natural, I created a simple pet tower for them to perch on.

For mechanics, I added a look-at-user script to rotate pets. It is simple but powerful, since it enhances the presence of user.

I also looked more into optimizations earlier in the week, before I added more objects into the scene. I was hitting a wall for a while, since no matter what I did, I can only get around 25-30 fps. Once in a blue moon it would shoot to 45, but it wasn’t a consistent experience. Fortunately on Thursday night, two local OLP members organized a Unity Help session night, and they invited someone from Unity. There were about 5 of us OLPer showed up that night and we talked about our progress. The Unity lady showed up for an hour and answered as many questions we have for her. I brought the latest unity build on laptop, Gear VR & Samsung phone, but I forgot to bring USB cable to connect the phone to the laptop, stupid me.

When it was my turn to ask questions, the Unity lady sat with me and looked over Unity profiler to figure out why my frame rate couldn’t go over 30. The main thing she noticed was my day/night system, which use dynamic lighting. She changed a few lighting setting, like using culling mask on specific layer so Enlighten (Unity lighting system) does not have to compute light on everything, just the few objects within that layer. She also noticed that although I baked my lighting, I have Realtime Global Illumination activate on my baking setting, so she turned it off. The performance seemed a bit better on laptop, but it was hard to tell unless we tested it on the phone.

The following morning I applied the same changes on my work PC and deployed a build into the S7, and to my surprise, the fps went up to around 45-55. Then I proceed to clean up all the geometry in the scene, by changing their lighting setting (under mesh renderer) to use appropriate probes and shadows. Also, since I use raycasting to identify and interact with certain objects, I read up on optimization on that as well. The raycasting script I use has an option to exclude certain layer, so I put all non-interactable objects in that layer. I also removed any unnecessary colliders from these objects. Doing these changes brought my fps to above 55.

After I get to the good fps, I started adding more objects into the scene, one at a time, then rebake lighting and tested the change on phone every single time. It seemed like a lot of work, but it really help me keep track of the performance. To speed up the process, I did most of the testing using GearVR developer mode, so I don’t have to put and pull the phone off the GearVR hardware all the time. I only use the HMD hardware to test for user comfort.

During the Unity help session, I asked a few people to try my latest build. The feedback was pretty uniform, that they felt pretty comfortable in the experience, but they weren’t sure what to do at first. Based on this feedback, I felt the need to create some sort of tutorial for the demo. Due to time constraint, I decided to create a floating manual with text & picture instructions. I started this module last night using Unity geometry & UI elements, until around 3am in the morning. The module worked pretty well, but unfortunately during testing on the phone, I could only get around 35-45 fps.

Since PAX Dev & PAX West are happening this week, here are my to-do list:
– Submit a stable build for demo by Monday night (8/28)
– Optimize tutorial module to bring fps back to 55+ by Monday night
– Build freeze, no more new features unless it’s emergency
– Play test latest stable build at PAX Dev & PAX West to collect feedback
– Start the documentation process (website, updated proposal, screenshots, etc)
– If time permitted, polish existing features, like pet animation and object placements to ensure user comfort.

Have a great Labor day weekend! See you next week.


Week 7 Update

This week I did not have any time to work on my own project due to conferences and local game jam.  There was DevGAMM on Monday, Casual Connect on Tuesday til Thursday, Seattle Indies game jam on Friday til Sunday. In fact, I’m writing this from the game jam, now that I have some free time after churning art assets over the last 3 days.

I took a lot of notes from the conferences:

From DevGAMM:

  • localizations boost sales & visibility / discovery (which is a big issue for current game developers)
  • VR experience has more impact than video games. Positive – people who play more VR can overcome certain phobias, such as height. Negative – can be traumatizing & bad experience, user no longer interested in VR (not just your product). Design carefully

What I learn from Casual Connect:

From Things to learn from Raw Data:

  1. know target audience (hard core players, expressive, or creative).
  2. Community management. Target VR enthusiast, engage them. Know where to reach the community based on audience (pc gamers use reddit). Do content marketing – for example create graphic novel about in- game content for Comic Con. Run Freaky Friday event where VR devs from different companies try each other’s products, share the result with the community.
  3. Identify the influencers. Give keys at pre-alpha to streamers or influential VR enthusiasts
  4. Key relationship. Build & maintain relationships with platform companies, like Oculus, HTC, Microsoft, etc. Look for opportunities to get featured in the platform store.
  5. Analytic – use data to find information
  6. Launch. Use appropriate information to present at conferences & events. For example, highlight technical talk for GDC, while talking more about contents and features at E3.
  7. Demo & arcade. Give public opportunity to try the product.
  8. Optimizations – happen early & often; level of detail art assets need to be tested in VR; sound location matters (ex, for a gun, the cocking sound, bullet, shooting sfx need to be separated)
  9. Breaking rules- let user have control of the head; if dev need to add feature, make sure it enhances the experience, not making user uncomfortable.

From Shipping the first VR game (Archangel):

  • make no assumptions
  • VR / AR needs a lot more testing than games
  • Need testing using all elements (textured 3d assets, effects, lighting, audio) instead of just greyboxing/ temp assets
  • understand what works
  • expect delays, need more iteration in VR than games
  • expect technical challenges, especially if managing multiple platforms
  • expect process challenges. It’ s even more difficult when working with remote team members
  • When designing interactions, consider different skill levels. Consider user position. There’s a reason stand up meetings are super short. Consider players daily activities. People who work long hours standing might not want to go home & play standing up.
  • UI – text is hard to read
  • Locomotion – still the hardest problem in VR
  • Budgeting – add 100% to budget compared to game dev due to delays, not including hardware. Always add extra time.
  • Stylized art asset can affect a lot in VR. Can’t rely on existing video game assets; they will need to be optimized for VR

From Opportunities and Challenges of Building Games in VR/AR/MR:

  • Challenges: user comfort & fatigue. Unlike video games, it’s hard to play game in VR when we are not feeling well. VR can be isolating than shared experience. Locomotion & motion sickness are still big issues
  • Design tips: don’t go immediate hi-resolution with assets. If needed, reduce amount of things in the world. If it’s not interactable, maybe they need to be removed.   Keep frame rate minimum to 90 (check with device specs). Use spatial sound, cues, haptics. Get experimental to enhance presence.
  • For testing, put a lot of people through the experience, listen to them and also watch their body language. The first test is motion test, to see how the user react to motion sickness. Second test is to see if people understand the experience. The third test is design balancing.
  • Comfort & accessibility – consider designing experience that can be used with only 1 controller. Implement height adjustment for those in wheelchair. Give people alternatives for sensor (?), for example after a while player can get tired of talking, standing, etc.

From State of VR Content in 2017:

  • VR is not casual, nor it’s like everything else (mobile game, PC games, etc)
  • Don’t coast on novelty. People are no longer shocked. They want more experience.
  • Explore both PC and mobile. Mobile systems are getting closer to closer to PC.
  • Be more broad on genre. People are curious.

From Investment Opportunity for VR Gaming Studios:

  • In VR/AR, a lot of creative works come from women. However, lots of funding are not going to them.
  • Investors look at company’s portfolio before investing. They look at the leadership of the company, what they’re thinking about the future, whether they have longetivity, have enough money to survive 18-24 months, also if they’re willing to learn and grow.
  • VCs currently avoid investing on hardware, since they take a while to tweak. They like to invest in market trend, not in long-term R&D projects. They invest in tools & contents. They’re big on location-based entertainment.

For the game jam, the theme is “it’s not a scam”.  I’m working in a team of 5 people (2 programmers, one audio person, one 3D artist.)  We are working on VR experience of a buyer’s remorse, when the user bought their dream space ship, only to find that things break along the way. The development process is interesting. We have one person with Oculus Rift + touch, and two people with HTC Vive, all working on the same project at the same time.  We are using mainly SteamVR and Newton VR for physics.  If anyone is interested, you can download our project at https://github.com/chinnie/SpaceLemonVR .   Currently the main scene is integrated for HTC Vive build, but the unity project main scene can be run on both HTC Vive and Oculus Rift + Touch.

Here is video from the presentation:

Plan for next week:
– sleep in and relaxing on Monday.
– create art assets for Enliven VR

See you next week!


Setting up for Samsung GearVR

In the past, I have only worked with PC VR/AR devices such as HTC Vive and Microsoft Hololens. Mobile VR development is totally new for me. I attempted to create Google Cardboard experience last year during Ludum Dare but after having too slow of a progress, I dumped the idea.

To prepare for Oculus launchpad bootcamp, I actually went through some tutorials from Udemy Make Mobile VR Games in Unity with C# for Google Cardboard one week before the event. I only had access to Google Cardboard and Daydream at that time, but it gave me some idea how to build and deploy VR apps from Unity onto the phone.

Then yesterday, as I fought nasty sore throat/cold after the bootcamp trip, I managed to successfully built and deploy simple test scene to the new Samsung S7 phone and the GearVR, yay.

I work on Windows 10 system, so here were the steps for my setup:

Unity:

  • Install the latest Unity. At the moment, I have Unity5.6.1f1, which is the latest release version.

Android development software:

  • Install Java Development Kit (JDK) Most people probably already have Java installed on their machine, but not JDK. Most have JDE, which is Java Development Environment. Make sure to find and install JDK as well, or else Unity will complain when building to android phone. Also, make note of the JDK installation path directory. You will need this later in Unity.
  • Install Android Studio Development Bundle or Standalone Android SDK Tools. I went with the standalone SDK tools, since I will be building mostly in Unity. It doesn’t harm you to install Android Studio as well. I just don’t have too much free space on my hard drive, so I go with minimal requirement. If you do get Android Studio, make sure you download the version that has the SDK or else there’s no point. To download just the SDK, scroll down all the way to the bottom to see different software options. Make note of the Android installation path directory. You will need this later in Unity and when identifying phone device ID. Also, make sure to run the Android SDK Manager and install the corresponding tools and API version.

 

Oculus development software:

  • Install Oculus Utilities for Unity 5. I downloaded OVR Unity utilities version 1.15.0, since I don’t feel like writing everything from scratch.
  • Optional: download and install the Oculus Sample Framework for Unity 5 Project. Do be careful when importing this package though. Make sure not to overwrite the latest Oculus Utilities files, or else you will get tons of Unity errors, which will prevent you from even running any scene.

We’re almost there! To be able to deploy to our shiny Samsung S7 phone, we need to identify and get its device ID, then create an Oculus signature file. Here are what to do:

  • Make sure your phone has developer mode enabled. To to this, go to your phone Settings -> System -> About device -> tap on Build Number 7 times. Just keep tapping until it finally says you’re a developer. Then go back to System -> Developer Options and turn it on. I also turn on Stay awake, USB debugging, and Verify apps via USB. Not sure what the last part does, but it sounds safer. Better safe than sushi.. err, sorry.
  • connect your phone to your working PC though USB cable
  • bring out command prompt window  (hit search icon on desktop, type cmd and enter)

  • go to android sdk installation folder. (Use cd.. to go up a folder, cd foldername to go in a folder. My installation is at c:\Program Files (x86)\Android\)
  • Go into android-sdk\platform-tools\ folder (my full path is c:\Program Files (x86)\Android\android-sdk\platform-tools\)
  • type ‘adb devices’ (without the ‘ ‘). This will give you the device ID

Most of the prep work is down. Now in Unity, do these steps:

  • Go to Edit -> Preferences -> External Tools
  • Copy and paste the path to Android SDK tool (mine is C:/Program Files (x86)/Android/android-sdk )
  • Copy and paste the path to Java JDK tool (mine is C:\Program Files\Java\jdk1.8.0_92 )

 

  • Go to File -> Build Settings -> Player Settings
    • Under Other Settings, enable Virtual Reality Supported. Use Oculus Virtual Reality SDK
  • If you’re using spatial audio, go to Edit -> Project Settings -> Audio. On Spatializer Plugin, pick Oculus Spatializer
  • Remember the signature file you created? Place a copy of it under Project/Assets/Plugins/Android/assets/

 

Now create your test scene

  • remove current Main Camera. Drag and drop OVRCameraRig prefab from OVR -> Prefabs folder (part of the Oculus Utilities package. Import this if you haven’t done so.)
  • Create a new 3D object -> Cube, place it 2-3 unit in front of the camera.
  • Save this scene as ‘test’
  • Go to File -> Build Settings, hit Add Open Scenes button to add the scene to the list
  • Still in Build Settings, under Platform, select Android and hit Switch Platform. Let Unity does its thing for a minute or so.
  • Once Unity finished, and your phone is still connected, hit Build And Run. You will be prompted to name and save the apk file. I usually create a new folder called Build and save the apk there.

  • Once Unity finish building, the phone will load the file right away. Just place it in GearVR and give it a test.

Hopefully this guide helps. See you next time.