Week 12 – Post PAX and Oculus Launchpad Submission

This week is the final week for Oculus Launchpad program. For those who has gone through game jam and hackathons, you know what this means…  the most productive time!

On Monday, I stayed at PAX until it ended. I took one day on Tuesday to recover from PAX Dev and PAX.

Unfortunately I did not have much time to demo my latest build there, since there were too many things going on. I did sat in the AFK room for about an hour, to observe how they run things there. I noticed the room was very quiet. There were people going in and out, but most of them sat down and either take a break, or talked to each other in very soft voice. It was a peaceful place. There were several tables inside with chairs around them. On each tables, there were several children and adults coloring books, crayons and color pencils. There were also flyers about the organizers (TakeThis.org), information about mental health and where to seek help. They even provide professional helps for those who need it during the event.

I flipped through several coloring books, and spent a lot of time going through one book called “Color Me Calm” by Lacy Mucklow. This book was designed for adults who needed to relax, and I took a lot of notes while I was in the room, which I listed below:

Images used to evoke more relaxing responses in people:

  • mandalas – for concentration, meditation & calming
    • sanskrit word for circle or round, even container for secret space
    • in book: circular, repeatable geometric pattern, round or curvy in nature, not jagged
  • nature aspects and patterns – inspirational calming
    • wooded scenes – quiet scenes of forest area evoke calm feeling
    • animal patterns – fish, nautilus, etc
    • natural pattern – snowflakes, ammonites, trees, shells, leaves, flowers
  • geometric pattern – regular, symmetrical pattern can be meditative
    • math basis, symmetrical design = balanced, evoke order, calmness and focus
    • asymmetrical patterns also convey the sense of balance because of exponential and logical pattern they create
  • water themes – envisioning steady or rhythmic flows and picturesque scenery can be soothing
    • water scenes are relaxing for many people, especially photographs and artwork of beaches, ocean
    • music with ocean waves in the background has a calming effect. Some says it reminds them of heartbeat sounds
    • environments of still water such as lakes, ponds, etc give soothing effect similar to white noise
    • water scenes are often associated with vacation, relaxation and fun times

With all these information, I will have to change the other rooms design for Enliven VR.

Back to project. For the final submission, I tried to cram as many polish as I could. Changes for this week:

Per feedback I got from a couple Oculus Launchpad participant and some friends, I decided to remove the keyboard. Most people expected to be able to change the background music by interacting with the keyboard. It was also hard to see from below, and felt unnatural.

Added butterfly behavior and modify existing wing material. Previously, when user gazed at animated creatures, my script enabled and disabled their animation property in Unity, which caused them to animate or freeze in the middle of their movement. It did not feel satisfying, and one feedback I got even expressed concern about ‘harming’ the butterflies. So this week I created a new script that made the butterfly hovering around when we gaze at them. The starting positioned seemed a bit jittery, but once it moved, it looked natural. I also set the butterfly to always animate, so they either float or hovering this time. For the wing material, it was originally very transparent, with a bit black outlines. Based on optimization documents I read, transparency really affects performance, and it is generally better to have less transparent areas. So I updated the butterfly texture to reduce the transparency and made it more colorful as well. As a result, they are easier to spot this time.

Added pets behavior. Similar to the butterfly, previously each pet started with disabled animation, and as user gaze at them, they started to move. I used the generic animator controller too, so their movements were not the greatest. Those who tested my app could tell that although the red panda was very energetic, the kitten only blinked and the husky pup walked in place unnaturally. This week I spent some time to create custom animator controller for each animal. Each pet now has four to five different animation cycles, and as user gaze at it, the pet movement changed randomly. The cat starts at idle, then it can meow, jumped a bit, or do a high jump. The pup starts at idle (sitting on its butt), then laying down, scratching its head, or sleeping. The red panda also has different movement per gaze, instead of one giant set of movements like before. These behavior will need to be tweaked more in the future to give even more personality and natural feeling.

Updated terrain geometry. Previously, I have had no luck in building Gear VR application when using Unity terrain tool. My builds tend to not running on the phone during testing. I have tried smaller terrain, with like 10 x 10 and still facing the same issue. The terrain geometry also cannot be scaled using transform properties. So in the previous builds, I just use a quad as a terrain and colored it green. This time around, I created a bumpy 10×10 planes in 3ds Max, brought the mesh to Unity, applied material and it worked like a charm. No issue in building, no performance issue during run time, and it looked a lot better than the flat quad.

Added temporary art for pillows and blanket. One of my tester was confused about the bed, since it is round, and not a standard rectangle-shaped bed. For a clear visual, I added pillows and blanket, but since time was limited, I did not have much time to implement any textures. Originally I wanted to create a Turkish comforter and bedding set, after my conversation with a fellow Launchpadder with Turkish background who mentioned about the dying tradition of handmade comforter. This will definitely be implemented in the future.

Fixed spotting and weird shadows on light-baking. After I started playing around with light baking in Unity, some assets, like the gazebo floor, pet tower, and the new added blanket had mold-like spotting or hatched lines. I was not sure what causing it and had been ignoring it for a while. Near the end of the week, one of my client started getting his hands wet lighting for his game as well, and he saw similar issue. He asked me about it, and I told him some lighting setting I have tried, like turning off shadows, etc. Also, out of curiosity, I looked up this behavior on google, and found out that it was caused by overlapping UV in the model. I updated the blanket model, fixed its UV overlap, re-baked the lighting again and voila, the spots were gone!  I shared my finding with my client, and proceed in updating the rest of problematic geometry. My environment looked a lot cleaner afterward.

Due to the natural disaster happening in Florida and Texas, Oculus Launchpad submission got extended from Sunday night to Tuesday at noon. As much as I tried to get much done, on Saturday I started to feel sore throat, and by Sunday it proceed to sinus infection. My husband had been down with flu since the end of PAX, and I was miraculously unaffected. However, the bug in the air finally hit me. So on Saturday, I finished writing the proposal, budget list. I spent half a day on Sunday to set up a wordpress site for Enliven VR main page, but I couldn’t get any style to work to my liking, so I scrapped it and just built it from scratch, using handy dandy notepad++ and my outdated knowledge of HTML and CSS. I spent Sunday night building the latest version of the app, and as previously having issue with jarsigner.

After taking some medicines this morning, I proceed to rebuild the app, resigned, and after a couple failed attempts, I noticed typo on the application path during jarsigning session. Once I use the correct path, it worked!  Again, based on my experience from game jam and hackathon, when you have a working build very close to submission, might as well submit it before proceeding into making a broken build in the next few minutes. I did the submission for both builds and documentations, and that’s it for this week.

Time to fight the PAX pox now. Next weekend will be the anticipated Seattle VR Hackathon, so I will try my best to rest before then. I might also take a bit of break from Enliven VR development to get a clear vision and reconsider design changes. So, see you in a week or two~

Week 10 Update

This past week has been very productive. I have created and placed more objects in the scene for more interaction, such as a musical keyboard that plays short sound clip as player look at it and some animated butterflies. For static objects, I created an oriental lantern for both hanging lights and standing light by the bed, some outdoor stone light to give more illumination at night, and simple star system made of quads and emissive material that glow in the dark and disappear during day time. For the laying down experience, I noticed during play test that it was hard to interact with the pets, so they needed to be propped up higher. To make it look natural, I created a simple pet tower for them to perch on.

For mechanics, I added a look-at-user script to rotate pets. It is simple but powerful, since it enhances the presence of user.

I also looked more into optimizations earlier in the week, before I added more objects into the scene. I was hitting a wall for a while, since no matter what I did, I can only get around 25-30 fps. Once in a blue moon it would shoot to 45, but it wasn’t a consistent experience. Fortunately on Thursday night, two local OLP members organized a Unity Help session night, and they invited someone from Unity. There were about 5 of us OLPer showed up that night and we talked about our progress. The Unity lady showed up for an hour and answered as many questions we have for her. I brought the latest unity build on laptop, Gear VR & Samsung phone, but I forgot to bring USB cable to connect the phone to the laptop, stupid me.

When it was my turn to ask questions, the Unity lady sat with me and looked over Unity profiler to figure out why my frame rate couldn’t go over 30. The main thing she noticed was my day/night system, which use dynamic lighting. She changed a few lighting setting, like using culling mask on specific layer so Enlighten (Unity lighting system) does not have to compute light on everything, just the few objects within that layer. She also noticed that although I baked my lighting, I have Realtime Global Illumination activate on my baking setting, so she turned it off. The performance seemed a bit better on laptop, but it was hard to tell unless we tested it on the phone.

The following morning I applied the same changes on my work PC and deployed a build into the S7, and to my surprise, the fps went up to around 45-55. Then I proceed to clean up all the geometry in the scene, by changing their lighting setting (under mesh renderer) to use appropriate probes and shadows. Also, since I use raycasting to identify and interact with certain objects, I read up on optimization on that as well. The raycasting script I use has an option to exclude certain layer, so I put all non-interactable objects in that layer. I also removed any unnecessary colliders from these objects. Doing these changes brought my fps to above 55.

After I get to the good fps, I started adding more objects into the scene, one at a time, then rebake lighting and tested the change on phone every single time. It seemed like a lot of work, but it really help me keep track of the performance. To speed up the process, I did most of the testing using GearVR developer mode, so I don’t have to put and pull the phone off the GearVR hardware all the time. I only use the HMD hardware to test for user comfort.

During the Unity help session, I asked a few people to try my latest build. The feedback was pretty uniform, that they felt pretty comfortable in the experience, but they weren’t sure what to do at first. Based on this feedback, I felt the need to create some sort of tutorial for the demo. Due to time constraint, I decided to create a floating manual with text & picture instructions. I started this module last night using Unity geometry & UI elements, until around 3am in the morning. The module worked pretty well, but unfortunately during testing on the phone, I could only get around 35-45 fps.

Since PAX Dev & PAX West are happening this week, here are my to-do list:
– Submit a stable build for demo by Monday night (8/28)
– Optimize tutorial module to bring fps back to 55+ by Monday night
– Build freeze, no more new features unless it’s emergency
– Play test latest stable build at PAX Dev & PAX West to collect feedback
– Start the documentation process (website, updated proposal, screenshots, etc)
– If time permitted, polish existing features, like pet animation and object placements to ensure user comfort.

Have a great Labor day weekend! See you next week.

Week 9 Update

This week was a bit like a roller coaster for me. There were tons of progress done, a lot of design changes, but there were also tons of crashes, weird unexplained issues, and also that big Unity emergency patch. I lost track of how many builds I had to deploy to the phone each day to test performance and comfort, something around 10-15 probably.

Following the plan, I changed the bedroom design from a closed room into a gazebo-type, to give a more open space feeling. The bed and roof were modeled last week, and I worked on its UV and materials earlier in the week. For the wall panels, I found a Chinese wall screen design that reminded me of something I saw in my grandparent’s house.

Using the same design, I replaced all the room wall with this, and I was really pleased with how it look afterward.

As the next step, I replaced pet placeholder art with real 3D assets. I found three low-poly and really inexpensive asset packages from the Unity store and use them in the scene. The pet spawning script was tweaked to show only one pet at a time, at specific place based on user’s position (laying down, reclined or sitting up).  Once the pets spawned at the right locations, I changed the script to treat these pets as pooled objects, so I don’t need to keep instancing and destroying them at run time.

Afterward, I looked up some information to create day & night cycle. I tried following two different tutorials, and the last one seem to work really well. Since this has a lot to do with lighting, I baked lighting on every single updates. The effect was pretty nice, although it tend to be really dark at night time. To compensate, I added one spotlight from the ceiling, and surrounded player’s camera with light probe. This added a bit of ambient light so the scene was not totally dark during night time.

I also adjusted user camera’s height based on position, as a feedback from the user test during game jam. Unlike Rift or Vive where we can track headset position automatically, we need to fake this on mobile VR. During the previous play test, I only had one position for the camera, which was for the reclining option. For those who test it laying down, they felt like floating, and when sitting down, they felt like sinking into the bed. With this height adjustment, it feels more realistic. The camera is higher when we sit down, a bit lower when reclining, and very close to the bed when we lay down.

Things were nice and peachy at this time. Then, as I tried to add more 3D objects onto the scene, like planes with transparency, terrain, vegetation, things started to go wrong. Unity refused to bake any lighting, while spouting nonsense error like ‘cannot generate lighting file’ and that was it. My scene turned completely dark because of it, which made it hard to test. Then, every single build that got deployed onto the phone just crashed. I noticed that before Unity stubbornly saying no to baking, my light map increased from 2 maps to 27 different maps. That day I just walked away from my work PC and went to Seattle Unity Group meetup, where I met an artist friend who was familiar with Unity lighting. I described my problem to him, he asked a few questions.. and we all agree that I might need to just restarted the PC. We also learned about the security patch update in the middle of the meetup. Then I went home, applied the patch, got an error in the middle of patching, and then Unity spout baking error every single time. Fun times. I ended up staying up til 3 am reinstalling Unity, repatching, and just rolled back my project to the time when things were peachy. Hooray for github and version control!

On Friday, after the office hour, I started learning about optimization. I also pay more attention to the fps. My scene performance was really bad. It was around 25-30 fps on Tuesday, and by Friday it was around 12-25. To check for fps, I use Unity profiler, adb catlog, and this script attached to the camera. I wasn’t sure what to do at this point. Then Gabor posted a review from Lauren’s King Crab game, where he pointed out similar fps issue as mine. I followed his suggested setting, and the fps went up a little bit. I wasn’t able to do single-pass render, since my app just stuck at Unity logo every time I tried that.

On Friday night, I joined the OLP discord channel that Tyrus set up. We talked about optimization, and he mentioned batching and watching for draw calls. Through him, I learned about the Unity statistics option during run time on the editor. Thank you, Tyrus!  I started watching the line that says SetPass calls, combined with using Unity frame debugger  (under Window -> frame debugger). I noticed that all the interactable objects in the scene had very high number, something about rendering, I guess. Then I realized that those objects contain outline script. As I researched more, it seems outline effect usually cause performance issue, as shown in a few situations below.  I removed this effect, and my fps was back to around 25-30.

I spent Saturday learning how to modify Unity shader script and managed to create a few variation of mobile shader, convert existing standard materials into mobile mats. After doing this, I was able to add more objects into the scene. I also added a custom skybox, and had to modify the existing day/light script to control skybox exposure. Then call it a day.

Next week plan:
– work more on optimization
– add more object without crashing the app
– add a little bit interaction aside from the pet
– figure out a way for pets to focus on player’s camera for presence
– learn and submit a build to Oculus store.
– build freeze by end of week, due to PAX dev & PAX west.

See you next week!

Week 8 Update

This week was one of those time when I lost track of days and suddenly it was the end of the week. If it wasn’t for persistent calendar notifications, I would totally missed the office hour on Friday too. Thank you, technology!

I spent two days resting to recover from the conferences and game jam. Then, following advice from the conference, I worked on making real assets for this project. The first room to tackle is the bedroom scene. During the game jam in the previous week, I managed to grab a couple friends to try the early unpolished prototype, and they both like the outer space scene, and dislike the bedroom scene. They like the open space feeling, and bedroom barely had any opening. So I spent a few days researching open space room design as an alternative. Initially I wanted to create a Japanese inspired room, with tatami floor, ceiling, shoji wall panels & doors. Moving ahead, I still want to have some Japanese room elements but combined with modern design for the open space feeling. I’m hoping to get this all done by the end of next week.

Here are a couple model I created for bedroom scenes:

Then after listening in the office hour Q&A, I decided to update Samsung S7 O/S from Marshmallow to Nougat. I went to At&t first, explained my situation to them, but they were not able to help. We tried with inserting their sim card onto the phone and looked for update, but nothing happened. Not only that, the phone suddenly said I couldn’t try to update for another 20 something hours. Then I messaged Caroline, another OLP participant. She mentioned to go to Best Buy, and look for Samsung representative there, since she had better luck going by this method. I followed her suggestion and went to the closest Best Buy, and voila, the Samsung guy just plugged my phone to his laptop, push a few buttons and I have Nougat installed. Thank you Caroline!

Plan for next week:
– finalize bedroom level design with real assets
– add 1-2 animal companion to the scene
– start implementing interaction for space and bedroom scenes

See you next week!

Week 6 Update

I started this week by tinkering pet swapping module, by creating 3 different points in space related to user position (sitting, reclining or laying down flat), placed planes to represent the pets, and wrote scripts to instantiate and destroy objects at run time based on preference. I was a bit worried about performance at this point, but since there would be only one pet at a time in the level, the instancing/destroying method shouldn’t affect performance much.

Afterward, I modify the menu system a bit. For easy access, I want the menu to be available at all times, probably floating somewhere the user can gaze at. However, I don’t want the menu to obstruct the user’s view of the world. After pondering for a while, I use the menu system from my favorite mobile game, Neko Atsume as reference. I like how its menu always available at the top left corner, and it is really easy to use.

Here is a sample of the pet swapping module and the new menu system.

For positional menu, I decided to line the buttons vertically instead of horizontally like before. This way, it is a lot more comfortable for the user in any position to pick the button menu based on his or her current position.

As I test the opening scene in space, I realize this level can be pretty relaxing as well. So instead of just floating in space, I started creating a mock-up space-pod. For the art direction of this project, I try to avoid square, or sharp edge geometry, and create more curvy shapes. So for the space-pod, I made it a transparent sphere.

For the pod seat, I looked up some reference pictures for sci-fi captain seats and found out that most of the designs look really uncomfortable, like the ones from Star Trek. Then I turn my search to pool floats, and they look really comfortable. However, having a pool float inside a bubble feel a bit weird, so I added some structure, like curvy metal beams to the pod. The beams help in creating the feeling of inside a personal space pod, and interestingly enough, I feel safer being inside this room compared to when I was just floating in space.

Below is a video of the space pod experience. I haven’t had the time to implement the space as its own relaxation level.

I let my husband tested this build last night. Unfortunately, due to time, I wasn’t able to create custom materials for the mock up space pod, so I used existing glass material from my previous project, which was a survival horror game, and the glass was cracked. As my husband tried the demo, he felt very uncomfortable in the space area since the cracked glass made him nervous.

For next week, most likely I won’t have much done in term of development, since I will be at DevGAMM, Casual Connect Seattle and also a 48-hours game jam hosted by Seattle Indies. I will definitely bring my latest build and GearVR, and test it to whoever wants to give it a try. We’ll see how it goes.

Here is the video of the latest build. See you next week!


Week 5 Update

My anxiety problem continued from last week, and I figured it came from a really dense scheduling. I took a few step back, looked at current schedule and milestone, and started rearranging priorities to be more reasonable. It is great to have goals, but it’s an issue if we break down before reaching that goal. So in the beginning of the week, I dedicated a couple days away from Unity development and concentrated on paper design. I spent those days to relax myself using the latest Enliven VR build. It actually worked pretty well with the few features available right now. I only had issue when I started analyzing things while using the application instead of just simply relaxing, and triggered the anxiety. I had to keep reminding myself to take off the developer hat while I was using the headset.

In the middle of the week, I experimented with menu range and rotation based on user position. This short video below showed the mock up I had, where user started at reclined position, then selected sitting and laying down positions. The five buttons per row represent user comfort range for horizontal view. I can actually go up to 7 buttons, but the extra two buttons might cause neck discomfort over time. In actual experience, this menu will be hidden until user decide to activate it.

This user position is then carry over as persistent data from one scene to another, using Level manager with DoNotDestroy() script attach to it. Below is a mock up video of the latest build of Enliven VR.  The music is a temporary placeholder, composed by Niki Wonoto, a talented friend from Indonesia. He’s given me permission to use his pieces in my free projects. For Enliven VR, I’m requesting a couple new pieces from him.

Plan for next week:
– modify persistent data to implement instance based on user option
– implement menu hide/show
– start creating art assets

See you next week!

Week 4 Update

This week I started the production phase of this project by creating a series of different levels to mimic the flow of the experience, starting from Main Menu up to lobby selection. I spent some time watching Unity live stream tutorial video for how to create, save and load persistent data, since I will need to implement similar thing to record users preferences, like their favorite animal, things that made them uncomfortable, etc.

During this time, there were some level design changes as a result of rapid prototyping and user testing. The most constrained setting, which was the laying down position, had a very small range of viewing. During several testing on the previous room level, I noticed that it was hard to access the menu placed on wall to the side. I really had to strain my neck to be able to select the menu items and it was really uncomfortable. This needed to change. I also realize the lobby level can be used as a relaxation place as well. Also I started to design around adding downloadable contents (DLC) to the project.

This week was super busy in term of professional meetups in the evening, so I had to cut some development time.

On Tuesday, Unity 2017 was released. After backing up the project, I decided to upgrade. To my surprised, I had almost no issue. Usually when I upgrade to a newer Unity version, a lot of scripts and sometimes prefabs would break. I was expecting tons of errors, but so far there were only a couple issues.

The first one was in the OVROverlay.cs script from Oculus Utilities 1.16.0 beta. The method Cubemap.CreateExternalTexture used 4 variables, but somehow the script used 6 variables instead. Once I fixed that particular line, the error disappeared.

The second issue came from a custom outline shader that I used before the upgrade. Somehow after upgrading, it added dark blue tints to all the outline materials. Since I didn’t know much about writing custom shader from scratch, I had to look for another. I found three different scripts online, tested each of them, and found one that a lot easier to implement than the old one.

In the evening, I went to an Indie Game Developer social meetup, which was held monthly. I met several other VR content creators there.

On Wednesday, I created the different levels, played around with different UI elements and changed design several times to ensure user comfort. Things were good. In the evening, I went to a local Hololens meetup. I wasn’t planning to go, but then I heard they would demonstrate Microsoft new Acer VR HMD there. I glad I went though, since the talks were really interesting. The first speaker, Sean Ong, shared his experience in creating virtual apartment tour for a company in Dubai. The second speaker, Thomas Wester, shared his experience in capturing dancing motion into VR and AR experience in his team project, “Heroes – a Duet in Mixed Reality”, which was created as 2D film, 360 video (available on GearVR) and for the Hololens. I really like this talk, since it was the first time I have seen Hololens not used for business purpose. You can view these talks here.

My impression on Acer HMD… It was very light, even lighter than GearVR + phone. However the demo used a PC with older graphic card, so graphic-wise it was similar to PSVR. In term of room-scale, it felt like Oculus Rift, for seating experience. We can move a little bit, but there was not much room to walk around, unlike HTC Vive. They didn’t have the VR controller yet, so instead we had to use Xbox controller. I personally dislike VR experience using game controller. At the moment I was unimpressed with the HMD. I will have to try this device again with better PC and the hand controllers in the future.

On Thursday, I did some code and assets cleaned up. I then noticed that the new outline shader was acting strange. It would work fine for a bit then it wouldn’t work at all. I wasn’t able to touch more on this since I had to go to Seattle Unity user group meetup and learned about a real-time geo-spatial plugin called Mapbox. Having to drop was I was doing while it was still unresolved actually gave me a lot of anxiety. Those who befriend me of Facebook probably saw me complaining about it. Thankfully the content of the meetup talk was really interesting and I also got to meet some old friends that I haven’t met in a while. My anxiety was reduced a lot afterward.

The next day, instead of jumping straight into figuring out the issue, I started the day with me-time, taking extra time with hot shower, made delicious breakfast (usually I forgot to eat breakfast) and organized the house a bit so it didn’t look like a typhoon just passed through. When working solo, it is really easy to get burned out, and I noticed the anxiety might be the first sign. So after I felt more relaxed, I opened up the project and tried to figure out what causing the shader issue. After several testing, I noticed that during run-time, there was extra camera under OVRCameraRig. In my project, I modified CenterEyeAnchor game object by adding more children for UI and gaze interaction and their supporting scripts.

During run-time, usually CenterEyeAnchor was placed under LeftEyeAnchor automatically. But during my testing, my custom CenterEyeAnchor was placed under LeftEyeAnchor as usual, but then extra CenterEyeAnchor sub-object was created with a camera attached to it and it prevented the new shader from displaying correctly.

After I modified the OVRCameraRig.cs script and disable this extra camera, everything worked again, hooray. Let’s hope this won’t create a new wonky behavior in the future.

In the evening, I went to an event hosted by TPCast, which was a device that transform HTC Vive into a wireless HMD. I was skeptic about it before, thinking it would have latency issue, and the battery pack would be uncomfortable. However I had a good experience with it. It didn’t have any latency, and I forgot about the battery pack. It felt a lot more comfortable than having the long heavy cable, that’s for sure.

[Now, for a bit of rambling.]
This week I met a couple people that made me think more about VR industry and where we are heading. Both actually happened before and during TPCast event.

We arrived about 45 mins early for TPCast. The lobby had some nice sofa, so I let my husband tried the latest build of EnlivenVR while killing some time. We also met with another 2017 OLP member and chatted for a bit. Then an older gentleman approached us. He looked somewhat disturbed. He saw my husband using GearVR a while back, and wanted to share his concern. The gentleman was a senior composer working in movie industry, and had some people told him to look into VR. This event was his first time going to any VR event, and he noticed all the demo were on ‘violent’ games. Then as he talked to one of the organizers, he was told that that was what VR all about, and that really upset him. I was really surprised, since I have seen many interesting projects, games or non-games ones. I told him there were a lot more than violence in VR, and that I was in a middle of making a relaxation VR experience. He seemed happy to hear that. However he seemed disinterested from looking more into VR based on this first experience, which made me pretty sad. The gentleman left before the event even started.

As we made our way to the event room, one of the organizers was curious about what happened. We explained to him, and he seemed shocked as well. It turned out they were showing Space Pirate Trainer, and a bow/arrow game, which to most gamers were considered non-violent. I was expecting to see something like Raw Data, Arizona Sunshine or other zombie survival shooter games instead. The organizers then tried to catch up to the gentleman to talk to him, but he was long gone.

At the event, there were a total of 6 ladies: four attendees and two organizers. I chatted with these ladies. Like most VR events, the attendees were mostly men. One of the lady, just like me, has been working in tech for a while so she was fine. But another lady, who was there with her mom, was very new to VR. Just like the gentleman from before, this was also her first VR event. She was a student from business school, curious about this new technology but felt really intimidated for being a minority. We talked for a while, and I shared my experience that although it were very common to be minority in this kind of tech events, the ladies in Seattle and all over the world are trying to make VR/AR industry more inclusive to women and other minorities. She seemed relieved to hear that, and interested to come to more local meetups.

When it was my turn to try the device, I asked the organizer if I could try different app, a non-game one. They had Tilt Brush installed, so I went with that. I had a good experience being wireless, able to move around and draw from different angles without the need to teleport around or worry about stepping on tangled cable. When my turn was over, the two female staffs came to me, shared how they never try Tilt Brush before and now they were really interested of what else VR were capable of aside from gaming.

In the end of the day, I was left pondering. As a gamer, am I desensitized to violent contents? I don’t feel disturbed for shooting zombies or slashing monsters. For me, they’re no different than the fruits we slice in Fruit Ninja, just some objects to interact with. But for those who are not familiar, do we look like violent people for enjoying these kind of games? As content creators, what considerations should I put when creating contents, to make people like that gentleman not to stay away from VR?
[End of rambling]

Back to the project talk. To do list for next week:
– Create and test saving/loading custom data.
– Use custom data to drive object generation in room and garden level.
– New menu design that won’t hurt my neck to much.

See you next week! And don’t forget to take a break and treat yourself once in a while. It really helps.

Week 3 Update

I did a few more testing for user view range early in the week to continue gathering more data. Then based on these data, I created white box level design of the bedroom, using Unity basic 3D geometry.

In the beginning, I used real world measurement for the bed and the room size, but somehow it felt really small. Afterward I tweaked the dimension more to make the room more comfortable. Once I found something that feel right, I added more objects, applied flat materials and set up early lighting.

Plan for next week:
– add gaze interaction modules
– create more white box for other levels
– prototype and test game play flow
– look into save setting system
– if time permitted, start working on 3D assets

See you next week!


Week 2 Update

Earlier this week, I managed to figure out several scripts for gaze-click interactions, such as to change scene, hide/show objects and turn animation on and off. This is a big victory for me, since this interaction is the main mechanics for my project. Now the fun part begins! Below is a capture from Unity, with hide/show object on the left, play animation kitten on the right, and scene changer cube on the center.


Art direction:
In the middle of the week, I spent one whole day collecting reference pictures for art direction. I like the art style of old Disney movies like Pinocchio, Dumbo, etc as well as Studio Ghibli movies. For the demo submission, I decided to create two different levels:

1. Bedroom, for starting level or lobby, where user can share their likes and dislikes by selecting items around them. I find this level to be super important, as everyone has different preferences, especially when it comes to dislikes. I want the users to be as comfortable as they can in this experience, and hopefully we can identify things that trigger their discomfort before sending them into the next level.

2. Secret garden or hidden spot on a hill. This will be an example of the area where user can spend 5 to 15 minutes trying to relax, surrounded by their favorite animal companions. There will be interactive objects as well to keep the user entertained if they decide to be more proactive instead of just being passive. These interactive objects will be populated in the level based on the the user’s preference. For example, if normally we have butterflies in this level, but the user were somehow afraid of them, then we could disable the butterflies and replace them with birds.

On the reference pictures above, I also listed secret cave / hidden beach, which I would like to implement if I have extra time before demo submission. Right now this level and about 3-4 others are listed under implement later list, when it comes to the final product.

By the end of the week, I started doing some user experience testing to figure out comfortable viewing range. In my design, I want user to have option to run the experience in three different resting positions:
1. laying flat on a bed or sofa, with a pillow under the head
2. recline, for example with 2-3 pillows behind the back
3. sitting up straight

For the initial testing, I created an array of interactive blocks in Unity, as shown below. The blocks are about 1x1x1m, and their location range from about 2 m behind the the user to about 6 m in front of them.

Then we try to interact with these cubes, for the three different resting positions. From several sessions, we found out that position 1 (laying down flat) has the smallest range, while position 3 (sitting up straight) has the largest range.


Next week to do list:
1) More testing – based on this week’s data, try with smaller objects in various distance.
2) Create white-boxed levels based on the gaze data.
3) Test these levels.
4) Start creating custom art assets for each level
5) Implement audio (temp background music, sound effects) within the test levels
6) Figure out how to capture and record VR test sessions.

That’s all the update I have for this week. Happy fourth, everyone. I know everyone’s working hard, but don’t forget to take a break once in a while. :smile:


Week 1 Progress

As a requirement to qualify for Oculus Launch Pad scholarship, each participant is required to have a weekly blog post at Oculus forum. Since it’s not visible for non-participants, I will have a copy of it on this site as well. Here it is for this week:

Hello, I am a 3D environmental artist who also been teaching myself Unity development so I can make my own games and VR/AR experiences. I met so many brilliant and wonderful people at the Oculus Launch Pad (OLP) boot camp, and learned a lot. There were many great ideas and I’m looking forward to see them into fruition.

About my project:
EnlivenVR (working title) is an interactive relaxation experience for mobile VR platform, where users can take a break from their busy life and escape to virtual safe space, surrounded by their favorite creatures and soothing sounds, while laying on their back in a comfortable place like a bed or a sofa.
By mainly using gaze selection and head movement, users can interact with the environment. Tap and Gear VR controller will be implemented as well to give audience more input options.
I am hoping that after 10-15 minutes of using the app, users feel more relaxed, refreshed and in a better mood to face the rest of their day.

Why this (VR) project:
Gaze-based interaction in mobile VR is something I have been trying to explore. The first VR system I tried to build for was Google cardboard, since I only had access to android phone at that time. I got myself a cheap paper cardboard viewer, and for some reason the clicker did not work with my phone. As a result, I would have to put my finger by the viewer’s nose opening and tapped the phone screen that way. It was really uncomfortable, and I wished I had super power to click with only my gaze.

During a local VR meetup, I had a conversation about this gaze-click interaction with a lady who was really interested in this idea. She has spinal muscular atrophy, a genetic disease affecting part of the nervous system that controls voluntary muscle movement. On some bad days she was unable to use her arms and legs. This type of control would enable her to still interact with the virtual world even as her limbs refused to listen to her brain.

In another VR meetup, there was a conversation about accessibility. One of the problems mentioned was how to accommodate for people who were bedridden. We talked about ways to set up different cameras to make  people laying on a bed feeling like they’re standing up or sitting straight.

As I started thinking of a product when writing pitch document in preparation for OLP Bootcamp, these things came back to me. I was thinking, why don’t we make an experience for users who lay flat on the ground, a bed or a sofa, without using their hands. Instead of creating complex camera systems, why can’t we just make the interaction happening above the users, where their gaze can reach. This way, I can target both people who needs accessibility and those who are just too lazy to move their arms.

Why relaxation experience? I believe mental health is important. In our current political climate, I feel anxious and depressed more often than usual, and I know a lot of people also who have similar issues. As someone who had been through severe depression, sometimes it takes a lot of willpower just to get out of bed in the morning. I am hoping to create a tool for people like me and help finding that extra energy, extra push to face the day. I’ve read articles about successful people who also spend some time in their daily life to relax and meditate before starting their day.

Progress of week 1:
1) Hardware & software setup
The first thing I did with the Samsung phone was enabling developer mode and removing unnecessary bloatware, then install necessary apps that were needed for development. I already have the latest Unity and Android/Java SDKs installed on my work machine from previous projects, so I downloaded Oculus Utilities for Unity and VR examples.

2) Learn how to build simple scene
Before going crazy with building levels and behaviors in Unity, I needed to learn how to deploy GearVR app to the phone first. So I created a simple scene with a cube, a OVR camera and build it. I noticed unlike for Google Cardboard, GearVR app actually try to detect for the hardware before it can be loaded.

3) Playtest and analyze different existing GearVR apps, especially something with similar theme
At first I downloaded several free apps and 360 videos especially the ones mentioned during Launchpad presentations, like Zero Days VR, In the Eyes of the Animals, Face Your Fears, and Strangers with Patrick Watson. Then Oculus summer sale was happening, yay! For experiences, I downloaded Relax VR, Jurassic World: Apatosaurus, Singularity, Land’s End,  A Night Sky and Bait!. I found similar design elements as my project in Relax VR, A Night Sky and Bait!, which is a good thing.

4) Explore Oculus and Unity toolkits
So far I have spent the largest portion of my time on this. Oculus development is new to me. The Oculus Utilities for Unity had very small number of example scenes, and the Oculus Sample Framework for Unity  project ended giving tons of errors that prevent me from even building any scene. After 2 days struggling with the sample framework, I just tossed it out the window and downloaded Unity VR Samples from the asset store. This one didn’t give me errors, and gaze interaction was similar to what I have in mind.

5) Create initial design
To prevent feature creeps during development, I created initial design document, mainly identifying what needs to be in the experience, what can be added during polish time, and what is nice to have for future contents. I left out a lot of the details since this document will most likely change a lot in the next few weeks, and those details will be added as we build and test the prototype.

6) Create deliverable timeline & consider public events for testing
This one goes in hand with the initial design document. To plan for public testing of this product, I start listing when local VR meetups usually happen. In September, one week before Launchpad submission deadline, there will also be PAX Dev and PAX Prime. At PAX, I can probably get some feedback while waiting in line for panels or going to the safe space room. The great thing with mobile VR is I don’t need to lug around laptop or beefy machines to showcase my app. All I need are the phone, the viewer, cleaning supplies and a charger.
Knowing the dates of these event help in planning deliverables. Based on all this, I pretty much need to have working prototype in a month, then spend the rest on time on more testing and polishing.

7) Start prototyping, tackling mechanics first
As a 3D artist, programming will be my greatest challenge. So, instead of concentrating on art, this week I started with tackling mechanics first, to make sure the building blocks are working properly. I use a lot of art placeholders with cubes and other Unity 3D objects. Pretty stuff will be added later.

8) Setup streaming using Chromecast
People say that mobile VR is an isolated experience, but guess what, GearVR is compatible with Google Chromecast. Last night we went to Best Buy and got one, then tested it right away. I was able to stream my session to a TV. The frame rate was pretty choppy, but this will be great for testing later, to watch for user interaction. I will write my setup for streaming in a later blog.

Things to consider during development cycle:
1) Keep it simple. Since I will work on this project solo, it’s important to know my own abilities and not going for crazy ideas. From my experience with hackathons and game jams, if I’m struggling with the same problem over 1-2 days, maybe I should find and try another solution.

2) Why VR. This question keeps popping in my mind after going through the boot camp, and it really affect my design decision.

3) Think of user needs first. Since I’m creating a safe space for people, I want to make sure user comfort is the main priority. For example no jump scare, have a way to exit the app fast if needed, and have a way to remove any trigger or uncomfortable objects from the environment.

4) Test ideas through prototype. Fail fast and fail often.
Sometimes, especially in VR, ideas looks good on paper, but as we implement it in VR, it turned into really bad experience. Many questions can be answered through prototype as well. Don’t know what’s considered close or far? Put several cubes in different location. Put on the HMD and check for yourself.