Modelling for an Augmented Reality environment meant I had to keep two important things in mind throughout the process.

Frame Rate

Firstly understanding the importance of frame rate in our game and keeping it as high as possible. With the tech we are using already being so demanding, means we don't have much room for fancy graphics. The original model of the ACC has a somewhat lengthy load time on the Hololens so I made my first goal to lower the vert count on all components but keeping as much detail as possible. I was also able to add many details through texture maps eliminating many high vert areas on the model. Another step I took to improve performance was to remove the need for lighting in our scene, I done this by baking all shading into the texture maps.

Graphics

Second challenge was creating assets that are visually pleasing when viewed inside the Hololens. How graphics look on a monitor can be very different to when viewed inside the Hololens. This is due to the similar effect from projectors unable to project the colour black through light. To prevent this issue I added subtle colour tints to many areas within the textures, some suggest just lightening the the colours but I found it gave a bland look to the models. I mostly used the three additive primary colors(red,blue,green) when adding tints as they showed the most improvement.

Voice Controls - Lexicon Issues

One of the biggest issues I've encountered in this project so far has been to do with pronunciation issues with voice commands. Certain words are not being picked up by the GrammerRecognizer while other, similar words and phrases are working perfectly fine. Some things I have found that help is defining the phonetic spelling of certain words in the srgs.xml file. An example of this would be trying to access the ABC component. Abbreviations do not work very well  and even defining the phrase to be recognized as "a b c" does not help. 

To solve this particular example I've changed the phrase to be recognized to be "ay bee sea". This phonetically matches what a person would say and can be recognized. There are many other examples to be solved and it is a slow process as testing the commands requires a new build being pushed to the HoloLens or the emulator.

Recently

• Redesign Gus Character 
• Redesigned Gus textures 

Process

Redesigning character model 

After looking at G.U.S in our alpha release I spoke with some of the members as I was not happy with how he was looking in the HoloLense. It is extremely difficult trying to get the hologram effect looking presentable in the HoloLens especially when I didn't have direct access to the HoloLense. The character model that I developed looked very presentable in the unity scene on the desktop but when it was built to the HoloLens the lighting of the hologram completely changed. I attempted to try different methods and shades, but none of them looked good in the HoloLense.

I don't have time to start this character from scratch so I decided to re-gig him from what I have. Simply removing the hologram and attaching G.U.S' face to the orb.  I spent some time trying different things with this until I was finally happy with Gus. The first thing I did once I had the model finalized is sent it to Arnas to build it to the HoloLene for testing.

Re-design textures

The prevision textures where a little unclear to see in unity. Outline of new textures:

• G.U.S' eyes are now blue with a gradient and black lining.
• Shell is metallic dark brown with the ABB logo attached to the rear in black
• Unity shader attached the front screen of G.U.S 

Process

Write additional GUS dialogue

This week I wrote the majority of the remaining dialogue needed for GUS. I wrote it systematically by looking at the systems that lacked GUS interactions. These systems were chiefly the Voice Commands script and feedback system. 

Adapt AWS Polly to use a .CSV instead of .txt

I encountered a problem with the way Polly had previously been implemented. File names were previously set by getting the first 10 chars of the dialogue being generated. This posed a problem when lines with identical initial ten chars were being generated. Only the first file would be created. To rectify this I changed the script to read in a .csv instead. This meant that the first element could be set as the file name while the second element could be passed to Polly and generated.

Generate GUS dialogue 

Utilizing the embedded Amazon Polly I generated 107 lines of dialogue. These lines were for GUS responses to Voice Commands, good and bad feedback from GUS, comments on the user score and game and scenario introductions

Implement Sounds

I implemented a lot of sounds. To implement the GUS responses to voice commands, I generate a random number between one or two and add it on to the string of the sound name. It will then play one of two possible responses. I used a similar technique when implementing other sounds in the game. Good and Bad GUS feedback is implemented the same way. When the user makes a right or wrong move an error or ding sound will play respectively.  When the game starts GUS now says an introduction. He also has an introduction when the scenario starts. If the user tries to work on a powered component a shock sound will play.

ABB update email 2

I looked at everyone SCRUM tasks for this sprint and made a list of systems worked on for this sprint. I then sent this list to the rest of the team to ensure it was accurate and that all work was accounted for. I made adjustments based on their feedback and then sent the update to ABB.

Spark sound

I created a Spark sound for when the player tries to work with a powered component. To do this I sourced three sounds and cut/layered them to create the Spark sound I wanted.

Exploratory Testing

This week I did some exploratory testing after we could not build to HoloLens the last sprint. I found three bugs relating to the ACC. When trying to replace the ABC while wires were still attached the game would crash. When the power switch was flicked the entire ACC would disappear. After testing voice commands with GUS, I tried to reload a different Scenario. The game got stuck returning to select scenario screen. 

I tested voice commands and GUS responses. All What Is, Get Another, Destroy and xRay commands were not working. The only two components being recognized were the ABC and the DAQ.

Issues:

ABC Bug: There was a bug in which the ABCs were teleported to the rear of the ACC unit when the game started. This made it very difficult to test the implementation of sounds and slowed me down quite a bit. Arnas has since solved this issue.

Building to HoloLens on my home machine: When trying to build the HoloLens on my home computer I encountered an issue. I was unable to do so.

ABB Meeting

We had previously organized a meeting with ABB in order to get some user testing for our game. This was key as the software had progressed significantly since the last time we met. While the user testing was ongoing we recorded the conversation in case we needed to go back to the key points that were raised. The first thing to note was that the game was very buggy and prone to crashing, I myself had not experienced this in my testing. This highlights the importance of getting someone other than yourself to test your software. Secondly, while Brian was able to progress through the puzzle with relative ease, another member of the ABB team had an extremely difficult time getting through the puzzle as he was not an engineer. We quickly realized that this meant that only those with a certain level of baseline knowledge would be able to play the game. Ideally, anyone should be able to pick up the HoloLens, launch the game and then develop some engineering skills. After the user testing, we recorded Brian saying a few words about the applications of Augmented Reality in industry, which we will use in our documentary/trailer

Tweak Implementation

As previously mentioned, one thing that we took from our meeting with ABB was that a player with no training will have almost no idea how to play the game. Before each step is processed by the event system, I implemented functionality that first checks if the step was correct or not. If the step was incorrect, it is not completed and a GUS sound is played which tells the player which step he should do instead. This ensures that a player with no knowledge will be able to complete the puzzle and is actually a better system overall as the player is immediately told what the correct procedure is. Points are still deducted for incorrect answers, and this also solves an issue where the player could get lost in the puzzle and not know which step he needs to complete in order to finish the game.

This week I worked on getting the test results into a nicer format and summarize what we needed from our user testing. I also worked on the roadmap side of our group portfolio. 

Test Results

I gathered our results from last weeks user testing session in ABB and displayed them in a more functional way. I calculated out our SUS score into excel sheets and gave a general breakdown of what it means. We had a score of 60 which is just on the point of being a usable application. I expressed my thoughts on potential bias and how the SUS score is still only a general guideline that people follow. I also re-hashed what difficulties came about. I focused on specific areas where there were issues and noted all bugs people found when playing. These were all wrote out in the testing document set up between me and Kieran Keegan.

Mahara Roadmap

I attempted work on the Mahara road-map. I began thinking of a potentially viable way of constructing the road-map in Mahara using the knowledge from previous pages. I spent quite some time attempting to get some visual resources for the page and attempting CSS animations on parts that would need it. I had some difficulty getting this right as Mahara is tricky to develop these kinds of web designs in. I ended up leaving this as there were a number of issues I could not sort out from Mahara. I made the plans to then develop a poster version that could just be a large image to use instead. It would have been nice to have the interactivity and more responsiveness however I found it to be non-viable in Mahara.

Issues

The issue this week was on developing the road-map. I spent quite a number of hours attempting this before eventually summing up that it was not possible in the time we had.