Mixed Reality Educational Experience
In October 2015, the Oregon Story Board and Clackamas Community College team was one of 5 grant recipients chosen for Microsoft’s HoloLens Academic Research Grant. With this $100K grant and two HoloLens “mixed reality” devices from Microsoft, along with resources from partners at Intel, OSB created a custom classroom for teaching mixed reality content creation.
Using software such as Unity and Maya, I played a key role as principal designer in defining interaction patterns and design strategies for holographic interactive auto transmission that was then integrated into CCC’s Automotive Services Training classroom curriculum. Governor Kate Brown visited OSB and viewed the completed transmission though the HoloLens at the Inclusive Startup Fund media event at Oregon Story Board.
OPB's radio show 'State Of Wonder' included the making of this module as a feature story in an episode entitled 'Virtually Unlimited/Virtual Reality In The Classroom' (28:19).
Challenges:
-
Balancing big ideas and innovative concepts against limited resources and time frame.
-
Conceiving of intuitive interaction patterns within the context of a emergent paradigm/modality (holographic elements), with no clear set of best practices to rely on.
Roles:
-
Principal Ix Designer
-
assist on Product Mgmt.
During the user research phase of this project, it was discovered that educators struggled with the aspect of illustrating abstract concepts to students in relation to the inner workings of a transmission, which is obscured from view by a outer container. As originally conceived, users were able to drill downto a holographic representation of inner components anchored to the actual surface of a transmission, akin to an "x-ray vision" mode. This provided context and enhanced comprehension.
Users could then "pull out" a given component set, such as the planetary gears (as depicted here) and run a desired simulation via the contextual menu. Voice activation also played prominently in accessing a given scenario, as did a nested annotation layer that facilitated the identification process of a given component(s).
During the user research phase of this project, it was discovered that educators struggled with the aspect of illustrating abstract concepts to students in relation to the inner workings of a transmission, which is obscured from view by a outer container. As originally conceived, users were able to drill downto a holographic representation of inner components anchored to the actual surface of a transmission, akin to an "x-ray vision" mode. This provided context and enhanced comprehension.
From 2D to hologram
Once the basic interactions scenarios/patterns were agreed upon and validated in 2d storyboard form and prototypes, transmission components were drum scanned, refined and optimized for performance, and painstakingly reassembled and animated in holographic form. Related grouping of components were color-coded to emphasize their inter-dependence.
(animation by Sonya Neunzert)
screenshot of Intake Processor Tasks Mindmap
Success Metrics
Findings indicated that students who used the HoloLens content gained a stronger understanding of the concept than learning through the traditional training medium of text and lecture. An average of 94.3% of participants indicated that they strongly agreed that the HoloLens transmission matched their existing instructional manual. The instructor also emphasized the correlation between the manual and holographic transmission during his HoloLens-assisted lecture. The instructor was involved in the DMC199 holographic transmission development process and thus was familiar with how to use the hardware and associated content. Based on participant response, an average of 87.1% of students strongly agreed that the instructor used the HoloLens effectively for in-class lecture.*
*source: OSB Academic Research White Paper; author: Anthony Piers
“Traditionally, in my class, [auto transmission] powerflow was taught through power point presentation and hands on with actual transmission components. Powerflow has always been a huge obstacle for students to overcome. The average student would spend a minimum of three weeks mastering this concept and some as much as five weeks. I often found myself demonstrating powerflow to the students more times then I care to count. After the introduction of HoloLens, most all of my students were able to demonstrate first gear powerflow in the first week. In my opinion, HoloLens made a significant difference in the student’s ability to learn powerflow.”
-Rick Lockwood
Clackamas Community College Automotive Service Technology*
