Our client, a renowned social media company and product developer, hired us to work on an environmental and human data collection study. The focus of this study was on hand gesture recognition on head-mounted devices (Head mounted virtual reality) from demographically diverse participants in multiple environment types. The work done on this project would contribute to the development of the next generation of head mounted virtual reality VR/AR/MR devices. This was one of the most complex projects the Q Analysts Ground Truth Data Services team had ever planned and executed. This case study demonstrates Q Analysts’ ability to scope a complicated project, expand quickly, and adapt as a project evolves.

Challenge

We were tasked with creating, managing, and operating all aspects of the project. The client provided the training, the environmental specifications, and the scripts for the gestures, but the rest was up to Q Analysts. The project had specific and highly demanding environmental and participant requirements: we had to set up five environment configurations in five dedicated rooms. Environments were modified between each participant (e.g., changing throw pillows, bedsheets, dining ware, etc.). In addition, entire room furniture layouts were redone at the end of each day. Each week, every room layout had to be redesigned and restaged with complete furniture change-outs. On top of that, each week the room walls were repainted or retextured for different looks, and the flooring had to be redone as well, with different selections including vinyl, carpet, hardwood, etc. We had to manage and navigate through various logistical complexities, which were made more complex with COVID-19 in the picture.

Solution

This was a project that required processes up to sixteen hours a day, seven days a week. To accomplish this, we established two teams. One team focused on execution and the other team focused on environments. Three phases were required to conduct and complete this project, each with its own headset, for a total of thirteen weeks (ten weeks of execution and three weeks of training).

Each participant started the capture with a 3dMD hand scan and was then moved to one of five staged rooms. After each 3dMD hand scan, a technical moderator performed annotations of each of the hand gestures/positions from each of the participants (ten per day). During phase three of the project, an extra step was added that required a 30-minute headset calibration – to accommodate this required readjustment of staffing resources.

Using Head mounted virtual reality, the participant was escorted to the capture room, the moderator instructed them on the basics of the capture and what to expect. The moderator then fit the participant with a head-mounted device, reviewed the series of gestures, and began execution. The first segment included six to seven one-minute captures. Once the participant completed that segment, the lighting would be changed, and the moderator would repeat the process two more times. For the second task, the participant was re-positioned, and the segments would repeat three additional times for a total of six segments. Each capture lasted an average of three hours.

After each capture, the staging team would perform minor changes in the rooms (e.g., rearrange bedspreads, throw pillows, dining room sets, small lamps, small decorations, etc.) before the second capture of the day. Once the second participant finished, the staging team completely rearranged all the furniture and contents of the rooms to get ready for the next day’s participants. This was done for all five rooms each day. Once a week, furniture was completely removed, walls re-painted, and rooms re-staged.

Documentation of the environments was extremely important to this project. The staging team was required to take photos from multiple angles of every room with each room layout change and then submit the photos to the client. In addition to the daily photos, all staging plans were required to be submitted for approval before implementation.

Results

We successfully completed this project and delivered high-quality data to the client. We gathered a large number of demographic diversities, evenly distributed and mutually exclusive. The results had a mere ~9% over-capture rate enabled by the well-planned recruitment of our participant pool. These balanced percentages were maintained through each of the phases. The data will enable our client to innovate and develop the next generation of VR/AR/MR headsets ready to use for real-world applications.