A well-known social media network tasked us with an external egocentric data collection project to gather natural movements and activities from a first-person perspective in unique and changing environments.

Challenge

The Client required assistance with the collection of egocentric ground truth data to train the AI powering next-gen head-mounted MR devices.  

The overall database needed to include both exhaustive local environment and human derived data.  

Solution

Comprehensive Quality Assured managed study into egocentric data.

1020 participants, 35 unique environment configurations, 6783 captures, 47 scripts, and 82 activities.

Results

Successfully captured the required complex ground truth data to drive next-gen VR headsets.  

~9% over-capture rate enabled by the well-planned recruitment of our participant pool. 

Client overview

The Client operates a global social media platform and digital advertising enterprise.  As an early adopter and technology pioneer, the Client has also made forays into emerging technologies with a roadmap that will deliver on future tech products enabled for the virtual world. This case study is one such example.

Complex egocentric data required to facilitate next-gen VR headsets

Our team was tasked to collect data on 100 participants initially, then later adding an additional 900 participants performing a variety of daily tasks while utilizing a standard market AR headset. Our objective was to collect and deliver ground truth data to the Client to develop MR products for everyday use.

We first received a one-page script of activities (i.e. actions and movement scenarios to perform), which then expanded to six pages that included additional descriptions of environments and objects the Client needed, i.e. (Rooms in an ordinary household environment as well as the everyday items that reside in those rooms). We were instructed to obtain about four hours of data capture per participant. Initially, we were given one AR device to test its limits for operation. As the project progressed, we received more instruction and direction about the scripts we needed to build and made sure we had a plan of action on how to ramp-up our work schedule in time to match the short timeline the Client requested for delivery.

Flawless execution of comprehensive egocentric data capture project

We developed a series of scripts to collect the data with different sets of requested egocentric activities. We designed the scripts to be part of an overall story to perform the scenario actions as naturally as possible. Each script had its own narrative sequence, so it could lead to the next expected activity in that sequence to achieve a natural flow and follow each scenario logically.

These scripts covered expanded all the activities requested in the initial one-page sheet. We established a three-person team for each collection location consisting of one lead and two moderators. These locations were set up to collect data at the Client’s campus staged site and two local Airbnb’s.  We staged the Client’s campus location to look like specific rooms in a house, and we restaged those rooms every two weeks. The Airbnb sites were changed each week. We used our internal participant database for recruiting participants with specific demographics.

As the data collection stabilized, the project ramped up four Airbnb locations with two additional data collection teams. We changed Airbnb sites’ setting every two weeks to match the restaging environment of the Clients’ campus. We added more scripts to cover a larger number of egocentric use cases to include office settings.

We began building a framework appropriate enough to handle the increased participant activity, which continued to be developed during execution. As the project continued, the Client requested several different staging configurations that were not in the original scope. The Client requested us to design, develop, build and stage a complete convenience store down to food items, cash register and a slushy machine and a small clothing store environment at their campus location. The Client requested that the convenience store and clothing store environments be accurately staged to scale to be used for future projects. We took on that challenge and designed, procured and produced a realistic site that delivered excellent results.

Key benefits

Our precision project management of this complex undertaking resulted in the Client gaining:

• Effective insight from 1020 participants, 35 unique environment configurations, 6783 captures, 47 scripts, and 82 activities.
• ~9% over-capture rate enabled by the well-planned recruitment of our participant pool.
• A refined script development process, with its focus on narrative creation. It helped participants execute the actions in the egocentric scripts in a manner that matched real-world practice.
• Increased operational time and data volume storage of the AR glasses used for data collection well beyond its regular use capacity. This was derived from our early-focused testing.
• Script format revisions that made tracking activities and objects in each script much easier, and improved clarity.
• A sophisticated and well documented data collection methodology, which was dynamic enough to allow for constant changes and experiments with new technology.

Final project data delivered ahead of schedule, under budget and exceeding the Client’s “Exceptional” expectations.