The AWARE project aims to augment human capabilities through assistive AI that is aware of the human user’s cognition and tasks.

AWARE – Human Augmentation through Aware Extended Reality

We decode the various signals obtained from wearable sensors, headsets, and cameras, yielding estimates of the user’s biological state, cognitive state, and the surrounding context.

Augmenting humans with extended reality (XR) systems gives them improved situational awareness, new information and abilities and also enables co-operation through remote presence. Machine learning (ML) based context understanding and human state estimation enable an artificial intelligence (AI) to intervene in stressful situations and ensure optimal performance by adapting the multimodal XR capabilities. Environmental sensing will be implemented by a combination of sensors. The cognitive state of the human will be estimated by wearable biosensors and eye-trackers in conjunction with XR devices.

The results of the project will enable optimal use of XR systems for co-operation of humans and AI technologies. We will measure the cognitive responses, task performance, and user experience of humans interacting with this AI-powered XR technology that combines cognitive state modelling, detection, personalization and prediction with task modelling, context awareness and HCI adaptation.