An Imperative – Developing Standards for Safety and Security in XR Environments

RESEARCH

In partnership with emteq labs

Whilst virtual reality technologies have been available for decades, we are now in a period of rapid growth in virtual, augmented and mixed reality technologies, collectively known as XR.

The acceleration in XR availability and adoption can be attributed to many factors, principally, a reduction in hardware cost, increases the availability of high-speed high-quality connectivity, and most recently, shifts in society brought on by the global pandemic.

As with all emerging technologies, the confluence of new economic and societal norms with new techniques and capabilities gives rise to new opportunities, challenges, and entirely new paradigms.

This paper is based on discussions held during a roundtable conversation between members of the XR Safety Initiative (XRSI) panel, in partnership with emteq labs. The paper examines the opportunities and perils of XR technology and what should be done from a safety and security point of view to maximise the potential for good, whilst mitigating opportunities for harm.

Contents

Executive Summary

Introduction

Unprecedented Growth

The Benefit of Internet Hindsight

Evolution of Personal Data Availability

Safety and Security in XR

Example Opportunities for XR

XR and Digital Divides

Risks – Intimate Data and Profiling

What Kind of Regulation is Most Suited?

What Needs to Be Regulated or Standardised?

XRSI – community-led standards for safety in XR


Executive Summary

The “Link Flight Trainer” was developed in the 1920s to improve the safety of trainee pilots. Grounded in ‘safety’, this kickstarted the development of technologies we now know as “XR”. 

XR is the term used to refer to technologies that merge virtual and real-worlds, commonly known as virtual reality, augmented reality and mixed reality. 

Virtual reality applications offer immersive experiences that captivate and stimulate users, in powerful ways, improving knowledge retention and transferring improved performance and knowledge to the real-world. Augmented reality gives users the ability to interact with virtual objects in the context of the real-world, providing enhanced capabilities or informational awareness. Mixed reality merges the two concepts, allowing participants to physically interact with virtual constructs.

Since the 1920’s, XR technologies have developed significantly. The rapid growth of cheap computing power since the 1990’s moved XR into a new growth phase, and whilst researchers focused on a wide variety of applications, public adoption grew in the entertainment and gaming sectors. In the last five years, as computing power became mobile, hardware costs reduced and high-speed connectivity became virtually ubiquitous, XR technologies experienced another step-change..

Beyond gaming and entertainment, the broader benefits of XR to individuals, enterprises, researchers and national institutions is flourishing. The benefits are huge, but with this growth comes the risk that misuse, and abuse could derail progress and undermine the positive societal benefits.

The benefits of XR can only be capitalised on if:

  • It is safe 
  • It is trusted 
  • It is proven effective 

This paper looks at the potential risks and rewards that await the XR ecosystem. It looks at a mega-trend that could influence the next step-change for the industry and leverages the Internet’s modern history to look for warning signals. 

Taking into consideration the risks and rewards, the paper discusses what can be done to ensure that XR technologies are widely distributed to deliver positive, safe and trusted experiences to as many people as possible. 

Concluding, the paper outlines the roles of regulation, ‘intended use’, certification and community-led approaches to XR safety and security.

Introduction

Whilst virtual reality technologies have been available for decades, we are now in a period of rapid growth in virtual, augmented and mixed reality technologies, collectively known as XR. 

The acceleration in XR availability and adoption can be attributed to many factors, principally, a reduction in hardware cost, increases the availability of high-speed high-quality connectivity, and most recently, societal shifts brought on by the global pandemic.

As with all emerging technologies, the confluence of new economic and societal norms with new techniques and capabilities gives rise to new opportunities, challenges, and entirely new paradigms.

This paper is based on discussions held during a roundtable conversation between members of the XR Safety Initiative (XRSI) panel, in partnership with emteq labs. The paper examines the opportunities and perils of XR technology and what should be done from a safety and security point of view to maximise the potential for good, whilst mitigating opportunities for harm.

Unprecedented Growth

According to market analysts, Fortune Business Insights, the size of the VR market in 2019 was USD 3.10 billion and is forecast to grow to USD 120.5 billion by 2026. This growth reaches across virtually all industries, which have found value in XR across an initial set of harmonised application areas and use cases.

XR Touches Virtually All Sectors
  • Retail
  • Construction
  • Gaming & Entertainment
  • Media
  • Healthcare
  • Education
  • Automotive
  • Aerospace & Defence
  • Industrial & Manufacturing
  • Oil & Gas
  • Travel & Tourism
  • Telecoms and IT
Common Areas of XR Application
  • Training and Simulation
  • Education
  • Recruitment
  • Health and Safety
  • Marketing and Attraction
  • Research and Development
  • Medical and Therapeutic
  • Maintenance
  • Sales & Customer Support
  • Research

The increased accessibility of these technologies over the last five years has created new commercial momentum, which in turn has spurred further cost reductions and inspired innovative applications and concepts, which, in turn are driving the market faster and further.

Emerging from these cycles are novel applications of XR technologies which build momentum for existing use-cases and push known boundaries to create entirely new XR paradigms.

XR and the Convergence of Physical and Virtual Worlds

Leveraging their 2014 acquisition of Oculus, early in 2020 Facebook launched a closed-beta of their VR social network, Horizon. If even a fraction of Facebook’s 2.7 billion users sign up to the new immersive platform, that alone will push VR penetration and adoption further away from the current XR mass-adopters – enterprises.

However, Horizon is only part of a broader sea of change known as the “Metaverse”. The Metaverse can be defined as the sum-total of Internet-connected virtual, augmented, and mixed reality worlds. Offering fully and semi-immersive experiences, an individual can represent themselves as a graphical representation known as an Avatar and interact with other Avatars, in a multitude of other real and virtual environments. Avatars will be controlled directly through human interaction, but also include artificially intelligent entities.

Spanning the full breadth of XR, the Metaverse will blur the distinction between our online and physical worlds even more. The Metaverse will consist of experiences that leverage fully immersive VR technologies as well as enhanced reality applications that overlay information into the physical world. This paradigm shift will leverage augmented reality devices, nascent technologies such as holographic displays, and traditional information systems, including billboards, televisions, shop windows and transport information screens.

XR Will Impact the Lives of All 

Facebook isn’t the only player in this market. The world’s fastest growing game platforms, Fortnite, Minecraft, Animal Crossing and Roblox are all paving the way to the Metaverse, allowing billions of users to create and engage virtually, as well as build their own worlds with their own rules, and their own tradable currencies. 

Whilst these companies might be classified as game platforms, their reach and potential are far greater than the gaming communities. Add to that the acquisitions, investments and statements made by the big Internet companies, the Metaverse, like the physical world, will involve all of us.

In short, the Metaverse will be the merging of the physical and virtual worlds, where an action in one medium will translate to an outcome in another. As we will explore below, these important developments necessitate a discussion of Safety and Security in XR.

The Benefit of Internet Hindsight

Immersive technologies can improve both access to, and outcomes in educational, healthcare, training, and safety situations. In a previous white paper we looked at the positive impact VR and emotional data can have in training and therapeutic scenarios too.

However, with XR technologies becoming increasingly prolific, the opportunity for deliberate and accidental misuse multiplies. The risks to privacy, security and safety and the opportunity for discrimination and bias are derived from three main sources. These are i) the new types of data captured in XR, ii) the level to which participants can become detached from the real world, and iii) the potential long-term psychological impact of using new technology mediums.

In the early days of the Internet, there was a sense of utopia, accompanied by open collaboration, and free dissemination of ideas. We were all going to be “Netizens” with our own personal Blogs, discussing ideas, sharing information, and making the world a better place. The vision was of a decentralised information and collaboration tool that could overcome knowledge inequality and help make the world a fairer, better place.

Whilst the Internet has created growth and opportunities that is perhaps even beyond the wildest imagination of its early pioneers, there are important developments in the modern history of the Internet that we should learn from when thinking of such a powerful, transformative, and personal platform as XR.

Today the Internet’s decentralisation is less than perfect, with big companies and governments controlling key pieces of the infrastructure. It is plagued by misuse and mistrust, derived from the systemic exploitation of a by-product created from the fundamental principles that drove the development of the Internet – data.

The Internet grew so rapidly, across so many borders that the business models and practices developed organically at an incredible pace, with very little regulatory or legal oversight or leverage. On one hand, this freedom to innovate is what fuelled its success, on the other hand it is what continues to hold it back. 

Let’s look at two examples of how a lack of regulation and oversight led to misuse, abuse and the breakdown of trust.

Governments and Corporations Battle to Take Back Control of Their Property

The Internet made it incredibly easy to share data and within a short space of time this was being exploited in the form of copyright infringement of movies and music. Through the late 1990’s and early 2000’s big industries grappled to take back control of their property, implementing digital rights management standards and practices. Yet the reward was so tantalizing for both individuals and malicious entities, that the practice evolved and continued. Governments were then lobbied to act, and many implemented strict laws, even blocking and censoring specific websites within their countries.

However, today anyone with intermediate computer skills and a basic Internet connection can download movies, TV shows and music using technologies such as BitTorrent and Virtual Private Networks (VPNs).

Awakening the Understanding of the Value of Personal Data

One of the core tenets of the Internet has always been “free”. This led to the development of the sophisticated, extremely valuable and turgid Internet advertising market. 

The first online advertising banners were introduced onto the Internet in 1994, and in 2021 the Internet Advertising market is projected to be worth close to USD 400bn, and growing at over 12% year on year

This huge economic growth was built on personal data that was being freely and mostly unwittingly shared by millions of Internet users. As the number of users, applications and technologies grew, the quality, variety, quantity, and value of the data skyrocketed – mostly unbeknownst to the user to which it belonged.

The traditional advertising market is substantial, yet dwarfed by the wealth of companies like Amazon, Google, Facebook, and many others which are also built on the foundations of vast quantities of individual personal data.

In 2018 the world woke up to the value of personal data as the Facebook and Cambridge Analytica scandal came to a head, wiping USD 100bn off Facebook’s valuation. In that same year, the European Union launched its General Data Protection Regulation (GDPR) to help protect the personal information and privacy of individuals on the Internet.

In other words, it took 24 years for the regulation to catch up with the market, and even today GDPR and its international counterparts struggle to fully protect end users from the misuse and abuse of their personal data. 

The two examples above are but just a miniscule representation of the level of misuse currently happening across the Internet. Each breach or scandal erodes trust, reduces the overall value and hinders the bigger purpose. Together the XR ecosystem needs to learn from the Internet’s history and take action to mitigate the harmful impacts of misuse and abuse.

Evolution of Personal Data Availability

The personal data that built today’s Internet giants and led to global regulations primarily, but not exclusively, consists of identifiers such as name, age, sex, physical and email addresses, phone numbers, Internet behaviours, an individual’s inferred likes and dislikes, and similarities with others, which can generate further inferences.

These in themselves are very powerful, and can be used to track, understand, and even predict human behaviours. However, the growth of new technologies such as wearable devices with biometric sensing capabilities open a new world of data use and abuse.

Safety and Security in XR 

There is much to gain from the new data that can be extracted and extrapolated from XR experiences, but also much at stake. Learning from the growth of the Internet, it is imperative that we act to protect individuals whilst not impeding innovation and growth.

During November 2020, emteq labs and XRSI partnered to conduct a roundtable discussion on the growth of XR and the safety and privacy implications of XR experiences.

XRSI and emteq labs’ Safety and Standards in XR Roundtable, November 2020

Participants:

  • Charles Nduka, MD, Co-Founder and Chief Scientific Officer, emteq labs
  • Professor Walter Greenleaf, PhD, Neuroscientist, Virtual Reality, and Digital Health Expert at Stanford University Virtual Human Interaction Lab 
  • Rachel Michelson, Medical XR Advisor, XRSI

Facilitated by Dr. James Sumauroo, HealthTech expert, Forbes contributor, and innovation reviewer for BMJ

The roundtable covered the opportunities, risks, and challenges that the accelerating XR market is creating, summarising on the steps that should be taken to mitigate risk, and “do no harm”.

The sections below outline these discussion areas.

Opportunities

The benefits of XR are broad and deep. 

  • Enterprises are exploring augmented reality (AR) technologies to provide expert support where it’s needed, anywhere in the world, without having the cost or delay of sending specialists to the physical locations. (Example solutions from Deutsche Telekom and Qualcomm)
  • Industries are utilising AR to improve efficiencies, reduce errors and support customers world-wide. (Example solutions from FieldBit)
  • Retailers are exploring Virtual and Augmented Reality applications to get closer to their customers and deliver tailored, memorable experiences – especially in the wake of COVID-19. (Example solutions from Obsess)
  • Healthcare providers are exploring XR techniques to help consult, diagnose, treat and manage both physical and psychological conditions.
  • Organisations of all types have adopted VR for training and development programmes.

In all these use-cases, the potential to gather specific, personal, and biometric data from the user can provide opportunities to make the experience more effective and rewarding for the user and, or the provider.

Virtual Reality has been shown to create high degrees of empathy, to accelerate learning, knowledge assimilation and recollection, and to show increased willingness to apply and transfer from virtual learnings to the physical world.

VR techniques create powerful immersive graphical and audio environments that allow the users to detach from the real-world, creating virtual experiences that stimulate, guide, cure, and educate more effectively than even some of the more traditional approaches. Beyond this, the intimate nature of the technology facilitates detailed biometric data capture that can be used to tailor the experience for individual users.

Developers of VR experiences are exploring the datasets that can be derived from VR headsets, and how these can be used. Today even some of the simplest VR technology set ups can determine:

  • Motion data
  • Gaze and eye tracking
  • Course facial expressions  
  • Usage and interaction 
  • Location and proximity of other systems

More sophisticated devices, such as those leveraging emteq labs’ technology, can detect posture, heart rate, blood pulse volume, subtle facial expressions (using Facial Electromyography), perspiration and skin conductivity, cognitive load, and are able to interpret a wide spectrum of human emotional responses to specific stimulation.

With this level of detailed data it is possible to create experiences that take into consideration what the user is experiencing, and adapt accordingly. It is also possible to capture and measure specific characteristics and biomarkers that make it possible to securely and reliably identify an individual beyond the capabilities of volunteered information, such as name, email address and postcode.

Together, the data that can be derived and inferred from XR experiences can create a digital phenotype, a unique fingerprint that identifies specific users by the observable behaviours and traits.

“Privacy is no longer synonymous with anonymity”

Rachel Michelson, Medical XR Advisor, XRSI

The ability to identify users, their behaviours and even their emotional states so clearly and reliably can be used to provide many positive outcomes, including:

  • Compliance: Ensure specific individuals are completing required activities such as training and therapies.
  • Security and Safety: Ensure only designated users have access to appropriate or specific content, and ensure experiences are aligned with the individual’s need and capabilities.
  • Usability: Adjust content and delivery based on how well the user is engaging with it, helping make the content more accessible and usable for all.
  • Healthcare: Individual assessments, diagnosis, treatment, care and therapeutic development, stress management.
  • Personalised digital health coaching: Create digital projections of the user’s future-self to help them through challenges.

Example Opportunities for XR

Training

VR’s roots are in training and simulation, in fact the earliest implementation of immersive simulation, known as the Link Flight Trainer, was designed in the late 1920s for the purpose of increasing the efficacy and the safety of trainee pilots.

Since then, the application of immersive technology has gone through massive development, yet the core principals remain the same. The intention is to use simulation to reproduce environments and experiences that feel, look and sound real, without the cost or risk of doing it in the real world.
A PWC study showed that VR training experiences can outperform traditional training methods in many aspects, delivering high-performance outcomes at a fraction of the cost, with more convenience and less geographic constraint. Additionally, the inclusion of technologies within the VR experience that can measure the individual user’s response to specific environments and scenes enables the development of detailed analysis and personalisation.

In the safe, simulated world of a VR environment, the application of multi-sensory stimulation helps individual users escape the real-world and have a high degree of belief in what that experience. The psychophysiological effects are profound and have been shown to have measurable, direct impacts on their real-world body, enabling them to easily transfer their experience into the real world, post-training. 

The application of VR in training is diverse, below are some examples. 

  • Crime scene and fire forensic investigations 
  • High risk operational environments, such as offshore oil rigs

and mining

  • High performance training such as Formula 1 racing
  • Hazardous environment, including security services and front-line journalism 
  • Soft skill development, including behavioural awareness and change such as diversity, harassment, cognitive bias, and, 
  • Leadership, resilience and change management 

For more information on the application of VR in training download the emteq labs white paper, “Improving Emotional And Psychological Well-Being In Distributed Digital Times

Rehabilitation, Therapy and Well-being

XR technologies are being increasingly applied to clinical and therapeutic situations, and in our world today with hospitals and healthcare professionals swamped with COVID-19 cases, the ability to safely, securely, and efficiently treat patients away from hospitals, clinics and surgeries has never been more critical.

XR is being used today to treat a plethora of mental health conditions, from phobias, anxiety disorders and depression. Beyond this, the psychophysiological actuation potential is finding application in the treatment of health conditions such as pain management and facial palsy.

There have been decades of studies that have looked at how VR and AR technologies can address a variety of conditions, including autism, Asperger’s syndrome, addiction, cognitive ageing, stroke rehabilitation, physical and occupational therapy. However, the technology has historically been expensive, difficult to manage and distribute, and uncomfortable to wear, but that has changed and the opportunity for XR in these scenarios is potentially now more accessible than ever.

COVID-19 Increased XR Relevance and Need

XR technologies not only help deliver healthcare at a distance, but they are well suited to help address the challenges and changes that our societies find themselves addressing in confronting the global pandemic.

XR is helping provide mental health first aid for both enterprises and health care organisations, helping organisations spot potential indicators of mental health stressors. 

COVID-19 Stressors

TIME magazine reported that the number of US adults showing signs of mild depression and anxiety rose from 16.2% pre-pandemic to 24.6% between March and April 2020, when the lockdowns were introduced. 

The number of adults experiencing severe levels of depression rose by over 700% in the same time period. 

As mentioned in a recent report by the NHS, organisations should look to make use of XR and artificial intelligence techniques such as natural language processing and sentiment analysis, to interpret both written and verbal communications and identify the early warning signs of well-being and mental health issues, thereby allowing effective, proactive interventions and preventative measures to be applied.

The emteq labs white paper, “Improving Emotional And Psychological Well-Being In Distributed Digital Times” explores the topics of VR in mental health, well-being and clinical therapeutics in more detail.

“Future-selfing” an example of data driven XR and Preventative Medicine

VR technologies have been shown to enable users to willingly suspend disbelief and functionally disassociate from the real-world in a way that feels real and believable. With this ability, even the most surreal virtual constructions can have a profound impact on the user, such as an encounter with their own future-self.

Long term healthcare is notoriously difficult because humans over-value the pursuit of short term gratification at the expense of certain detrimental long-term impacts. Examples of this can be seen with victims of substance abuse, obesity, and smoking, amongst others.

With the ability to create artificially intelligent digital representations of a user using an Avatar, which is based on actual measurements of their current state and wellbeing, the Avatar can be programmatically aged to demonstrate the long-term impact for that individual.

Further, in a broader XR sense, the data models created here can be leveraged by other technologies such as smartphone-based AR applications to intervene in potentially damaging situations for that specific user. As Professor Walter Greenleaf described during the roundtable: “One of the things XR can do is compress time, allow us to see the future and help people be more proactive on their own health. It’s hard to do the things you need to do to help your future health, because we don’t see the results. We see what happens to us daily, and that’s where our brains are really focused, our immediate horizon. If somebody is challenged by drinking too much alcohol or other substances abuse, they just don’t see the daily accumulation, it’s just impossible for them to keep that in our mind. XR technology, though, can help with that it. We can create an Avatar of our future-self, we can have a dialogue with our future self. We can take sensor technology on our cell phone and have our future-self give us a phone call saying, “Hey, I noticed you’re not getting much exercise” or positive reinforcement such as, “I see that you’ve been moving around a lot more and have lost some weight, here’s a reward”.”

Preventative medicine is a hard thing to do, but I think with XR technology we have some new tools”, Professor Walter Greenleaf, PhD, Neuroscientist, Virtual Reality, and Digital Health Expert

Talking of his clinical career, Charles Nduka commented that, in his experience “80% of the things we treat in hospitals are preventable”. XR presents the opportunity to diagnose and manage physical and mental conditions in unobtrusive and intuitive ways that could help prevent many hospitalisations related to fatigue stress, poor nutrition, lack of mental well-being, pain, and addiction.

XR and Digital Divides

The promise of XR is significant. In healthcare alone, XR has the potential to improve lives, reduce suffering and inequality, and reduce costs for recipients, providers, and healthcare organisations. But will XR be the great enabler, or will it only benefit those who can afford it?

There is often a notion that game-changing, life-changing technologies should be delivered to everyone, everywhere, immediately. Unfortunately, there are many reasons why this is often infeasible, leaving Enterprises and wealthy nations to be amongst the early adopters.

Overtime, as the technologies, markets and business cases develop, costs drop and the technologies become more accessible, with new entrants competing on use-case and price.

The opportunities for XR are built on three technology pillars: connectivity, computing power and immersive displays, devices and sensors. The maturity of each of these pillars has grown exponentially in the past decade, and with it, the costs have also declined. 

Whilst high-end XR equipment can still cost thousands, entry level devices that comprise of low-cost smartphones and headsets, like Google Cardboard, are at an affordability point that fit within individual, if not, local community budgets.

Whilst these lower-end devices might not deliver the deep biometric insights of the premium devices, they could extend the reach and accessibility of critical services to millions of underserved, at risk communities. But this is unlikely to happen at scale in isolation, so is there a compelling business case that will support this?

The panellists of the roundtable believe the healthcare potential provides a compelling economic opportunity to continue to drive adoption, reduce costs and increase availability. 

For example, healthcare areas that XR could help reduce costs and drive adoption include:

  • Pain relief: The cost of pain-related health care in the USA is estimated to be USD 300 billion/year, which is greater than the combined cost of cancer and diabetes. XR technologies are already helping manage pain related conditions.
  • Mental health: emteq labs data indicates the UK alone could save in excess of GBP 163 million/year through the use of prescribed personalised immersive remote mental health therapies.
  • Prevention Training: VR training solutions have been shown to have higher efficacy than traditional methods and can be delivered at a fraction of the costs, even in remote scenarios. The use of XR training and coaching to prevent illness could help reduce hospitalisations, freeing up clinicians and resources, increasing availability and access to treatment and reducing costs for both the healthcare providers and patients.

Risks – Intimate Data and Profiling

Many of the opportunities XR could address rely on the capture and interpretation of personal data that goes beyond the types of data that previously caused industries, governments, and civil rights groups to act, as in the cases of digital rights management and GDPR.

XR experiences present the possibility to gather large amounts of intimate biometric data. Currently these data points can be directly captured, derived, or inferred, with and often without the participant’s knowledge.

During the roundtable, Charles Nduka shared that it is currently possible to determine the early signs of Parkinson’s disease from voice analysis many years before the participant may show any classic physical symptoms of the disease. Continuing, he commented on a study that showed how, by measuring the movement and focus of participant’s eyes (“gaze tracking”), researchers were able to identify the sexual orientation of participants pupillary responses.

Many technologies and applications are already converging towards what is known as the Metaverse. By definition, the Metaverse and XR technologies are symbiotic, and as such the challenges and risks of XR should be considered within the scope of the broad range of platforms that are coming together in this paradigm shift.

Alongside these developments, technologies such as machine learning and cloud computing, vast lakes of data can be processed in novel powerful ways, enabling sophisticated profiling of individuals across even anonymous interactions with multiple systems, services, or devices. 

“Now that we have such powerful tools such as machine learning, we have to consider that personal health information is sometimes escaping the clinical context, that we might be playing in a multiuser online video game where our voice is being recorded, our movements and our choices are being used, and eventually a profile can be built up and even if we’re part of that game anonymously.”

Professor Walter Greenleaf, PhD, Neuroscientist, Virtual Reality, and Digital Health Expert

Children and young people are less likely to understand or pay due care and attention to the information they are inadvertently giving away, to whom they are giving it and for what it might be used. With high participation of these demographics in major platforms like Minecraft, Roblox and Fortnite, careful consideration should be paid to the data these platforms collect from uninformed, unaware users.

More broadly, consumers continue to willingly give away personal data in return for free services. As Facebook’s Horizon takes flight and opens beyond the closed beta, will consumers understand that their long-term health or sexual orientation can be determined from how they play games or interact with AI avatars?

If the regulation around XR takes as long to come into force as it did for the Internet, there is a material risk that profiles and models made from direct and derived XR data could be used to identify, target, and discriminate, be it deliberately or inadvertently.

For illustration, perhaps a mid-twenties online gamer might find it difficult to get a loan to buy their first house because insurance companies refuse to offer life insurance due to the higher risk of the onset of Parkinson’s that they have derived from analysis of their conversations in the online games?

Time to Tame XR’s Wild-West 

With some similarities to the late 1990’s fevered growth of the Internet, XR is currently in a period of exponential industry- and community-led growth, with too little collective oversight to minimise the likelihood of systemic abuse that the Internet, and its 4.66 billion users are now plagued with.

In a vast majority of cases, XR technologies will be used for good, if not, for benign purposes. However, there will be situations where personal, intimate data is leaked, where companies or malicious actors seek to manipulate and exploit that data for power and, or profit.

Beyond the distress and harm these events will cause individuals, incidents of this nature leave deep scars across the entire ecosystem, wiping out trust, extinguishing innovation and slowing down progress of not just the bad, but the good too.

There has never been a more pressing time to discuss the opportunities alongside the risks and build a collective approach to mitigating the long-term harm of potential misuse, carelessness, or malicious event.

Intended Use

The healthcare and medical industry has rigorous checks and bounds to ensure the safety and security of patients comes first, and it all starts with the notion Intent. From a physician’s point of view, intent begins with ‘do no harm’, and the Hippocratic Oath. For medical devices, systems and data, intent comes from the intended use. 

If you were to walk into a shop and purchase a walking stick, you would walk out with a piece of wood or alloy-based material fashioned into something that resembles a walking stick. If a physical therapist were to prescribe a walking stick, the patient would be taking possession of a medical device, with which there is an expectation of outcome through a prescribed, intended use. 

In order to prescribe the walking stick, that piece of equipment must meet or exceed certain agreed and published standards. It is important that, in order to ‘do no harm’, the physician provides a piece of equipment that can adequately help with the patient’s ailment, without buckling, snapping or shattering and causing further, unintended injury.

However, even with the concept of intended use as a guide, interpretations and compliance can vary, as the enforcement of GDPR and other privacy regulations can demonstrate. In the case of safety and security of personal and health data in XR, we must consider data that is generated from many different domains, for many different purposes. 

Amplifying this complexity, the current global emphasis on technology-led solutions towards scaling health care availability and delivery, is creating a wave of exciting new technologies with specific medical focus, yet little knowledge of existing protections or regulations that are in place for personal health data and medical devices.

“The regulations are not where they should be. Novel types of hazards and novel types of data being collected, for example the attributes of immersive technology, and the regulators are not used to dealing with that, so new regulations are needed. But an even a bigger problem, in my mind, is the one that we have companies entering the medical field and trying to become medical device manufacturers that are not familiar with the current level of regulations.”

Rachel Michelson, Medical XR Advisor, XRSI

What Kind of Regulation is Most Suited?

Looking across industries and sectors, various models of regulation exist, some of which are in well-defined verticals, others cross several boundaries. The breadth, diversity and global distribution of the entities involved in the XR ecosystem makes governmental or industry specific control and regulation particularly challenging, whereas autonomous self-regulation might be open to interpretation and incompatible variance. We should also be mindful that a heavy-handed or single-focused approach could stifle innovation, and splinter cross-industry collaboration.
The telecoms industry is similar in terms of the number of sectors it touches, the actors it includes and the countries it covers. This industry has many standardisation bodies including, ITU, TIA, ETSI, IETF, 3GPP and GSMA. Most of these are communities of organisations across various industries, steered by telecom natives. The GSMA, for example, has 750 telecom natives (mobile operators) and 400 members from organisations that rely on, develop or contribute to the broader telecoms ecosystem.

What Needs to Be Regulated or Standardised?

An XR systems is the combination of hardware, data, software, and connectivity. Each element of the system, and the end-to-end system needs to be considered from a regulatory, standards and safety perspective.
However, as described in the “walking stick” example above, technology might be manufactured and applied to different scenarios, as such the scenario and intended use also need to be factored in.

Data: Consideration needs to be paid to how data is collected, stored, secured, shared, reported, scored, aggregated, assembled, and modelled. Amongst the areas of concern, instruments should be put in place to monitor for and protect against loss, theft, unauthorised and unintended use, and discrimination or bias.

Connectivity: Whilst communications technologies are already regulated and standardised across numerous organisations, the appropriateness of specific communications technologies should be considered for the intended XR use-case. In some situations, wired or wireless communications are preferential, in others specific types of wireless communications might interfere with specialised equipment. 

Software: Extending existing software certifications, XR systems should adhere to additional requirements based on their intended use. Beyond the operational capabilities of the software, procedures, therapies, and programmes fall into this category. Each of these will need to meet or exceed certain thresholds, carrying with them certifications that ensure security and safety and allow appropriate use.

Hardware: Beyond existing safety standards such as the CE or FCC marks, the intimate contact an XR device will often have with the participant will require due consideration of hygiene, allergies, comfort and ease of use, too.
Language: Due in part to the multitude of parties involved and the abundance of use-cases and stakeholders, a common understanding and language needs to be developed that allows effective communication and debate to take place around these issues.

Together these languages and understandings need to encompass the entire ecosystem, enabling transparent and fair standards to be regulated and applied for both specific intended usages. We must also be mindful that the intended use might facilitate unintended outcomes. For example, a virtual reality game might capture peripheral data, unnecessary for the intended use (also known as “exhaust data”), which when analysed with advanced technologies, and combined with other data sources could derive sensitive, personal and even intimate insights.

XRSI – community-led standards for safety in XR 

The XR Safety Initiative (XRSI) is a non-profit organisation developing a community-led approach to establishing standards, requirements and policies that help build a safe and inclusive XR ecosystem. The organisation’s focus is on discovering and mitigating the novel cybersecurity, privacy, safety, and ethical risks of the rapidly evolving XR ecosystem, thereby providing impartial, practical information about XR related risks and opportunities to individuals, corporations, universities, government agencies, and other organisations worldwide.

emteq labs’ Charles Nduka is part of the XRSI medical advisory council and hopes to contribute to their mission of safe, and privacy focussed XR.

To find out more about XRSI and how to get involved, visit www.xrsi.org

This paper was produced in partnership with emteq labs. 

emteq labs was founded in 2015 by two successful and experienced entrepreneurs, Graeme Cox and Dr Charles Nduka and the company has strong foundations in health, technology, research and business.

emteq labs focuses on research technology and uses a state-of-the-art brain-computer interface to quantify emotion, which enables the best quality research and improves patient care. By measuring emotional response and facial movement with biometrics and machine learning, in combination with virtual reality (VR), the integrated platform is the most ecologically valid and user-friendly system available.

emteq labs applies signal processing and detection algorithms to calculate likely human responses by collecting data that quantifies:

  • Expression
  • Heart Rate
  • Asymmetry
  • Valence & Arousal
  • Heart Rate Variability
  • Movement & Gestures
  • Eye Tracking

As well as the highly accurate measurement of human emotion and responses, emteq labs forms part of new and innovative services in the management of facial palsy, Parkinson’s disease and mental health disorders.

To find out more about how emteq labs supports cost-effective, high-quality research and innovative approaches to patient care, visit emteqlabs.com

EXTEND REALITY WITH AWARENESS! XR Safety and Privacy Guide for Artists

STANDARDS

This guide serves all of the artist and media communities. However, it zooms in on the subject of privacy and security for Individual creators, creative collaborators, and artistic developers. No matter which role you play, this guide can be used to ensure the key privacy and security considerations are taken into account during the design of XR products and experiences. Early adoption of these practices will help limit potential cybersecurity and privacy threats.

Taking privacy and security into consideration might not be a part of the natural creative process. However, it is essential to pay attention to these aspects when designing and developing. These proactive actions protect the artists themselves and the audience that may potentially interact with the art or enter into the same thought space as the one intended by the artist. Today’s artists and developers are setting a precedent that will be followed by generations. Therefore we must get it right to create a safe and trustworthy ecosystem.

Captain Cuttle and The Quest for Virtual Safety!

RESEARCH
Hiya! I’m Captain Cuttle. Come along with me on an adventure into brave new realities!

On this adventure into Virtual Worlds together we will go on a  Quest for Safety for all explorers. Together we will make sure everyone, even your friends and family, read the Explorer Notes so we can all play safe.

(more…)

VR Safety Tips for Parents: Top 7 XRSI Recommendations

RESEARCH

Important SAFETY TIPS Parents NEED TO KNOW about VIRTUAL REALITY

With the Holiday season around the corner, there are going to be a lot of kids with new Virtual Reality (VR) headsets. One of the most common uses of VR is gaming because it gives players an incredibly immersive experience. VR can be a fantastic medium for education, letting adults and children alike feel immersed in the subjects they’re exploring, whether that’s a historic city or human anatomy.

If you are a parent, a teacher, or a guardian, you may be confused about whether VR is safe for children. There are several ways for you to enable safe exploration of VR technology, with a key understanding in mind: It is your responsibility to guide your kids to facilitate safe, responsible, and healthy relationships with VR. With an objective to help facilitate safe exploration of VR for kids, XRSI has put together Top 7 Safety Tips for parents.

(more…)

The XRSI Privacy Framework version 1.0

FRAMEWORK RESEARCH

XR Industry is moving fast, so is the urgency to create standards, guidelines, and awareness for XR stakeholders. Recent news about data, privacy, and safety concerns are growing as the technological advancements take place. To address this urgent XRSI is releasing a novel Privacy framework version 1.0.

The framework is a free, globally accessible baseline rulebook built by bringing together a diverse set of experts from various backgrounds and domains, including privacy and cybersecurity, cloud computing, immersive technologies, artificial intelligence, legal, artists, product design, engineering, and many more. 

(more…)

The XRSI Definitions of Extended Reality (XR)

STANDARDS PDF 4.79 MB

XR Safety Initiative (XRSI) releases the first set of standard definitions and taxonomies for the immersive technologies and related domains. The document sets the common baseline for a shared vocabulary in the industry, marking the first step in the creation of a full Data Classification Framework, therefore, is not to be considered complete. Additional taxonomies and new publications are to be added continually.

(more…)

Immersive Technology Standards

STANDARDS

The XR domain needs a new mindset. As we examine this concept collectively, we learn the magnitude of the potential impact on humanity’s well-being. Furthermore, when we introduce the exponential global aspect of immersive technology and the effect of the recent catalyzing events, we sense the urgency to create a baseline for designing and developing these ecosystems.

The Immersive Technology Standards for Accessibility, Inclusion, Ethics, and Safety is written for a time such as this, with the intent to serve as the blueprint for all that have a stake in the vastly evolving and emerging XR ecosystem. Now is the time to lay down the foundation on which these new ecosystems provide more equitable opportunities and inclusive advancement for all.

Immersive Technology Standards – Diversity and Inclusion

STANDARDS

We are at a crossroads between emerging technologies, data sciences, and cybersecurity — fueled by the renewed, global necessity of diversity and inclusion (D&I) in each domain.

Technology jobs are still facing high levels of gender and ethnicity inequality, and with the rise of Artificial Intelligence-based solutions, the gender and ethnicity exclusion issue is becoming more relevant. The overrepresentation of white men in the design of these technologies could undo decades of advances in gender and racial equality.

Emerging technologies are not going to solve the issue if we’re not going to change the input: If that data carries stereotypical concepts, the resulting application of the technology will perpetuate this bias. The models and systems we create and train are a reflection of ourselves.

Immersive Technology Standards – Ethics and Awareness

STANDARDS

The emerging digital technologies, from artificial intelligence to ingestible sensors, create various ethical challenges, which need to be considered by all stakeholders, including the scientists and engineers involved in their development.
What changes will the emerging technologies have on our minds and bodies? What effects will they have on our relationships with each other, our interaction with machines, and the state of the environment? Shaping the future for the best possible outcome is why defining digital ethics is an imperative for us today, and the challenge must be faced by building an aware and prepared industry and society.
It is wrong to rely on the caution and awareness of the single company or developer, and that is why we need to have ethical guidelines. Having a Universal XR standard covering the ethical field offers the unique opportunity to create an ethical filter in the digital transformation process for everyone and every organization involved in the process. Building ethical cultures in the emerging domains is stronger, and more durable, than regulation alone.

Immersive Technology Standards – Trust and Safety

STANDARDS

The mass adoption of immersive technologies is changing our lives, offering new opportunities, and posing new threats. The large amount of data we deal with, and the expanding attack surface, require us to build trust to a new degree, beyond the traditional concepts of privacy and security. We need a new mindset that incorporates all dimensions of the attack surface, whether it is hygiene for sharing headsets, preventing dizziness and fall due to motion sickness, or minimizing abuse of data and privacy. XR requires us to understand the associated risks in terms of safety, which incorporates privacy and security but also takes into account physical and psychological harm.

Access to XR experiences is progressively becoming more attainable and cost-effective, and some immersive experiences can be as realistic as the “real” world experience. That’s why we need to protect everyone, especially the most vulnerable among us, by creating standards that encourage preventive design, solutions, resources, and tools. The abuse or misuse of XR technologies can cause risks and effects on a physical, emotional, developmental, and material level. The rapid adoption of XR, without appropriate standards, research, and guidelines is particularly problematic for children as their brains are not fully developed. Take for instance, the fact that the industry has not yet established the appropriate age for children to be exposed to virtual reality, nor for how long they should experience a particular platform or virtual environment. Children’s safety online has suddenly become more urgent and we need to ensure a special focus is being given to protect children from harm, as they get exposed to XR.

Privacy and security also remain critical to the mission of creating trust. XR technologies have the potential to record all new kinds of user information, from eye movements and emotions to the movement of a user’s entire body through space, so the data must be managed responsibly.
When we talk about “data” in XR, we’re talking about an ocean full of information on any given object, place, or individual. Consumer VR systems typically track body movements 90 times per second to display the scene appropriately, and high-end consumer systems record 18 types of movements across the head and hands. Consequently, spending 20 minutes in a VR simulation leaves just under 2 million unique recordings of body language. Given the enormous amount of data collection and blurred lines of accountability, privacy and data protection has become more complex than ever, and clarity around data ownership, usage, and meaningful consent is more urgent.

XR technologies broaden the range of ways in which a person’s identity, as well as public and private assets, can be at risk. The attack surface has expanded, and with the rise of data breaches and hacking, security is at stake, to the point that XR is considered one of the top 10 privacy risks to watch in 2020 by members of the Future of Privacy Forum.