Preparing for the era of brain controlled machines
Analyse Tomorrow’s Worlds

Preparing for the era of brain controlled machines

11 May 2022 - 12:55
Photo: Brain waves are registered by an electrode helmet and analysed by a computer. Via brain computer interface you can operate a computer. © Ars Electronica - Flickr
Terug naar archief
Author(s):

                                                                                                                                                  Button Tomorrow's Worlds

Communicating directly with your brain is possible with brain computer interfaces (BCI). How will this technology change medicine, the workplace and the military? In the latest contribution to our topic ‘Tomorrow’s Worlds’, futurist and biohacker Peter Joosten presents three scenarios to examine how we can prepare for the era of brain controlled machines.

A brain computer interface (BCI) translates brain signals into instructions for software or hardware.1  There are various techniques for picking up signals from the brain. This can be done, for example, on the outside of the brain with sensors. Commercial manufacturers such as NextMind or Emotiv offer headbands for playing computer games or controlling the lighting in a house.

In contrast, with the semi-invasive method, the sensors are located under the skull and on top of the brain.2  Even beyond that, in the full-invasive method, the sensors are located inside the brain. A well-known example is Deep Brain Stimulation, a medical application used by some patients with Parkinson’s disease.3

Joosten - Regina Dugan presents a project that would allow people to type words using censors that measure brain activity at Facebook's F8 conference in 2017. Anthony Quintano - Flickr
Presentation of a project that would allow people to type words using censors that measure brain activity at Facebook's F8 conference in 2017. © Anthony Quintano / Flickr 

There are several uses for BCI. In healthcare for instance, patients with locked-in syndrome (being fully conscious but not able to move, or only the eyes) or patients with tetraplegia (paralysis of arms and/or legs) use it to communicate with caregivers and their relatives. BCI also seems a logical next step for those working in situations where every second counts, such as stock traders or jet pilots. For them, BCI can provide a competitive advantage.

What will life with BCI’s look like? How can the technology change medicine, the workplace and the military? This piece describes three brief stories to explore the use of BCI for human augmentation, extending and improving human skills.

BCI scenario 1: patient Dennis

After waking up, Dennis lifts his numb legs into his exoskeleton. On the bedside table lies the neural interface by manufacturer Liviu (type: Alpha), which translates brain signals into data for the computer in his exoskeleton. The computer uses algorithms to translate this data into instructions for the little motors and tiny engines that sustain balance and realise movement.

After a few weeks of training and adjustments, Dennis controls the exoskeleton seamlessly. It almost feels like the old days. He does not even need to consciously think about walking; having the intention is already enough. Dennis feels the small engines spinning, and his right leg moves forward – exactly at the speed and distance he had envisioned.

After the shock, disbelief, anger and sadness, he is now quite used to walking with his exoskeleton

Three years ago, Dennis woke up startled in a hospital in Innsbruck, Austria. His snowboarding holiday had ended dramatically after a sudden crash against a boulder off-piste. It had left him paralysed from the fourth vertebra down. After the shock, disbelief, anger and sadness, he is now quite used to walking with his exoskeleton.

Multitasking is not a problem. While Dennis is walking to his favourite coffee shop, unconsciously controlling his exoskeleton, he is able to listen to the illustrious jazz musician Chet Baker and think about the offer he received from Liviu as well.

Dennis recently received a newsletter from Liviu about a new software update: snowboarding with your exoskeleton. The Shaun White edition looks really slick. Nevertheless, Dennis has some doubts: if the external skeleton performs all the twists and turns, does a perfect descent on his snowboard give him the same amount of joy and satisfaction like in the old days?

BCI scenario 2: stockbroker Olivia

Olivia rushes up the stairs of the subway, almost spilling her coffee-to-go. Her autonomous coffee machine started brewing precisely in time, so the steaming drink was ready at the exact moment she left her apartment. Her coffee is synthetic, which means it is made in a lab and contains extra cognitive boosts. The boosts give her more focus, mental energy and an increased memory.

According to Olivia, this synthetic coffee tastes better than the original drink, and it is more sustainable and cheaper as well. Due to the climate crisis, there are hardly any coffee plantations left; old school coffee made from beans is therefore more expensive.

Olivia arrives at the office. She works at Axion Capital, a company that manages the assets of large pension funds and wealthy private investors. Olivia’s workplace distinguishes itself from fully algorithmic traders, because of the combination of man and machine.

The headband analyses her cognitive and emotional state while working

She puts the Liviu (type: Beta) headband on her head. The headband analyses her cognitive and emotional state while working. The aim is to recognise human biases and automatically adjust Olivia’s intended actions on the stock market accordingly, while at the same time using the advantage of human intuition.

But the Beta goes even further than other types, like the Alpha. The headband is able to determine how focused Olivia is. When her cognitive condition drops below a certain value, a subtle electrical pulse travels from her forehead to just behind her ears. “Qzzzz!” It feels like she is infused with a quadruple espresso.

Olivia receives this concentration boost every morning after a few hours of work, probably because the effect of her morning coffee begins to wear off at that moment. The timing is perfect, since the New York Stock Exchange opens every day at 11 am sharp.

Olivia feels a slight tingling through her skull, but this time it feels different. She feels a sense of restlessness in her lower abdomen. Normally, the intensity of the pulse would now decrease. That is strange.

In the reflection of one of her computer screens, she sees that in addition to the red light, the blue light on her headband is flashing as well. This means something is not right.

She looks around. There is a lot of rumour. Olivia sees blue and red lights together blinking on the headbands of all her colleagues. They look at each other, bewildered and worried. First there is silence, followed by a loud scream.

BCI scenario 3: fighter jet pilot Alison

Alison has heard the stories. She learned that a guy she flirted with on a winter sports holiday in Austria had suffered a spinal cord injury a few years later and is now powering his exoskeleton with one of the early versions of the Liviu.

She has also trained with the Beta type, both in virtual reality combat simulations and in physical training. Using the Beta was bizarre, referring to the instant focus she got. She sometimes felt the concentration boosts nanoseconds before she needed it. As if the system could react and anticipate faster than her own biological cortex.

The latest release of Liviu is de neural interface type Foxtrot. “The Foxtrot was developed by Liviu in a special facility. Nobody besides their best engineers, and we know about this version”, says Pedro, her personal air force technician. Alison and the other jet pilots are already used to technologies that improve their capabilities, but this is next level.

“This is full integration between human, aircraft and the supporting drone swarm.” Alison knows about the theory from experts in artificial intelligence and warfare. Fully automated swarms of drones are the norm in air fights. However, the addition of the human element makes it more difficult for hostile algorithms to manoeuvre in air fights, because of the unpredictability, emotion and surprise of people. 

Alison is aware of the price to her identity. The Liviu Foxtrot needs to be inserted directly into her brain via a chip, to pick up the highest quality of brain signals. She is not worried about the operation; her concerns lay elsewhere. Will she still be Alison if she puts a piece of hardware from a company and the army, her employer, into her brain?

The swarm module, where she controls the drones with her thoughts, is even more powerful

However, she is able to push the feeling away. The ecstasy of the Foxtrot takes care of that. She has felt one with her plane before, but with the implant she literally is. The vibrations on the wing hit her nervous system directly. This is what it is like to fly like a bird she thinks to herself.

And the swarm module, where she controls the drones with her thoughts, is even more powerful. It is as if her thoughts and intentions are linked to the twelve drones now whizzing around the plane.

Alison hears her colleague Ethan approaching her. He is flying thirty meters to her left. His Augmented Reality glasses – by which he sees visual information projected on the glasses – switch to transparent. They look at each other, he nods. Despite the distance, she can tell from his facial expression that he is nervous too.

The commander gives the signal: “Hive module activated.” When she hears those words, the feeling sets in. They had already trained so much together in virtual reality. Now, eight kilometres above the Nevada desert, she fuses together with Ethan. She can feel his thoughts, considerations and intentions; it is thrilling. They are one entity: two fighter jets, two drone swarms and two connected brains.

The fifteen minutes fly by. Alison and Ethan follow the procedure to disconnect. This is weird, Alison thinks, and she realises Ethan notices the same thing. She presses the button to deactivate the module. Nothing. She tries it by using voice command and says: “Unpair Hive module.” She feels Ethan is trying this as well. Still nothing. She contacts the base station by radio. No reaction. Her heart rate increases. She can feel Ethan’s slight panic.

Then she feels they are not alone anymore. A third brain is in the Hive. It does not feel familiar; not American, and maybe not even human but like a sort of artificial intelligence. “Who is this?” She sends this question as a thought to Ethan and the other entity. Nothing. That is odd.

The third brain is growing in strength. It seems to slowly take control of the drones, of the fighter jets, and then of Alison’s thoughts. She screams. Alison looks to Ethan and also sees the panic in his eyes. This is really not good.

an impression of The g.tec Brain-Computer Interface Exhibition as part of BR41N.IO Hackathon
Impression of the g.tec Brain-Computer Interface during the BR41N.IO Hackathon in Austria, 2017. © Ars Electronica / Flickr

The impact of BCI
Various applications of BCI emerge in the three described scenarios. Dennis uses it within a medical context, to control his exoskeleton. Olivia goes one step further. Her Liviu Beta not only analyses her brain data, but stimulates her brain too. Reading as well as stimulating the brain, as is the case in Olivia and in Allison’s story, is called bidirectional BCI. Dennis’s neural interface only reads his brain signals, and is therefore one directional. Finally, Alison uses her system to control her plane and merge her thoughts with another pilot. 

In addition to the different applications, the three stories highlight all kinds of dilemmas in the use of BCI’s. In Dennis’s case, technology seems to have dramatically increased his quality of life. However, his reluctance regarding the snowboard module is also noticeable. Does a perfect descent on a slope, where the external skeleton performs all the movements, provide the same sense of accomplishment as learning and performing it yourself?

With Olivia, a question is whether it is desirable for an employer to have access to the brain data of its employees. Her company uses the technology to optimise productivity. What does this mean for the intrinsic motivation and self-esteem of the users of the BCI?

Furthermore, at the end of the story, hackers break into the system of Olivia and her colleagues. This raises important questions about the degree of security of these systems, especially when the headband can also stimulate the brain with electric or magnetic pulses.

uses an elaborate brain-computer interface (BCI) system where he controls a hi-tech robot with his brain via state-of-the-art technology.
An artist uses an elaborate brain computer interface (BCI) system where he controls a hi-tech robot with his brain during the Ars Electronica festival in Austria in 2017. © Ars Electronica / Flickr

Alison’s story is the most far-reaching, but not far-fetched. A programme of the American Defense Advanced Research Projects Agency (DARPA) is Next-Generation Nonsurgical Neurotechnology (N3). The aim of this programme is to develop non-invasive BCI’s that can both analyse brain activity and return signals. One of the projects in the N3 programme focuses on how pilots can use their thoughts to control a swarm of drones (supported by artificial intelligence) and receive signals. Soldiers call this concept the ‘loyal wingman’ as an analogy of jet pilots helping each other and providing cover.

In Alison’s story, she is asked if she is willing to implant a chip in her brain. This raises questions about ownership and identity. Will the chip soon be part of Alison? What happens to the implant when it is taken out of service? Will such an operation soon be a requirement to qualify for certain parts of military organisations? Does the chip change her personality? Can people with such an implant also be hacked, as happens in the story?

The stories of Dennis, Olivia and Alison show the insane possibilities of BCI’s in the future. For some specialised professions, the usage of BCI seems inevitable because of the extension and augmentation of human capabilities. But above all, it raises a lot of questions.

Brain Computer Interface
Austrian fire brigade using a brain computer interface during the Ars Electronic festival in 2018. © Ars Electronica / Flickr

How far are patients, employees, soldiers, companies and countries willing to go to exploit the potential of BCI? A lot of moral, political and philosophical questions should first be examined more closely.

These issues are already being discussed in some institutes. In 2019, the Organisation for Economic Co-operation and Development (OECD) emphasised that there are major risks surrounding brain computer interfaces. This warning was followed in 2021 by the UNESCO Bioethics Committee, with a report stating that there is hardly any legislation on neurotechnology.

Yet, countries are still not doing enough in this area. Currently, Chile is leading the way in creating proper legislation. In 2020, the Chilean parliament passed a law to establish neuro rights.4  The aim of this legislation is to give personal brain data the same status as an organ, so that it cannot be bought or sold, trafficked or manipulated.

Proper legislation is only one key aspect ensuring that this technology is used wisely. BCI manufacturers also need to protect their users. Especially when it comes to valuable and intangible affairs as someone’s thoughts, emotions, feelings and the biological brain.

Finally, future users, employees and employers must be aware of the various challenges that may arise in the further development of this technology, such as the increasing presence of hackers. Hopefully the stories of Dennis, Olivia and Alison contribute to this awareness.

  • 1Brain machine interface (BMI) is another commonly used term. In 1973, computer scientist Jacques Vidal first coined the term BCI in his paper ‘Towards Direct Brain-Computer Communication’ in the Annual Review of Biophysics and Bioengineering. Jacques Vidal, ‘Towards Direct Brain-Computer Communication’, Annual Review of Biophysics and Bioengineering, 1973. Authors Schalk and Allison provide this definition of brain computer interfaces (BCI) in their contribution to the book Neuromodulation: “BCI is a system that measures the activity of the central nervous system and converts it into artificial output that replaces, restores, complements or enhances, thereby altering the ongoing interactions between the nervous system and its external or internal environment.” Gerwin Schalk, Brendan Z. Allison, ‘Noninvasive Brain–Computer Interfaces’, in: Elliot Krames, P. Hunter Peckham, Ali Rezai, Neuromodulation, Cambridge (MA): AcademicPress, 2nd edition, 2018.
  • 2At the UMC Utrecht in the Netherlands, the Utrecht Neuroprosthesis research group of Professor Nick Ramsey uses this method.
  • 3The company Neuralink is further developing this method, inserting with tiny needles and a robot surgeon.
  • 4Report of the International Bioethics Committee of UNESCO (IBC) on the ethical issues of neurotechnology’, International Bioethics Committee, 2021.

Auteurs

Peter Joosten
Futurist & biohacker