Experiments with neural interfaces on JavaScript

The author of the material, the translation of which we are publishing today, says that for the last couple of years he has noticed a steady interest in neurotechnology. In this article, she wants to talk about her experiments with various hardware and software systems that allow you to establish a connection between the brain and the computer.
 3r3634.
 3r3634. Experiments with neural interfaces on JavaScript
 3r3634.

 3r3634.
Prehistory
3r3608.
 3r3634. I do not have basic computer education (I studied advertising and marketing). I mastered programming on courses in General Assembly.
 3r3634.
 3r3634. When I was looking for the first job, I started experimenting with jаvascript and with different devices. In particular, my first project was the organization of control of a robotic ball Sphero by moving the arm with the help of Leap Motion .
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. Sphero ball, hand-guided with Leap Motion
technology. 3r3600.
 3r3634.
 3r3634. This was my first experience using jаvascript to manage something outside of the browser. I was immediately, as they say, "hooked."
 3r3634.
 3r3634. Since then, I have spent a lot of time working on interactive projects. Each time, taking on a new project, I tried to find for myself more and more complex tasks. So I constantly developed and learned something new.
 3r3634.
 3r3634. After experimenting with various devices, in search of another interesting task, I came across sensors of brain activity from the company 3r-356. NeuroSky
.
 3r3634.
 3r3634.
The first experiments with the neuro headset
3r3608.
 3r3634. When I was interested in experiments with sensors of brain activity, I decided to purchase a NeuroSky neuro headset. She was much cheaper than other similar offers.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. NeuroSky NeuroSky
3r3600.
 3r3634.
 3r3634. I didn’t know if my qualification was enough to write at least something for such a device (I just finished programming courses at that time), so I decided to choose something cheaper so that, if the task turns out to be for me prohibitively difficult, do not waste too much money. Fortunately, a jаvascript framework has already been created for working with the headset, so it was quite easy to start the experiments. In particular, I used the assessment of the level of my attention to control the Sphero ball and the Parrot AR.Drone quadrocopter.
 3r3634.
 3r3634. During the experiments, I quickly realized that this neuro headset is not particularly accurate. She has only three sensors, so she allows to obtain rather approximate data on brain activity. The device gives access to the raw data from each sensor, which allows, for example, to visualize this data. But the fact that the headset has only three electrodes does not allow one to draw any serious conclusions about what is happening in the human brain from the data obtained from it.
 3r3634.
 3r3634. When I decided to look for other devices to read indicators of brain activity, I found a neural headset 3r3393. Emotiv Epoc 3r3627. . I had the feeling that this thing has more serious features in comparison with the NeuroSky headset, so I decided to buy it to continue my experiments.
 3r3634.
 3r3634. Before I talk about how Emotiv Epoc works, I suggest to briefly talk about how the human brain works.
 3r3634.
 3r3634.
How the brain works 3r3607. 3r3608.
 3r3634. I can't call myself a great neuroscience expert, so my story about the brain will be rather superficial. Namely, I want to talk about a few basic things that need to be known to those who want to better understand how the neurohearnes work.
 3r3634. The brain consists of many billions of neurons - specialized cells that are engaged in the processing, storage and transmission of information. Different parts of the brain, consisting of neurons, are responsible for different physiological functions.
 3r3634.
 3r3634. 3r32424. 3r3r166. 3r33535.
 3r3634. 3r3-3597. 3r3-3598. Different parts of the brain (source - macmillan.org.uk)
3r3600.
 3r3634.
 3r3634. For example, let's talk about how the brain controls the movements. Parts of the brain, such as the primary motor cortex and the cerebellum, are responsible for movement and coordination. The signals of the corresponding neurons act on the muscles, which leads to movements.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. Anatomy of the neuron
3r3600.
 3r3634.
 3r3634. As I have already said, an extremely simple description of the brain is presented here, but for us the most important thing is that the activity of neurons can be tracked using electroencephalography (EEG) methods by reading indicators of the brain's electrical activity from the surface of the scalp.
 3r3634.
 3r3634. Other technologies can also be used to monitor brain activity, but their use implies surgery. In particular, we are talking about electrocorticography - with this approach, the electrodes are superimposed directly on the cerebral cortex.
 3r3634.
 3r3634. Now that we’ve found that the brain generates electrical signals that can be read as we work, let's talk about the Emotiv Epoc headset.
 3r3634.
 3r3634.
How the neuroheck set works
3r3608.
 3r3634. The Emotiv company produces several types of neurogron sets:
 3r3634.
 3r3634.
 3r3634.
Emotiv Insight
 3r3634.
Emotiv Epoc Flex Kit
 3r3634.
Emotiv Epoc
 3r3634.
 3r3634. The Epoc headset has 14 sensors (also called “channels”) located in different places of the head.
 3r3634.
 3r3634. The following figure, on the left, shows the layout of electrodes 10-20 recommended by the International Federation of Electroencephalography and Clinical Neurophysiology. Each electrode corresponds to a specific area of ​​the brain. The use of the 10-20 system allows you to follow a certain standard when creating various devices and when performing scientific studies of the brain.
 3r3634. The figure on the right shows the layout of the electrodes of the Emotiv Epoc headset. To compare it with the 10-20 system, the selection is green and orange.
 3r3634.
 3r3634. 3r32424. 3r3192. 3r33535.
 3r3634. 3r3-3597. 3r3-3598. Comparison of the international electrode placement system 10-20 and the Emotiv Epoc 3r3607 headset. 3r3600.
 3r3634.
 3r3634. The 14 Epoc channels are not so much, but the electrodes are placed evenly across the scalp. This gives hope that with Epoc it is possible to obtain fairly accurate information about brain activity.
 3r3634.
 3r3634. The headset reads sensors at a frequency of 2048 samples per second (SPS). At the same time, the user has access to the signal sampling rate of 128 or 256 SPS. The device is capable of capturing brain waves with a frequency of ??? to 43 Hz. There are various rhythms of the brain, their brief characteristics are shown in the following figure.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. Types of brainwave
3r3600.
 3r3634.
 3r3634. Why is it important? The fact is that depending on the application that needs to be built on the basis of an electroencephalograph, we may need to pay special attention to brain waves of a certain frequency. For example, if we need to create a program to help meditators, then we are probably interested only in theta waves, whose frequency is 4-8 Hz.
 3r3634.
 3r3634. Having dealt with the principles underlying electroencephalography, let's talk about the possibilities of Emotiv Epoc and related software.
 3r3634.
 3r3634.
Features Emotiv Epoc 3r3607. 3r3608.
 3r3634. Emotiv software is not open source, a special license is required to access the raw sensor signal. Under normal conditions, when working with Emotiv Epoc, the following features are available:
 3r3634.
 3r3634.
 3r3634.
Measurement of indicators characterizing the position of the user's head in space using an accelerometer and gyroscope.
 3r3634.
Measurement of the level of arousal, involvement, relaxation, interest, stress, concentration.
 3r3634.
Recognition of facial muscle movements, giving an idea of ​​the user's facial expression. For example, we are talking about blinking and a smile.
 3r3634.
Recognition of mental commands (movement and turns).
 3r3634.
 3r3634. In order to use the recognition of mental commands, first the user needs to train 3r-3260. system 3r3627. . Learning data is saved as a file.
 3r3634.
 3r3634. If you want to develop your own programs for Emotiv Epoc, you can use the API Cortex and the corresponding SDK (its support is discontinued after the release of version 3.5). If you want to use jаvascript, you can take a look at my development, the library. Epoc.js .
 3r3634.
 3r3634.
Library Epoc.js
3r3608.
 3r3634. Epoc.js is a framework designed to interact with Emotiv Epoc and Insight devices using jаvascript. This framework gives the developer access to the above features of the systems from Emotiv and allows you to interact with 3r-3278. emulator
.
 3r3634. Here is the simplest project based on Epoc.js:
 3r3634.
 3r3634.
const epoc = require ('epocjs') (); 3r3634. epoc.connectToLiveData ('path /to /profile /file', function (event) {
var action = event.blink === 1? 'blinking': 'not blinking'; 3r3634. console.log (action); 3r3634.}); 3r33333.
 3r3634. In this sample code, we include the Node.js module. epocjs and create an instance of the corresponding object. Then we call the method. connectToLiveData This object, passing to it the path to the file with user data obtained after training the system, and the callback function. This function is passed an event object that contains various properties available for tracking. For example, if we want the program to react to blinking, the property 3r-3300 is used. event.blink
.
 3r3634.
 3r3634. Each such property can be set to either 0 or 1. A unit in the property value means that the system has recorded the corresponding event. A complete list of these properties can be found here is .
 3r3634.
 3r3634. The described library was created using the Emotiv C ++ SDK, Node.js and three modules for Node.js: Node-gyp, Bindings and Nan. Its development used an approach that can now be considered obsolete. Now the use of 3r33312 is relevant. N-API 3r3627. .
 3r3634.
 3r3634. Having discussed the various possibilities of neuroheternologies and the ways of program work with them, I’ll tell you about several prototypes that I created that use the neural interface.
 3r3634.
 3r3634.
Prototypes 3r3607. 3r3608.
 3r3634. 3r? 3539.
▍1. Keyboard
3r33542.
 3r3634. This is how the keyboard is controlled by eye movements.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. The prototype of the keyboard, driven by eye movements 3r3603. 3r3600.
 3r3634.
 3r3634. This was my first project created using Emotiv Epoc. I wondered if it was possible to create a simple interface using a neuro headset that allows a person to interact with a computer using eye movements. For example, when you translate the view to the right or left on the keyboard, the corresponding keys are highlighted. In order to "press" the highlighted key, you need to blink. The corresponding letter appears in the box above the keyboard.
 3r3634.
 3r3634. This project looks very simple, but the most important thing is that it works.
 3r3634.
 3r3634. 3r? 3539.
▍2. WebVR
3r33542.
 3r3634. In my second project I used mental commands. Creating it, I wanted to understand whether it is possible to control an object that is in three-dimensional space, just thinking about something.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. A thought-driven web interface
3r3600.
 3r3634.
 3r3634. Here, to create a simple three-dimensional environment, I used the Three.js library, the Epoc.js library was used to recognize mental commands, and web sockets were used to send data from the server to the client.
 3r3634.
 3r3634. 3r? 3539.
▍3. IoT
3r33542.
 3r3634. Starting the third project, I wanted to explore the possibilities of managing real devices using mental commands. I have been interested in IoT development using jаvascript for quite some time, so it was interesting for me to know what happens if I combine the Parrot quadrocopter and neuro headset.
 3r3634.
 3r3634. 3r32424.  3r3634. 3r3-3597. 3r3-3598. Quadcopter
3r3600.
 3r3634.
 3r3634. All the projects described above, all the prototypes created, are pretty simple designs that I created in order to test some ideas in practice and evaluate the possibilities and limitations of neural interfaces.
 3r3634.
 3r3634.
Limitations of neural interfaces
3r3608.
 3r3634. The word "neurointerface" sounds amazing, and when it turns out that a computer can be controlled by the power of thought, it may seem that here it is - the future, but, in fact, neurocomputer technologies still have quite a lot of limitations.
 3r3634.
 3r3634. 3r? 3539.
▍The need for learning 3r3-3607. 3r33542.
 3r3634. It is quite normal that users have to perform system training, during which they record brain waves and compare them with certain commands, but for many such a step is an obstacle in adopting a new technology. It is hard for me to imagine that someone will spend time on learning neurocomputer systems, except that such a system will be really needed by someone, and the accuracy with which it recognizes mental commands will be at a very high level.
 3r3634.
 3r3634. 3r? 3539.
▍ Delays
3r33542.
 3r3634. When I developed my prototype, based on computer perception of mental commands, I found out that there was some delay between the moment when I started to think and the program when I reacted to this thought.
 3r3634.
 3r3634. I think the point here is that the machine learning algorithm used in the prototype receives data from the device in real time. In order to recognize the thought, which he had previously learned, he needs indicators collected over a certain period of time.
 3r3634.
 3r3634. This has an impact on which programs can be built on the basis of a neural interface. For example, a program that helps to meditate looks quite realistic, since the delays between a change in brain state and the program’s response do not particularly affect the results of such a program. However, if you aim to create something like a wheelchair controlled by thoughts, the problem of delays becomes much more acute, calling into question a similar development.
 3r3634.
 3r3634. 3r? 3539.
▍Non-invasive technology and accuracy of indicators 3r3607. 3r33542.
 3r3634. Electroencephalogram removal devices are great for everyday use in ordinary life situations. It is enough to put on a headset, putting a special gel on the sensors, and everything is ready. However, the fact that signals generated by the brain are read from the scalp, and not, say, from the surface of the brain itself, impairs the accuracy of such signals.
 3r3634.
 3r3634. If we talk about the frequency of removal indicators, it is very good in existing devices. The same cannot be said about the spatial characteristics of the data obtained. EEG devices can only read signals originating in those parts of the brain that are close to the surface of the head. About what happens in the deeper structures of the brain, using a similar approach can not be known.
 3r3634.
 3r3634. 3r? 3539.
▍ Community acceptance
3r33542.
 3r3634. Neuro headset is not the cutest and most familiar device. I think that as long as these headsets look like they are now, they are unlikely to be worn in public places. As technology advances, it may be that devices are created that may be hidden in accessories like hats, but even here you may encounter a problem related to the fact that such devices will be inconvenient when worn for a long time.
 3r3634.
 3r3634. EEG sensors should fit quite tightly to the scalp to remove the quality indicators of brain activity. And if their pressure immediately after putting on the headset can almost not be felt, over time it begins to cause discomfort. Moreover, if the sensors also need to be applied gel, it turns into an additional barrier to the wide distribution of neurohears.
 3r3634.
 3r3634. As you can see, the current state of affairs in the field of neural interfaces suggests that they are unlikely to be widespread. However, if we talk about the future, then we can say that such devices have interesting perspectives.
 3r3634.
 3r3634.
Features neural interfaces
3r3608.
 3r3634. If you take into account the current state of technology and think about what they may become in the future, you can find several options for their application.
 3r3634.
 3r3634. 3r? 3539.
▍ Help people with disabilities 3r3607. 3r33542.
 3r3634. I would like the neuro headsets to help people with disabilities live a fuller life and be more independent.
 3r3634.
 3r3634. This is what I thought about when I created my first prototype - a keyboard, controlled by eye movements. This development of mine is far from the level when it could be used in practice, but working on this project, it was interesting for me to understand whether a quite affordable consumer device can really help someone. Not everyone has access to complex medical systems, and I was just amazed that the not very expensive thing that you can freely buy in the online store is able to solve important and necessary tasks.
 3r3634.
 3r3634. 3r? 3539.
▍Mental Practices
3r33542.
 3r3634. Mental practices, in particular - meditation - this is the scope of neurohears, which already today attracts a certain attention (for example, headset 3r313133. Muse 3r3627. Helps meditate). It is about helping someone who wants to meditate, to do everything right.
 3r3634.
 3r3634. 3r? 3539.
▍Help in solving health problems 3r3607. 3r33542.
 3r3634. If the neuro headsets would penetrate into our lives as much as mobile phones, we could probably create applications that can respond to health problems. For example, it would be great if there were applications that, based on an analysis of brain activity, would help fight strokes, panic attacks, epilepsy episodes.
 3r3634.
 3r3634. 3r? 3539.
▍Increase productivity 3r3607. 3r33542.
 3r3634. The neuroheck headset can help meditate, which means that with its help it is possible to actually figure out what time of day a person concentrates. This information, obtained by regularly wearing a headset, can help to understand when it is best to engage in certain activities. You can even imagine that the work schedule will be organized in accordance with the individual characteristics of a person, which will increase the productivity of his work.
 3r3634.
 3r3634. 3r? 3539.
▍Art
3r33542.
 3r3634. I like, on my own initiative, during off-hours, to explore phenomena at the intersection of art and technology. I believe that one should not underestimate the work in this direction related to neural interfaces, since, although they may seem "frivolous," they help to better understand technologies that will prove useful in more "serious" cases of their use.
 3r3634.
 3r3634.
Combination of sensors of electrical activity of the brain with other sensors 3r3607. 3r3608.
 3r3634. Recently, I had the idea that EEG sensors should not be considered as something completely independent. Our brain perceives the world through the senses. He is not able to see without eyes and to hear without ears. Therefore, if we want to extract the maximum benefit from data on the electrical activity of the brain, we may need to monitor other vital signs.
 3r3634.
 3r3634. The main problem here is that all this can lead to the fact that people will be literally hung with various sensors.
 3r3634.
 3r3634. 3r32424. 3r33535. 3r33535.
 3r3634. 3r3-3597. 3r3-3598. Are there too many sensors? (source of illustration - cognionics.net)
3r3600.
 3r3634.
 3r3634. Perhaps no one will constantly wear the sensors shown in the previous figure.
 3r3634.
 3r3634.
OpenBCI 3r3607. 3r3608.
 3r3634. A few weeks ago, I acquired something new - the 3r33586 complex. OpenBCI 3r3627. . My next step is to study the raw data obtained from EEG sensors, and apply machine learning methods to this data. OpenBCI is an open source project, so their development seems to me perfectly suitable for this purpose. I still didn’t work much with their headset, so far I only had enough time to connect it to a computer and set it up. Here is how it all looks.
 3r3634.
 3r3634. 3r32424. 3r3-3593. 3r33535.
 3r3634. 3r3-3597. 3r3-3598. OpenBCI 3r3607. 3r3600.
 3r3634.
 3r3634.
Results 3r3607. 3r3608.
 3r3634. The author of this material says that he continues to study neural interfaces. Hopefully, her story will help those who are interested in this topic, but do not dare to begin practical actions, to take the first steps in the field of the use of neurohears. If all this is interesting to you - here is 3r3627. one more of our publication on neuroheaders and jаvascript dedicated to Muse.
 3r3634.
 3r3634. 3r31717. Dear readers! 3r3618. Do you plan to do experiments with neurohearnes?
 3r3634.
 3r3634.
3r32424. 3r33625. 3r33535.
3r33535. 3r3634. 3r3634. 3r3634. 3r33232. ! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r33333. 3r3634. 3r33535.
+ 0 -

Add comment