Applications of the NuAmps Digital EEG Amplifier

We present two versatile software suites that interface with the 40-channel NuAmps Digital EEG Amplifier by Neuroscan. The manufacturer ships the hardware with Scan and/or Access SDK software. Both solutions allow to process the EEG data in real time:

Typically, the EEG and the real world are synchronized by sending 'stimulus codes' of values 0,1,...,255 via parallel port. For instance, when the subject is looking at a sequence of images, the onset of each image is accompanied by sending a trigger value, so that the EEG data can be related to the visual stimulus. The trigger values appear in a separate channel 41. Because the Access SDK does not forward the information, synchronization of EEG and the physical world has to be done in an alternative way: We devised a simple analog circuit with a photodiode and a microphone that translates optical or accustic stimuli into weak electronic pulses that fed into a spare channel of the EEG amplifier.

When using Scan 4.5, the configuration of the amplifier such as sampling rate, and filtering are selected in the software. Since the analog to digital conversion of the EEG amplifier has a 24 bit resolution, always select 32 bit mode!

On this webpage, we primarily investigate signal components in the EEG that originate from orienting the eyes, formally electrooculogram (EOG). In the past it was noted, that EOG exhibits an almost perfect linear correlation to the gaze for electrodes in the proximity of the eyes. However, this is only valid for short periods of time, as typically the baseline in the sensitive electronics drifts. We have developed a new scheme to compensate for the drift (see sections Breakout, Speller below).

The most prominent brain activity response to stimuli from the outside world is coined P300, because the response consists of a positive peak about 300 ms after the stimuli was presented. We report on the experiments conducted by Sarah Ting that compare different P300 for stimuli presented on an ordinary monitor, and on a 3D monitor.

NuAmps Digital EEG Amplifier Suite *	nuamps.zip	2.4 MB
Estimation of Gaze from EEG for Cursor Control	gaze_estimation...	1 MB

* requires MATLAB, Java, and Access Compumedics Neuroscan Real-Time SDK, or Scan 4.5 Acquire.

Credits: Hardware readout by Access SDK by Neisha Amy Chadwick with specifications in her thesis An investigation into the feasibility of an EEG-based data capture system for classification. The Breakout game engine was adapted from Per-Anders Ekström. Code for optimization l1-magic by Justin Romberg and others. Code for the export of figures by Oliver Woodford. Code for Scan 4.5 Acquire communication by Wang Chuanchu.

Setup

A screen will indicate live which channels are connected. A red dot symbolizes a disconnected channel. The other plots exhibit the spectrum and average signal. Usually there is 50 Hz, 150 Hz, ... powerline artifacts in the signal.

The peak at 50 Hz in the spectrum is rooted in the interference of the powerline. Sometime, there are peaks at 150, 250, ... Hz next to that.

Calibration 2D

Before launching the game, we let the participant undergo a calibration procedure of about 1 minute duration. During the calibration phase, the head is held stiff and the eyes follow the mouse which is moved across the screen. Subsequently, the computer linearly correlates the recordings with the trace of the mouse. This linear relation is used during the two minutes of the game to position the violet paddle at the bottom of the screen.

Breakout

The orientation of the eyes manifest in the signals of the electroencephalogram recorded in region of the forehead. In conventional applications of electroencephalography, these "visual artifacts" are undesired. However, we employ these signals to control the game Breakout using the electroencephalogram data exclusively. Breakout starts at 0:40 minutes:

The orientation of the eyes along the x-axis is roughly a linear combination of 2-9 channels of the electroencephalogram. For instance, F3-FT8 is a good estimation. However, two channels are insufficient to detect drift in the baselines in the channels. Presumably, the drift and variations in sensitivity are caused by the crystallization process of the conductive gel that connects the electrodes with the scalp. In our solution, we have used a linear combination of the channels: FT9, F8, X3, Fp2, FT10, Fp1, X2, C4, and ground GND.

Gameplay: the ball bounces from the paddle, the walls, and the bricks. Unless the ball has recently touched the floor, bricks are be removed upon contact with the ball. However, the participant might fail to control the paddle so that the ball touches the floor. Then, the ball turns red and does not remove bricks, until the ball bounces off the paddle the next time.

In a follow-up version, we provide feedback about the quality of the calibration procedure. The game is launched, whenever the correspondance of gaze and paddle position is established reasonably well. In some setups, we found the EEG signals to be subject to significant drift. In these cases, a recalibration procedure can be initiated at any time.

The results were presented in Dec 2010 at the EPSM-ABEC 2010 in Melbourne/Victoria under the title: A new paradigm to quantify the manifestation of eye movements in the electroencephalogram. Abstract: Eye movements have significant impact on electroencephalogram recordings. For medical diagnosis and brain computer interfaces based on EEG, the separation of signals originating from eye motions and brain activity are essential. In the past, the correlation of the orientation of eyes and the electric field across the scalp was established by employing an eye tracker alongside the EEG. However, the synchronization of the eye tracker and the EEG amplifier is not straightforward. To overcome this difficulty, we abandon the use of an eye tracker, and rely on the participants following a moving target cue on the monitor. Next to the moving cue, the display system periodically emits a trigger visual signal in a small designated area of the monitor. This flashing trigger signal is subsequently captured by an optical sensor and fed back to a channel of the EEG amplifier. We therefore achieve synchronization between the optical cue and the recorded EEG signals. We conduct twenty participant trials, in which the trace of the cue is fixed. We average the contribution of the eyes orientation to each EEG channel to obtain a transfer function. The inverse function combined with a calibration procedure of one minute duration yields the algorithm, that we propose to extract the signal component caused by the eye movements from the EEG channels. Our methodology greatly simplifies the experimental setup to quantify the effect of the eye orientation in EEG recordings. Our algorithm enables each participant to control the game Breakout just by moving their eyes.

Speller

The speller harvests the gaze, both vertical and horizontal, estimated from the EEG artifacts.

Any means to facilitate the spelling process are welcome: The software checks the current word beginning with the entries of the German-English dictionary Beolingus. In the screenshot to the right, the word beginning is gla, and the dictionary only finds words that continue with the letters b, c, d, m, n, r, s, u, and z. If there is no continuation of the word, the speller logs the current word and starts a new words. If there is only one continuation of the current word beginning, the speller adds the characters as long this can be done in a unique fashion.

Because the visual field of interest is the limited area of the monitor, we can compensate drift in the electronics by detecting outliers of the gaze estimate. The user is able to control our application by simply orienting the gaze, which is comfortable even over extended periods of time. A long term demonstration of the speller is here: [part 1] [part 2] [part 3] The user spells the last verse of Rudyard Kipling's poem "If—" (1865-1936).

The verse consists of 358 characters (including spaces), which are spelled over a duration of 27 minutes. This is equivalent to 13.25 characters per minute with an accuracy of 100%. A calibration procedure of 1 minute duration preceeds the spelling.

3D gaze estimation

We are also interested how 3D vision impacts the EEG. To create the illusion of depth perception for the subjects, our lab uses a True3Di monitor SDM-240 with 2 screens at a resolution of 1920 x 1080.

Eye blink detection to enhance estimation of gaze

The huge fraction of the error in the estimation of gaze originates from eye blink. The video to the side shows that the detection eye blinks and setting the estimation during eye blinks constant, the estimation error reduces by up to 50%.

P300

Sarah Ting used the code and hardware to perform P300 experiments. The results are of very high quality, possibly also due to the precise timing mechanism.

The main result is that the rate of occlusion of the target object has more impact on the P300 response than whether the object is presented in 2D or 3D.

Suplemental hardware

The Neuroscan amplifier comes with a serial port trigger. However, there is no guarantee that the serial port communication is in-sync with the visual presentation of the stimuli.

To achieve precise timing, we have developed two simple sensor circuits to synchronize the EEG signals with the real world events:

To register to Scan 4.5 server, send three packets of 12 bytes each - see haxadecimal values to the right. The first four bytes of each packet correspond to the letters 'CTRL'.	43 54 52 4C 00 02 00 01 00 00 00 00 43 54 52 4C 00 03 00 05 00 00 00 00 43 54 52 4C 00 03 00 03 00 00 00 00
After registration, the server will send packets of EEG data.
To gracefully disconnect from Scan 4.5 server, send three packets of 12 bytes each (values in hexadecimal notation):	43 54 52 4C 00 03 00 04 00 00 00 00 43 54 52 4C 00 02 00 02 00 00 00 00 43 54 52 4C 00 01 00 02 00 00 00 00