Intelligent robot chair with communication aid using TEP responses and higher order spectra band features

In recent years, electroencephalography-based navigation and communication systems for differentially enabled communities have been progressively receiving more attention. To provide a navigation system with a communication aid, a customized protocol using thought evoked potentials has been proposed in this research work to aid the differentially enabled communities. This study presents the higher order spectra based features to categorize seven basic tasks that include Forward, Left, Right, Yes, NO, Help and Relax; that can be used for navigating a robot chair and also for communications using an oddball paradigm. The proposed system records the eight-channel wireless electroencephalography signal from ten subjects while the subject was perceiving seven different tasks. The recorded brain wave signals are pre-processed to remove the interference waveforms and segmented into six frequency band signals, i. e. Delta, Theta, Alpha, Beta, Gamma 1-1 and Gamma 2. The frequency band signals are segmented into frame samples of equal length and are used to extract the features using bispectrum estimation. Further, statistical features such as the average value of bispectral magnitude and entropy using the bispectrum field are extracted and formed as a feature set. The extracted feature sets are tenfold cross validated using multilayer neural network classifier. From the results, it is observed that the entropy of bispectral magnitude feature based classifier model has the maximum classification accuracy of 84.71 % and the value of the bispectral magnitude feature based classifier model has the minimum classification accuracy of 68.52 %.

Introduction. Movement and communication are the basic needs of human beings in their daily life and to live a meaningful life with interpersonal interactions [1]. Neuromuscular Disorder patients, such as Amyotrophic Lateral Sclerosis, neurodegenerative disease, muscular dystrophy, high cervical injuries or loss of the ability to speak (due to an accident) and Brain Stem Stroke have their walking abnormalities due to postural Instability and difficulty in communication with others due to loss of muscle control and speech [2][3][4]. Over the last decade, there has been an increasing attention on these patients to provide a navigation system and communication aid to enable them to lead a normal life [5][6][7][8][9]. In recent years, variety of BMI applications have arisen, Encouraged by new understanding of the non-invasive acquisition of human perception using powerful EEG amplifiers [10,11], e. g. for cursor movement, acupuncture in pain relief [12], Neuro-prosthetic arm [13] and whole body movement [14], driver sleepiness detection [15], smart-living environmental control [16]. Currently, this research has been directed towards wheelchair navigation control and recognition of unspoken speech utterances without voluntary muscle activity [17][18][19][20].
Recently, several studies have examined thought evoked potential (TEP) based design of robotic wheelchair control using human thoughts [21], and communication systems using P300 speller and oddball paradigms [22]. Yet, the data acquisition protocols have shown a vital role in redefining the claimed action to command a navigation system or a communication system. The research work proposed by Kaufmann et al [17], involves positioning of four tactile stimulators and delivers navigation by concentrating their considerations on the desired tactile stimulus in an oddball paradigm to control the wheelchair. The results were validated through the participants navigating a virtual wheelchair. Theresa M. Vaughan [23], Frank H. Guenther [19] and Anne Porbadnigk [20] have developed several alternative communication systems using the recent developments in personal computers and new prosthetic methods to provide communication and control channels to individuals with difficulties in communication. Despite, none of the systems have produced an expanded utilization of the BMI technology to facilitate both navigation and communication through a customized brain activity recording protocol. Thus, in this study it is proposed to develop a customized thought controlled intelligent robot chair with communication aid (IRCC), as an initial step towards the possibility of navigation and speech production using a simple thought response based protocol ( fig. 1). Depicts the block diagram of the proposed customized classification system for robot chair control along with a communication aid. The motivation towards this research is to establish a simple robot chair along with a communication aid, that can be used by an differentially enabled person, to control a wheel chair and to communicate their needs with others using TEP's. A simple data acquisition protocol has been proposed to develop the thought controlled IRCC; the tasks (Forward, Left, Right, Yes, No and Relax) were initially simulated and the subjects were requested to imagine during the data acquisition process. Further, the subjects were taught to pronounce the word loudly for the 'Help' task. The EEG signals are recorded for 12 sec. for each trial per task and are segmented into 10 sec. during the pre-processing stage for uniformity. In the pre-processing, the recorded brain wave signals were band-passed filtered in the frequency range of 0.5 to 100 Hz and segmented into six frequency bands Delta ( ) δ , theta ( ) θ , alpha ( ) α , beta ( ) β , Gamma 1 ( ) γ1 and Gamma 2 ( ) γ2 . Thus, frequency band signals are segmented into frame segments (512 samples) and used to extract the features using higher order spectra (HOS) technique. The general motivation behind the use of bispectrum estimation is to detect and characterize the nonlinear properties of the TEP tasks, and they are potentially better to estimate the deviations from Gaussianness (normality) [24][25][26]. Thus in this study, the third order statistics bispectrum based feature extraction method has been implemented to extract the features from each frame of frequency band signals over each electrode position and the features such as the Mean of bispectral magnitude (M) and the bispectral entropy features (E) were extracted. The non-linear features were extracted and associated with the corresponding TEP tasks. Then, the extracted feature sets were modeled using a supervised learning-multilayer neural network (MLNN) classifier and the classification performance was validated. The research methodology and the developed model results are explained in the subsequent sections of this paper.
Intelligent robot chair with communication database. The experimental setup and data acquisition procedures were implemented in the research lab at the School of Mechatronic Engineering, University Malaysia Perlis. The proposed study has been registered and approved from National Medical Research Registration (NMRR ID: NMRR-13-51-14570) and obtained Ethical approval from The Medical Research & Ethics Committee (MREC), Ministry of Health Malaysia. (Ref: (7)dlm.KKM/NIHSEC/800-2/2/2Jld2P13-179). This section elucidates some fundamental methods on the experimental setup which includes the wireless bio-amplifier setup and the placement of electrode channels for brain wave recording. Further, Suitable task selection, the data collection procedure and the formation of IRCC database were also presented. These processes are essential for the classification of thought evoked potentials to command an intelligent robot chair with communication aid.
Experimental setup and data acquisition tasks. In the experimental setup, a standard bio-signal acquisition system developed by 'g-mobilab+' 8-channel EEG data acquisition system was used to record the brain wave signals [27,28]. The system consists of an electrode cap with nine differential screwable electrodes, bio-signal amplifier and wireless data acquisition using MATLAB® interactive programming environment. In this study, it is proposed to develop a BMI system which can be used to navigate the wheel chair and communicate with others using an oddball paradigm; through brain wave EEG signals when functional communications are disabled [29]. Thus, in the data acquisition protocol, three primary tasks that address the navigation of the robot chair and to select the isolated words in an oddball paradigm, such as Left, Forward and Right hand movement control are included. Further, three additional tasks have been included to use in emergency circumstances and to address the basic needs of a human being, they are Help, Yes, No tasks. Relax (normal) has been used as the reference signal in this experiment [30]. A semi-sound controlled room was used for the acquisition where the subjects were remained in a pleasant circumstance. The subject carried out seven different tasks. The EEG signals are recorded while the subject was settled comfortably and remained in totally static posture. No overt actions were made during the 12.0 sec. of the data acquisition process ( fig. 2) depicts the tasks that were implemented using the TEP responses to command an IRCC. The system records the motor imaginary signal from the eight electrode positions such as Temporal (T3, T4), central (C3, C4), parietal (P3, P4), and occipital (O1, O2) while the subjects were performing the seven thought response tasks. In the electrode placement system, reference recording schemes were used. The electrodes are placed on T3, T4, C3, C4, P3, P4, O1 and O2 position with one common electrode on the left ear lobe of the body where potential remains fairly constant [31,32]. The proposed IRCC system captures the brain wave patterns in order to identify the rhythmic activity for the seven different thoughts of an individual. Thus, during data collection, the EEG signals were recorded at a sampling rate of 256 Hz from a grid of 8 Ag/AgCl scalp electrodes which were placed on the scalp according to the international 10-20 lead system [31,32]. The electrodes are placed on the scalp of the selected locations and tested for level of impedance using g-tec impedance checker.
The impedance level was also tested after completing and maintained below 10 KΩ.
TEP data acquisition and IRCC database. In the data acquisition process, ten healthy BMI-naive volunteers (eight male, aged 21-30 years and two female, aged 24 years) were participated. During the data acquisition of each task, the subjects were requested to view the simulation of the specific task on the LCD monitor as depicted in fig. 2(1) to fig. 2(7) until recording all the trials. The simulation depicts the movement of a joystick moving left, forward and right movement for the left, forward and right directions respectively. For the additional tasks like 'Yes' and 'No', the simulation presents a volunteer performing head movements up-down and left-right movements and for 'Help' task, the subject was requested to pronounce imaginarily the word 'help' respectively. Then, the subject was requested to imagine the tasks as simulated on the monitor. When, the subject performs a specified task, the EEG signals emanated were recorded for 12.0 sec. from Parietal (P3 and P4), temporal (T3, T4), central (C3, C4), occipital (O1, O2), positions. Ground electrode and reference electrodes are placed in the Fpz position and left earlobe locations in order to make individual's thought evoked tasks, recording comparable over time and to another individual's record, International 10-20 system was used for the electrode placement [28,31,33].
The procedure of thought stimulus took the following format: 1. A simulation was presented on the LCD monitor (Left task) for 10 sec. 2. A 1 KHz tone (beep) sounded, the monitor displays the simulation of a moving joystick in left direction.
3. Then the monitor is turned off, the subject was given a time break of 5 sec. and requested to imagine the respective task.
4. The bio-signal recording was carried out for 12 sec. while the subjects performing the task. 5. The subject was given a time break of approximately two minutes after completing each trial. 6. The simulation continued until ten trials were performed. 7. Similarly, the next task simulation in the procedure was presented. *Note that eyes remain open in all the mental tasks. The recorded EEG signals are contaminated with unknown noise component lying within a 50-60 Hz frequency range, which are due to power-line noises. A simple first order IIR notch filter was designed for removing the Power line noise from the recorded EEG signals. The center frequency of the filter, F0 was chosen to be at exactly 50 Hz and the bandwidth, ΔF = 4 Hz. Then, the signals are converted into digital signals using a sampling frequency of 256 Hz simultaneously, the acquisition process was repeated 10 times for each task and the subject was requested to take a rest for ten minutes after each task. Similarly, this procedure was repeated for ten subjects and the recorded signals were combined and a database was formulated. The database was named as IRCC database. The IRCC database consists of data pertaining to 10 different subjects (for 7 tasks and each was performed for 10 trials). The collected database was validated using analysis of variance (ANNOVA) technique and the significance level of 4.29×10 -4 <p value was obtained when validated on a task basis.
Thus, each frequency band signals are segmented into frames such that a frame length of 2 sec. having 512 samples per frame along with an overlap of 1s (m = 256 samples). Thus, the first frame consists of n = 512 samples. The second frame was initiated after a lap of m-1 samples such that the second frame overlaps with the n-m samples of the first frame. This procedure was repeated until all the frequency band signals were counted. Then, each frame is considered as an input to extract the higher order spectra also known as polyspectral representations of higher order statistics.
Bispectrum estimation. In various BMI applications, EEG signals have been analyzed using power spectra in several distinctive frequency bands. The power spectrum estimation provides the good statistical description of signals with an arbitrary distribution function. Moreover, power spectrum representation gives us the full canonical description in the case of stationary signals. In case of nongaussianity or non-linear mechanisms, Higher order spectra can be used to determine the higher order moments or complaints which provide additional information on the phase characteristics and realistic information of the EEG signal [24]. In this paper, bispectrum B(f1, f2) analysis has been employed to study the brain wave patterns of the visual stimuli. The bispectrum estimation is particularly the thirdorder statistics of a signal, which represents the Fourier transform of the third order correlation with highly interdependent frequency components [24,35]. The mathematical representation of the bispectrum estimation is expressed in equation where X(f) is the DFT at frequency samples x(nT), using the FFT algorithm. The frequency (f) may be normalized by the Nyquist frequency to be between 0 and 1. * ( 1 + 2 ) denotes complex conjugate and therefore the bispectrum obtained using equation (1) is a complex valued function which represents the product of three Fourier coefficients. In this feature extraction process, the nonredundant region or the positive bispectrum sequence (Ω) = 0 ≤ 2 ≤ 1 ≤ ( 1 + 2 ) ≤ 1 has been used to extract the Mean of bispectral magnitude features and the whole bispectrum region of computation has been used to extract the and grand mean of the bispectral magnitude features respectively.
From the estimated sequences, two statistical features, namely, Mean of bispectral magnitude ( ) and bispectral entropy ( ) features are computed to characterize the distribution of bispectrum sequence as well as to minimize the dimension of the feature vector as shown in equation (2) where M and E represents the mean and entropy (bispectral magnitude of the Ω ) in the i th frame of the j th electrode channel position. 1 2 , Ω region of the bispectral magnitude. , Classification of imaginary tasks using MLNN. MLNN are biologically inspired tools used for information processing and they are nonlinear in nature [36]. Classification of TEP responses to categorize the navigational tasks basically falls on pattern recognition problem. In this analysis, generalized IRCC system has been developed using MLNN for Multi-class pattern classification. The feature vectors derived from the mean of bispectral magnitude (5600×48 feature vectors) and bispectral entropy (5600×48 feature vectors) are processed subsequently and then associated with the seven different visual response tasks. Also, the feature vectors are normalized using binary normalization methods, where the dataset is recycled between 0.1 to 0.9 and partitioned into training and testing sets [37]. The training set has 4480×48 (80 % of master data set) and the testing set has the remaining 1120×48 (20 % of master data set) for the classification of the TEP tasks.
In this work, the MLNN models were organized with 48 input neurons, 25 hidden neurons and three neurons in the output layer. As the logistic sigmoid function scales any range of values between 0.1 and 0.9, in the designed MLNN models, logistic sigmoidal function was used to activate the neurons in the hidden and output layer. The Mean Squared Error (MSE) tolerance of 0.1 was used for training the neural network. In order to improve the performance rate, the learning rate, momentum factor and number of iterations were chosen based on the experimental observations in different trials. The learning rate and momentum factor for the models were chosen as 0.1 and 0.8 respectively. The generalization capability of the model was validated by performing 10 trials for training and testing method. The network models were trained using Levenburg Marquth Model. The MLNN model for spectral features were trained with ten trial weights for each subset. On the first subset, the network model was trained using 9/10 of the feature set and the classification rate was estimated using 1/10 subset of the remaining feature set. This process was repeated until all the 2/10 subset are used for the validation set [36,37]. Further, the network training parameters, mean classification rate are shown in table 1, 2. Table 1 The mean classification performance of the IRCC system using MLNN classifier and the mean of bispectral magnitude features     fig. 3, it is observed that the MLNN model based on the bispectral features has the mean training time in the range of 1096 to 1366 sec. using the mean of bispectral magnitude subset of testing set and the bispectral entropy feature set has the mean training time in the range of 1140 to 1519 sec. respectively. It is also observed that the mean training time of 1240 sec. has been obtained using the mean of bispectral magnitude features and 1519 sec. has been obtained using bispectral entropy features. The mean maximum training time of 1519 sec. has been obtained from bispectral entropy features and mean minimum training time of 1096 sec. has been obtained for the mean of the bispectral magnitude features.  fig. 4, it is observed that the MLNN model based on the bispectral features has the mean number of epochs in the range of 137 to 166 epochs using the mean of bispectral magnitude subset of testing set and the bispectral entropy feature set has the mean number of epochs in the range of 142 to 211 epochs respectively. It is also observed that the mean number of epochs of 154 epochs has been obtained using the mean of bispectral magnitude features and 161 epochs has been obtained using the bispectral entropy features. The mean maximum number of epochs of 211 epochs has been obtained from bispectral entropy features and the mean minimum number of epochs of 137 epochs has been obtained for the mean of bispectral magnitude features.  fig. 5, it is observed that the MLNN model based on the bispectral features has the mean classification accuracy in the range of 68.52 to 82.95 % using the mean of bispectral magnitude subset of testing set and the bispectral entropy feature set has the mean classification accuracy in the range of 71.24 to 84.71 % respectively. It is also observed that the mean classification accuracy of 76.97 % has been obtained using the mean of the bispectral magnitude features and 78.57 % has been obtained using the bispectral entropy features. The mean max classification accuracy of 84.71 % has been obtained from the bispectral entropy features and the mean minimum classification accuracy of 68.52 % has been obtained for the mean of bispectral magnitude features.
Conclusion. The regards to the objective of this research work, a simple thought controlled intelligent robot chair with communication aid has been developed using statistical features of the bispectrum estimation and MLNN algorithm. The proposed system uses the TEP response task signals recorded from ten subjects and are segmented into six frequency band has been chosen to study the third order Fourier coefficient of the TEP tasks. Then, statistical features such as the mean of bispectral magnitude using the non-redundant region (Ω) = 0 ≤ 2 ≤ 1 ≤ ( 1 + 2 ) ≤ 1 and grand mean of the bispectral magnitude using the bispectrum region of computation are extracted. The extracted feature vectors based on third order higher order spectra features (mean and grand mean of bispectral magnitude) are distinguished easily for the different classes of TEP tasks. The feature vectors are associated with the corresponding output targets and are classified using MLNN classifiers.
The test results obtained from this analysis has a less misclassification error of 11.60 % (130/1120) samples during the testing stage. The obtained results open many possible areas of applications and improvements in thought controlled robot chair navigation and communication system for differentially enabled communities. In the future analysis, non-linear feature extraction algorithms, classification algorithms and online training sessions so as to be used to improve the recognition accuracy of the IRCC system. Further, it is propitious to explore useful characteristics of brain wave signals based on effective feature extraction and classification methods.