Decoding Sign Language Finger Movements from high-density ECoG using Graph-Optimized Block Term Tensor Regression

Axel Faes, Mariana P. Branco, Anais Van Hoylandt, Elina Keirse, Tom Theys, Nick F. Ramsey, Marc M. Van Hulle

Submitted to Journal of Neural Engineering (5.4 IF) (Under review) 2024

Objective A novel method is introduced to regress over the Sign Language finger movements from human electrocorticography (ECoG) recordings. Approach The proposed Graph-Optimized Block-Term Tensor Regression (Go-BTTR) method consists of two components: a deflation-based regression model that sequentially Tucker-decomposes multiway ECoG data into a series of blocks, and a Causal Graph Process (CGP) that accounts for the complex relationship between finger movements when expressing sign language gestures. Prior to each regression block, CGP is applied to decide which fingers should be kept separate or grouped and should therefore be referred to BTTR or its extended version eBTTR, respectively. Main results Two ECoG datasets were used, one recorded in 5 patients expressing 4 hand gestures of the American Sign Language Alphabet, and another in 2 patients expressing all gestures of the Flemish Sign Language Alphabet. As Go-BTTR combines fingers in a flexible way, it can better account for the nonlinear relationship ECoG exhibits when expressing hand gestures, including unintentional finger co-activations. This is reflected by the superior joint finger trajectory predictions compared to eBTTR, and predictions that are on par with BTTR in single finger scenarios. Subject 5, Dataset 1, during the movement of hand gestures reached 0.82 Pearson correlation for the pinky for Go-BTTR compared to 0.8 and 0.72 for eBTTR and BTTR respectively. Significance Our findings show that Go-BTTR is capable of decoding complex hand gestures taken from the Sign Language Alphabet. Go-BTTR also demonstrates computational efficiency, providing a notable benefit when intracranial electrodes are inserted during a patient’s pre-surgical evaluation. This efficiency helps reduce the time required for developing and testing a Brain-Computer Interface (BCI) solution.

Federated Block-Term Tensor Regression for Heart Diseases in Realistic Healthcare Settings

Axel Faes, Liesbet Peeters

2024 Spotlight

This paper presents an extension of the Block-Term Tensor Regression (BTTR) algorithm, termed Federated Block-Term Tensor Regression (FBTTR), designed to address the unique challenges of heart disease prediction in realistic healthcare settings. Traditional machine learning models often rely on centralized data, which poses significant privacy risks and limits the scalability of predictive models across multiple institutions. FBTTR adapts BTTR to federated learning scenarios, allowing decentralized data from various healthcare institutions to be utilized without compromising patient privacy. The proposed FBTTR method leverages the strengths of tensor regression in handling multi-dimensional data, enhancing predictive accuracy through collaborative learning while ensuring data confidentiality. By incorporating federated learning principles, FBTTR facilitates secure model training across different sites, effectively mitigating data silos and promoting a more comprehensive understanding of heart disease predictors. Empirical evaluations demonstrate the robustness of FBTTR, showing that it achieves high predictive performance comparable to traditional centralized models. Moreover, the algorithm exhibits efficient computational performance, making it suitable for deployment in real-world healthcare environments where timely and accurate predictions are crucial. The results highlight FBTTR's potential in advancing predictive analytics in healthcare by providing a scalable, privacy-preserving solution for heart disease prediction, ultimately contributing to better patient outcomes and more effective healthcare delivery.

Finger movement and coactivation predicted from intracranial brain activity using extended Block-Term Tensor Regression

Axel Faes, Marc M. Van Hulle

Journal of Neural Engineering (5.4 IF) 2023 Spotlight

Multiway- or tensor-based decoding techniques for Brain-Computer Interfaces (BCI) are believed to better account for the multilinear structure of brain signals than conventional vector- or matrix-based ones. However, despite their outlook on significant performance gains, the used parameter optimization approach is often too computationally demanding, so conventional techniques are still preferred. We propose two novel tensor factorizations which we integrate into our Block-Term Tensor Regression (BTTR) algorithm and further introduce a marginalization procedure that guarantees robust predictions while reducing the risk of overfitting (generalized regression). BTTR accounts for the underlying (hidden) data structure in a fully automatic and computationally-efficient manner, leading to a significant performance gain over conventional vector- or matrix-based techniques in a challenging real-world application. As a challenging real-world application, we apply BTTR to accurately predict single finger movement trajectories from intracranial recordings in human subjects. We compare the obtained performance with that of the state-of-the-art.

[Link]