Multimodal Autoencoder: A Deep Learning Approach to Filling In Missing Sensor Data and Enabling Better Mood Prediction
2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)
Authors: Jaques, Natasha; Taylor, Sara; Sano, Akane; Picard, Rosalind
To accomplish forecasting of mood in real-world situations, affective computing systems need to collect and learn from multimodal data collected over weeks or months of daily use. Such systems are likely to encounter frequent data loss, e.g. when a phone loses location access, or when a sensor is recharging. Lost data can handicap classifiers trained with all modalities present in the data. This paper describes a new technique for handling missing multimodal data using a specialized denoising autoencoder: the Multimodal Autoencoder (MMAE). Empirical results from over 200 participants and 5500 days of data demonstrate that the MMAE is able to predict the feature values from multiple missing modalities more accurately than reconstruction methods such as principal components analysis (PCA). We discuss several practical benefits of the MMAE's encoding and show that it can provide robust mood prediction even when up to three quarters of the data sources are lost.
Multiple Model Adaptive Estimation with Filter Spawning
PROCEEDINGS OF THE 2000 AMERICAN CONTROL CONFERENCE, VOLS 1-6
Authors: Fisher, KA; Maybeck, PS
Multiple Model Adaptive Estimation (MMAE) with Filter spawning is used to detect and estimate partial actuator failures on the VISTA F-16. The truth model is a full six-degree-of-freedom simulation provided by Calspan and General Dynamics. The design models Ire chosen as 13-state linearize models, including first order actuator models. Actuator failures are incorporated into the truth model and design model assuming a "failure to free stream". Filter spawning is used to include additional filters with partial actuator failures hypotheses into the MMAE bank. The spawned filters are based on varying degrees of partial failures tin terms of effectiveness) associated with the complete-actuator-failure hypothesis with the highest conditional probability of correctness at the current time. Thus, a blended estimate of the failure effectiveness is found using the filters' estimates based upon a no-failure hypothesis, a complete actuator failure hypothesis, and the spawned filters' partial-failure hypotheses. This yields substantial precision in effectiveness estimation, compared to what is possible without spawning additional filters, making partial failure adaptation a viable methodology in a manner heretofore unachieved.