Temporal Multimodal Multivariate Learning

Unobserved heterogeneity and randomness of the data have shown multiple modes in probability distribution in each location in a geographic region. A huge collection of multiple sources of granular microscopic data in each location may result in the loss of multivariate information if not retrieved properly. It is important to quantify the uncertainty of prior belief and update posterior when critical observations are obtained. However, traditional entropy theory cannot handle i) sequential learning multimodal multivariate information; ii) dynamic spatiotemporal correlation; iii) importance of observation for posterior approximation. In this paper, we develop a Spatiotemporal Information-retrieval through a Multi-variate Multi-modal Learning (SIMi) framework to address the above challenges by analyzing the similarities between the different types of mixture distributions of multiple variables in each cell and allocating a cluster to each cell across high dimensional time and space. Specifically, SIMi can track structural information flow of temporal multi-variate multi-modal correlation data and automatically update the posterior, with the weights to the new observations through an iterative process. Extensive experiments on real-world datasets, i.e., hurricane ensemble forecasting data and urban traffic data, demonstrate the superior performance of our SIMi method over the state-of-the-art baselines across various settings.