Speakers
Graph-Guided Networks for Complex Time Series
In many domains, including healthcare, biology, and climate science, time series are irregularly sampled with varying time intervals between successive readouts and different subsets of variables (sensors) observed at different time points. For example, sensors' observations might not be aligned, time intervals among adjacent readouts can vary across sensors, different samples can have varying numbers of readouts recorded at different times. While machine learning methods usually assume fully observable and fixed-size inputs, irregularly sampled time series raise considerable challenges. In this talk, I will introduce Raindrop, a graph neural network that embeds irregularly sampled and multivariate time series while also learning the dynamics of sensors purely from observational data. Raindrop represents every sample as a separate graph, where nodes indicate sensors, and time-varying dependencies between sensors are modeled by a novel message passing operator. I will describe applications of Raindrop to classify time series and interpret temporal dynamics on three healthcare and human activity datasets. Raindrop shows superiority on multiple setups, including challenging leave-sensor-out settings.
Bio
Marinka Zitnik (https://zitniklab.hms.harvard.edu) is an Assistant Professor at Harvard with appointments in the Department of Biomedical Informatics, Broad Institute of MIT and Harvard, and Harvard Data Science. Her research investigates applied machine learning, focusing on networked systems that require infusing structure and knowledge. Dr. Zitnik has published extensively in ML venues and leading scientific journals. She is an ELLIS Scholar in the European Laboratory for Learning and Intelligent Systems (ELLIS) Society and a member of the Science Working Group at NASA Space Biology. Her research won best paper and research awards from the International Society for Computational Biology, Bayer Early Excellence in Science Award, Amazon Faculty Research Award, Roche Alliance with Distinguished Scientists Award, Rising Star Award in EECS, and Next Generation in Biomedicine Recognition, being the only young scientist with such recognition in both EECS and Biomedicine.
Representation and Generation of Clinical Time-Series: Challenges and Opportunities
Multivariate time-series data are frequently observed in various healthcare applications where they are typically characterized by sparsity (missing information) and irregular time intervals. Existing approaches for learning representations in this domain handle these challenges by either aggregation or imputation of values, which in turn suppresses the fine-grained information and adds undesirable noise/overhead into the machine learning model. To tackle this problem, we developed a Self-supervised Transformer for Time-Series (STraTS) model which overcomes these pitfalls by treating time-series as a set of observation triplets instead of using the standard dense matrix representation. Our approach employs a transformer model with a novel Continuous Value Embedding technique to encode continuous time and variable values without the need for discretization. To tackle the problem of limited availability of labeled data, STraTS utilizes self-supervision by leveraging unlabeled data to learn better representations by using time-series forecasting as an auxiliary proxy task. Our results on real-world multivariate clinical time-series benchmark datasets which demonstrate STraTS has better prediction performance than state-of-the-art methods for mortality prediction, especially when labeled data is limited. Towards the end of the talk, I will also describe our recent work on generating medical records using a combination of generative adversarial networks and gated recurrent units.
Bio
Chandan Reddy is a Professor in the Department of Computer Science at Virginia Tech. He received his Ph.D. from Cornell University and M.S. from Michigan State University. His primary research interests are Machine Learning and Data Analytics with applications to healthcare, e-commerce, and transportation. His research has been funded by NSF, NIH, DOE, DOT and various industries. He has published over 150 peer-reviewed articles in leading conferences and journals. He received several awards for his research work including the best paper awards at SIGKDD and ICDM conferences, and was a finalist of the INFORMS Franz Edelman Award Competition in 2011. He is currently serving on the editorial boards of ACM TKDD, ACM TIST, and IEEE Big Data journals. He is a senior member of the IEEE and a distinguished member of the ACM. More information about his work is available at his homepage: http://www.cs.vt.edu/~reddy/
Signals, Structure, and Dynamics: Unraveling the Unknown in Complex Systems
With the decreasing hardware cost and increasing demand for autonomic management, many physical systems nowadays are equipped with a large network of sensors, which generate a huge amount of time series data every day. Due to the high system complexity, however, those time series contain heterogeneous dependencies among different parts across the system and are mixed with noise and operational patterns. It is a challenge to correctly discover the system operational status and healthiness from the collected data. In this talk, I will share our work in transforming the time series data into insight for system’s productivity, reliability and safety. I will start with discovering the “invariants” from massive time series and leveraging those “invariants” for system management. I will then present the deep learning based time series retrieval to digest and compress historical time series for explaining current observations. Finally, I will talk about our work in leveraging the attention mechanism and the prototype learning to discover important patterns from time series. If time is allowed, I will also introduce our recent work that incorporates text data to further interpret the behavior of time series.
Bio
Dr. Haifeng Chen is heading the Data Science and System Security Department at NEC Laboratories America in Princeton, New Jersey. He received the bachelor’s and master’s degrees in automation from Southeast University China, and the Ph.D degree in computer engineering from Rutgers University in 2004. He and his team members are working on various topics related to big data analytics, AI, software and system security, smart service and platforms. Dr. Chen has served in the program committee for a number of top AI conferences, and has been in the panel of National Science Foundation (NSF) programs. He is a member of the school of Systems and Enterprises Advisory Board in Stevens Institute of technology, New Jersey. Dr. Chen has co-authored more than a hundred conference/journal publications including the best paper runner-up at SigKDD’16, and has over 70 patents granted. Most of his research led to advanced solutions and products for various industrial domains including power plants, satellite, financial, retail, and so on.
Christopher Rackauckas
Director of Modeling and Simulation, Julia Computing, Director of Scientific Research, Pumas-AI, Research Affiliate, Co-PI of the Julia Lab Massachusetts Institute of Technology
Improving Forecasting by Merging Deep Learning with Mechanistic Modeling
Differentiable simulation techniques are the core of scientific machine learning methods which are used in the automatic discovery of mechanistic models through infusing neural network training into the simulation process. In this talk we will start by showcasing some of the ways that differentiable simulation is being used, from discovery of extrapolatory epidemic models to nonlinear mixed effects models in pharmacology. From there, we will discuss the computational techniques behind the training process, focusing on the numerical issues involved in handling differentiation of highly stiff and chaotic systems. The viewers will leave with an understanding of how compiler techniques are being infused into the simulation stack to provide the future of differentiable simulation which merges machine learning with traditional biological and physical modeling.
Bio
Chris is the Director of Scientific Research at Pumas-AI, the Director of Modeling and Simulation at Julia Computing, Co-PI of the Julia Lab at MIT, and the lead developer of the SciML Open Source Software Organization. He is the lead developer of the Pumas project and has received a top presentation award at every ACoP in the last 3 years for improving methods for uncertainty quantification, automated GPU acceleration of nonlinear mixed effects modeling (NLME), and machine learning assisted construction of NLME models with DeepNLME. For these achievements, Chris received the Emerging Scientist award from ISoP. For his work in mechanistic machine learning, his work is credited for the 15,000x acceleration of NASA Launch Services simulations and recently demonstrated a 60x-570x acceleration over Modelica tools in HVAC simulation, earning Chris the US Air Force Artificial Intelligence Accelerator Scientific Excellence Award.
Interpretable Mixture of Experts for Structured Data
With the growth of machine learning for structured data, the need for reliable model explanations is essential, especially in high-stakes applications. We introduce a novel framework, Interpretable Mixture of Experts (IME), that provides interpretability for structured data while preserving accuracy. IME consists of an assignment module and a mixture of interpretable experts such as linear models where each sample is assigned to a single interpretable expert. This results in an inherently-interpretable architecture where the explanations produced by IME are the exact descriptions of how the prediction is computed. In addition to constituting a standalone inherently-interpretable architecture, an additional IME capability is that it can be integrated with existing Deep Neural Networks (DNNs) to offer interpretability to a subset of samples while maintaining the accuracy of the DNNs. Experiments on various structured datasets demonstrate that IME is more accurate than a single interpretable model and performs comparably to existing state-of-the-art deep learning models in terms of accuracy while providing faithful explanations.
Bio
Sercan Arik is currently working as a Staff Research Scientist and Manager at Google Cloud AI Research. His current work is motivated by the mission of democratizing AI and bringing it to the most impactful use cases, from Healthcare, Finance, Technology, Retail, Media, Manufacturing, and many other industries. Towards this goal, he focuses on how to make AI more high-performance for the most-demanded data types, interpretable, trustable, data-efficient, robust and reliable. He led research projects that were launched as major Google Cloud products and yielded significant business impact. Before joining Google, he was a Research Scientist at Baidu Silicon Valley AI Lab. At Baidu, I have focused on deep learning research, particularly for applications in human-technology interfaces. He co-developed state-of-the-art speech synthesis, keyword spotting, voice cloning, and neural architecture search systems. Sercan Arik completed his PhD degree in Electrical Engineering at Stanford University.
RNNs for Time Series Forecasting
In a typical business environment, with new product introductions, new datacenters starting, etc., time series are of uneven length, many are very short. Recurrent Neural Networks (RNNs) are well suited to learn from and forecast these kind of data sets, as in contradistinction to CNNs, Transformers, etc., they have memory. The talk will describe a basic well performing RNN system based on LSTMs, and then describe more advanced versions with improved cells and network architecture. Finally, a relatively simple modification will be described that allows to train an RNN to predict any quantile on demand, providing a kind of probabilistic forecast.
Bio
Slawek Smyl received M.Sc. degree in Physics from Jagiellonian University, Poland, and the M.Eng. degree in Information Technology from RMIT University, Australia. He is currently a Quantitative Engineer with Meta Technologies working in the area of time series forecasting. Mr. Smyl has ranked highly in forecasting competition: he won the Computational Intelligence in Forecasting International Time Series Competition 2016, got a third place in the Global Energy Forecasting Competition in 2017, and won the M4 Forecasting Competition in 2018.
