This course provides an introduction to data analytics for individuals with no prior knowledge of data science or machine learning. The course starts with an extensive review of probability theory as the language of uncertainty, discusses Monte Carlo sampling for uncertainty propagation, covers the basics of supervised (Bayesian generalized linear regression, logistic regression, Gaussian processes, deep neural networks, convolutional neural networks), unsupervised learning (k-means clustering, principal component analysis, Gaussian mixtures) and state space models (Kalman filters). The course also reviews the state-of-the-art in physics-informed deep learning and ends with a discussion of automated Bayesian inference using probabilistic programming (Markov chain Monte Carlo, sequential Monte Carlo, and variational inference). Throughout the course, the instructor follows a probabilistic perspective that highlights the first principles behind the presented methods with the ultimate goal of teaching the student how to create and fit their own models.

Syllabus

Please note: The summer 2022 session of this course will be a condensed 8-week course. The fall 2023 session will be the full 16 weeks.

Section 1: Introduction

Introduction to Predictive Modeling

Section 2: Review of Probability Theory

Basics of Probability Theory

Discrete Random Variables

Continuous Random Variables

Collections of Random Variables

Random Vectors

Section 3: Uncertainty Propagation

Basic Sampling

The Monte Carlo Method for Estimating Expectations

Monte Carlo Estimates of Various Statistics

Quantify Uncertainty in Monte Carlo Estimates

Section 4: Principles of Bayesian Inference

Selecting Prior Information

Analytical Examples of Bayesian Inference

Section 5: Supervised Learning: Linear Regression and Logistic Regression

Linear Regression Via Least Squares

Bayesian Linear Regression

Advanced Topics in Bayesian Linear Regression

Classification

Section 6: Unsupervised Learning

Clustering and Density Estimation

Dimensionality Reduction

Section 7: State-Space Models

State-Space Models – Filtering Basics

State-Space Models – Kalman Filters

Section 8: Gaussian Process Regression

Gaussian Process Regression – Priors on Function Spaces

Gaussian Process Regression – Conditioning on Data

Bayesian Global Optimization

Section 9: Neural Networks

Deep Neural Networks

Deep Neural Networks Continued

Physics-Informed Deep Neural Networks

Section 10: Advanced Methods for Characterizing Posteriors

Sampling Methods

Variational Inference

Institution: PurdueX

Subject: Engineering

Level: Advanced

Prerequisites:

Working knowledge of multivariate calculus and basic linear algebra

Basic Python knowledge

Knowledge of probability and numerical methods for engineering would be helpful, but not required

Language: English

Video Transcript: English

Associated skills: Unsupervised Learning, Sampling (Statistics), Bayesian Inference, Gaussian Process, Machine Learning, Markov Chain Monte Carlo, Physics, Probability Theories, Linear Regression, Data Analysis, Teaching, K-Means Clustering, Convolutional Neural Networks, Data Science, Deep Learning, Artificial Neural Networks, Principal Component Analysis, Logistic Regression, State Space, Propagation Of Uncertainty

What you\\\'ll learn

After completing this course, you will be able to:

Represent uncertainty in parameters in engineering or scientific models using probability theory

Propagate uncertainty through physical models to quantify the induced uncertainty in quantities of interest

Solve basic supervised learning tasks, such as: regression, classification, and filtering

Solve basic unsupervised learning tasks, such as: clustering, dimensionality reduction, and density estimation

Create new models that encode physical information and other causal assumptions

Calibrate arbitrary models using data

Apply various Python coding skills

Load and visualize data sets in Jupyter notebooks

Visualize uncertainty in Jupyter notebooks

Recognize basic Python software (e.g., Pandas, numpy, scipy, scikit-learn) and advanced Python software (e.g., pymc3, pytorch, pyrho, Tensorflow) commonly used in data analytics