The course presents algorithms and advanced methods in machine learning, most are Bayesian-based methods. The course starts with a presentation of a few basic algorithms used for graphical models. We will learn how to compute a-posterior probabilities of hidden variables, present a universal-type algorithm known as the ‘sum-product’ algorithm for the computation of marginal distributions in complex graphs. We will extend the notion graphs to Markov Random Fields, and show some of its uses for example in computer vision. We will then investigate the case where there are no clear parameters for the model. We will describe Gaussian and Dirichlet processes and show some examples such as the problem of clustering where there is no a-priori number of clusters. Bayesian methods sometimes force us to evaluate integrals using sampling methods, in particular where such integrals have no analytic closed form. We will discuss some basic Monte Carlo sampling methods. The course also deals with cases where based on the data given we are asked to choose a model. We will discuss some Bayesian methods or alternatively hyper-parameter optimization to find an appropriate model. In addition the course will present some dimension reduction methods, their advantages and limitations. Moreover we will discuss semi-supervised methods where only part of the data is annotated. The course ends with an introduction to the theory of genetic algorithms.
The course discusses advanced topics in artificial intelligence, some have social and philosophical implications. We start with relatively new ideas such as adversarial attack, adversarial training, and graph neural networks and their use in science. We then present several topics concerning ethics, such as bias, fairness, privacy and transparency. The course will present several definitions of fairness and their corresponding algorithms, protocols for ‘differential privacy’ and other methods for privacy protection. We will then present the notion of explainable A.I., trying to reverse engineer neural networks, either internally by tuning internal components of the network or by a post-hoc analysis of the network outcome. On the opposite side graphical models are completely transparent, we will present basic algorithms to compute marginal probabilities in such networks, and then present large graphical models used in medicine. Next we will present artificial social algorithms, such as affective computing, body motion detection and more. One of the main question in A.I. is the possibility of Artificial General Intelligence. We will discuss the main ideas underneath AGI and some of their criticism. The course ends with the topic of Value Alignment, i.e. imprinting human values in algorithms.