Page 1
Page 1
Stochastic Learning and Optimization : A Sensitivity-Based Approach
Covers various disciplines in learning and optimization, including perturbation analysis (PA) of discrete-event dynamic systems, Markov decision processes (MDP)s), reinforcement learning (RL), and adaptive control, within a unified framework. (A simple approach to MDPs) This book introduces MDP theory through a simple approach based on performance difference formulas. This approach leads to results for the n-bias optimality with long-run average-cost criteria and Blackwell's optimality without discounting.
Simulation-based Algorithms for Markov Decision Processes
Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. It is well-known that many real-world problems modeled by MDPs have huge state and/or action spaces, leading to the notorious curse of dimensionality that makes practical solution of the resulting models intractable. In other cases, the system of interest is complex enough that it is not feasible to specify some of the MDP model parameters explicitly, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based numerical algorithms have been developed recently to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function.
Risk-Averse Capacity Control in Revenue Management
“If necessity is the mother of invention, then deregulation is the father, and r- enue management (also known as yield management) is the couple’s golden child – at least as far as operations research is concerned.” (Horner, 2000, p. 47) Deregulation had a signi?cant impact on the U.S. airline industry in the late 1970s. Charter and low-cost airlines such as People Express and Southwest wereabletoo?erseatsatafractionofthepricechargedbyestablishedcarriers like Pan Am and American Airlines. Due to their di?erent cost structure, it seemed to be impossible for the big carriers to o?er tickets at the same low price. Yet they had to ?nd a way to compete. Robert L. Crandall from American Airlines is widely credited with the - lution to the problem:yield management – today calledrevenue management, since it maximizes revenue earned on a ?ight rather than yield (revenue per passenger mile).
Distributed artificial intelligence ; 2nd International conference, DAI 2020, Nanjing, China, October 24–27, 2020, Proceedings
This book constitutes the refereed proceedings of the Second International Conference on Distributed Artificial Intelligence, DAI 2020, held in Nanjing, China, in October 2020. The 9 full papers presented in this book were carefully reviewed and selected from 22 submissions. DAI aims at bringing together international researchers and practitioners in related areas including general AI, multiagent systems, distributed learning, computational game theory, etc., to provide a single, high-profile, internationally renowned forum for research in the theory and practice of distributed AI.
Discrete-Time Markov Chains : Two-Time-Scale Methods and Applications
The motivation stems from existing and emerging applications in optimization and control of complex hybrid Markovian systems in manufacturing, wireless communication, and financial engineering. Much effort in this book is devoted to designing system models arising from these applications, analyzing them via analytic and probabilistic techniques, and developing feasible computational algorithms so as to reduce the inherent complexity. This book presents results including asymptotic expansions of probability vectors, structural properties of occupation measures, exponential bounds, aggregation and decomposition and associated limit processes, and interface of discrete-time and continuous-time systems. One of the salient features is that it contains a diverse range of applications on filtering, estimation, control, optimization, and Markov decision processes, and financial engineering.
Markov Decision Processes with Their Applications
Markov decision processes (MDPs), also called stochastic dynamic programming, were first studied in the 1960s. MDPs can be used to model and solve dynamic decision-making problems that are multi-period and occur in stochastic circumstances. There are three basic branches in MDPs: discrete-time MDPs, continuous-time MDPs and semi-Markov decision processes. Starting from these three branches, many generalized MDPs models have been applied to various practical problems. These models include partially observable MDPs, adaptive MDPs, MDPs in stochastic environments, and MDPs with multiple objectives, constraints or imprecise parameters.
Bayesian Networks and Decision Graphs
Probabilistic graphical models and decision graphs are powerful modeling tools for reasoning and decision making under uncertainty. As modeling languages they allow a natural specification of problem domains with inherent uncertainty, and from a computational perspective they support efficient algorithms for automatic construction and query answering. This includes belief updating, finding the most probable explanation for the observed evidence, detecting conflicts in the evidence entered into the network, determining optimal strategies, analyzing for relevance, and performing sensitivity analysis. The book introduces probabilistic graphical models and decision graphs, including Bayesian networks and influence diagrams.It contians two parts. The first part focuses on probabilistic graphical models. Compared with the previous book, the new edition also includes a thorough description of recent extensions to the Bayesian network modeling language, advances in exact and approximate belief updating algorithms, and methods for learning both the structure and the parameters of a Bayesian network. The second part deals with decision graphs, and in addition to the frameworks described in the previous edition, it also introduces Markov decision processes and partially ordered decision problems.
