INFORMS 2021 Program Book

INFORMS Anaheim 2021

TB16

3 - Understanding Road Users’ Behavior from Egocentric Video Data Yichen Ding, University of Iowa, Iowa City, IA, 52246-2872, United States Based on road users’ trip record data, we use deep learning methods to model and forecast their responses to the various traffic conditions and reactions in the complex road environment. Furthermore, we provide some case studies to generate insights on how to keep these road users safe and bring inspiration to facilitate the behavioral studies. 4 - Modeling Lengthy Behavioral Log Data for Customer Churn Management Daehwan Ahn, Post Doc, University of Pennsylvania, Philadelphia, PA, 19130, United States, Dokyun Lee, Kartik Hosanagar Churn management has benefited much from advanced feature learning techniques applied to large-scale behavioral log data. Despite its success, the current models can only address sequences of short length ranging from hundreds to thousands. In practice, however, customer log data has a very long sequence that can extend to millions in length that can only be utilized through manual and onerous feature engineering, which requires domain expertise and can be unreliable depending on the data scientist. We propose an automated log- processing approach that extends powerful feature learning approaches to extract valuable signals from lengthy log data. Our proposed framework achieves a significant improvement in customer churn prediction relative to existing manual feature engineering approaches developed by a global game company. TB16 CC Room 201D In Person: Reinforcement Learning General Session Chair: Zaiwei Chen, Georgia Institute of Technology, Atlanta, GA, 30318, United States 1 - Finite Sample Analysis of Off-policy Natural Actor-critic Algorithm Sajad Khodadadian, Georgia Institute of Technology, Atlanta, GA, United States In this paper, we provide finite-sample convergence guarantees for an off-policy variant of the natural actor-critic (NAC) algorithm based on Importance Sampling. In particular, we show that the algorithm converges to a global optimal policy with a sample complexity of O( -3log2(1/ )) under an appropriate choice of stepsizes. In order to overcome the issue of large variance due to Importance Sampling, we propose the Q-trace algorithm for the critic, which is inspired by the V-trace algorithm. This enables us to explicitly control the bias and variance, and characterize the trade-off between them. As an advantage of off-policy sampling, a major feature of our result is that we do not need any additional assumptions, beyond the ergodicity of the Markov chain induced by the behavior policy. 2 - Finite-Sample Analysis Of Reinforcement Learning Algorithms: A Lyapunov Approach Zaiwei Chen, Georgia Institute of Technology, Atlanta, GA, 30318, United States This paper develops an unified framework to study finite-sample convergence guarantees of a large class of value-based asynchronous Reinforcement Learning (RL) algorithms. We do this by first reformulating the RL algorithms as Markovian Stochastic Approximation (SA) algorithms to solve fixed-point equations. We then develop a Lyapunov analysis and derive mean-square error bounds on the convergence of the Markovian SA. Based on this result, we establish finite-sample convergence bounds for asynchronous RL algorithms such as Q-learning, n-step TD, TD(\lambda), and off-policy V-trace. As a by-product, by analyzing the performance bounds of the TD(\lambda) (and n-step TD) algorithm for general \lambda (and n), we demonstrate a bias-variance trade-off, i.e., efficiency of bootstrapping in RL. This was first posed as an open problem in (Sutton, 1999).

Performance evaluation frameworks for problems involving online stochastic combinatorial optimization are nearly always based on offline or omniscient bounds. While improved primal policies can be obtained using machine learning approaches, very little progress has occurred on finding dual (omniscient) bounds. We present a novel methodology to construct tighter performance bounds for dynamic resource allocation problems modeled on time-space networks. 2 - Demand Estimation for Low Cost Carriers Xiaodong Luo, Professor, Chinese University of Hong Kong, ShenZhen Campus, ShenZhen, Guangdong Province, China, Xiaodong Luo We present some interesting ideas for demand estimation/untruncation for low cost carriers. We will use a simplified spiked MNL model and then use advanced nonlinear programming such as majorization minimization/Frank Wolfe method to solve the underlying non-convex optimization problems. We will compare it with other MNL models and other expectation maximization methods. TB18 CC Room 202B In Person: Improving Distributed Energy Generation and Resilient Microgrid Modeling General Session Chair: Andrew MacMillan, Carleton University, Ottawa, ON, Canada 1 - Characterizing Wind Power Curtailment in Ercot Kristen Schell, Assistant Professor, Carleton University, Ottawa, ON, 12180-3522, Canada Curtailment of renewable energy occurs when the power system cannot accept the power available from the renewable generator, so this power is “spilled”, or not used. Curtailment can happen in a power system for many reasons market and dispatch decision, transmission constraints, errors in forecasting as the power grid is constantly balancing supply and demand. Using data from the power system operator of Texas, ERCOT, two models are developed to: 1) characterize the extent of wind power curtailment and 2) quantify the opportunity cost of past curtailment practices. 2 - Robust and Cost-effective Microgrid Design for Equitable Climate Resilience: A Case Study of West Oakland’s Resilience Hub Project Papa Yaw Owusu-Obeng, Rensselaer Polytechnic Institute, Troy, NY, 12180, United States Microgrids can maintain power to critical loads in events of utility power outages caused by extreme weather. This is useful in building resilience to climate crisis— as seen in the 2020 California heat wave and 2021 Texas winter storm where vulnerable populations depended on facilities with microgrids for essential energy services. This has driven the demand for resilience hubs to provide equitable energy access to low-income communities in the event of power outage. The challenge however is that the concept of resilience hub microgrids are new and there is no blueprint for robust and cost-effective design. This work presents advancement on two fronts: 1) energy efficiency analysis to determine the optimal energy conservation measures for resilience hubs, and 2) a multi- objective optimization model for least cost microgrid design to accommodate extended power outages. 3 - Predicting Hydrokinetic Power Potential in Short Stream Reaches Via Remote Sensing Methods Andrew MacMillan, Carleton University, Ottawa, ON, Canada Hydrokinetic power is a promising technology to address energy security issues for rural communities. Predicting the power potential of a location via resource assessment is a crucial first step in planning new energy projects. This is important for screening out locations and determining where on-site data collection is warranted. Recent studies have used digital elevation datasets to remotely predict flow characteristics where on-site data was not available. However, the studies focused on broader watershed regions and large rivers rather than specific locations. This study seeks to apply remote sensing methods to specific stream segments and smaller rivers for hydrokinetic feasibility and to validate it against obtained ADCP data from site measurements. Results provide insights into adapting first-principles equations of river flow to shorter stream reaches.

TB17 CC Room 202A In Person: Optimization Techniques for Airline Industry General Session

Chair: Xiaodong Luo, Professor, Chinese University of Hong Kong, ShenZhen Campus, ShenZhen, Guangdong province, China, China 1 - Efficient Performance Bounds For Online Decisions On Large Time-space Networks Lavanya Marla, U of Illinois at Urbana-Champaign, Urbana, IL, 61801-2925, United States

104