• Login
    View Item 
    •   DSpace@RPI Home
    • Rensselaer Libraries
    • RPI Theses Online (Complete)
    • View Item
    •   DSpace@RPI Home
    • Rensselaer Libraries
    • RPI Theses Online (Complete)
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Using reinforcement learning to improve network durability

    Author
    Hammel, Erik
    View/Open
    167189_Hammel_rpi_0185E_10100.pdf (3.647Mb)
    Other Contributors
    Mitchell, John E.; Bennett, Kristin P.; Ecker, Joseph G.; Sharkey, Thomas C.; Wallace, William A., 1935-;
    Date Issued
    2013-05
    Subject
    Mathematics
    Degree
    PhD;
    Terms of Use
    This electronic version is a licensed copy owned by Rensselaer Polytechnic Institute, Troy, NY. Copyright of original work retained by author.;
    Metadata
    Show full item record
    URI
    https://hdl.handle.net/20.500.13015/894
    Abstract
    Our goal is to determine and optimize the efficacy of reinforcing an existing flow network to prevent unmet demand from imminent disruptions. We are given probabilities of failures for edges in the network and are asked to find edges which will best provide durability to the network post-event. The problem is extended to multiple time steps to address concerns of available resources versus quality of installations: the farther away from the event one makes decisions the more resources are available but the less reliable the uncertainty information. This sequential decision-making process is a classic example of dynamic programming. To avoid the "curses of dimensionality", we formulate an approximate dynamic program. To improve performance, especially as applied to flow networks, we derive several innovative adaptations from reinforcement learning concepts. This involves developing a policy, a function that makes installation decisions when given current forecast information, in a two step process: policy evaluation and policy improvement.; With a trained policy, we compare its performance against traditional two-stage stochastic programs with recourse utilizing a sample average approximation model. We consider several implementations of the stochastic problem to gauge performance in a variety of ways. The material presented here is developed in the context of preparing urban infrastructures against damages caused by disasters, however is applicable to any flow network. This paper contributes to both the field of multistage stochastic programming and approximate dynamic programming by introducing factors to each other. We also apply innovative reinforcement learning techniques to flow networks that, as of this writing, have yet to be addressed.; The primary solution technique takes forecast samples from a Monte Carlo simulation in the style of stochastic programming. Once a forecast is obtained, the problem is set up by taking additional samples of the forecast probabilities to determine capacities for the given time step. This forms the state information used in performing the approximate dynamic program. The sampled outcome information is used to define network constraints for the policy improvement step. The approximation for future costs is then refined using the improved policy compared with a desirable target objective. This process is repeated over several iterations. Lastly, we provide empirical evidence which corroborates with basic theorems of convergence for more simplistic forms of the reinforcement learning process.;
    Description
    May 2013; School of Science
    Department
    Dept. of Mathematical Sciences;
    Publisher
    Rensselaer Polytechnic Institute, Troy, NY
    Relationships
    Rensselaer Theses and Dissertations Online Collection;
    Access
    Restricted to current Rensselaer faculty, staff and students. Access inquiries may be directed to the Rensselaer Libraries.;
    Collections
    • RPI Theses Online (Complete)

    Browse

    All of DSpace@RPICommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    Login

    DSpace software copyright © 2002-2022  DuraSpace
    Contact Us | Send Feedback
    DSpace Express is a service operated by 
    Atmire NV