@WarrenBPowell profile picture
Warren Powell @WarrenBPowell

Decisions under uncertainty (see jungle.princeton.edu): transportation and logistics, energy systems, e-commerce, health, learning problems

Joined October 2019
  • Tweets 138
  • Following 100
  • Followers 735
4 Photos and videos
… I think my new book lays this foundation. Https://tinyurl.com/RLandSO/

1
1
5
If I am right, there is no school that knows how to model and solve real world supply chain problems. The reason is the communities that have worked on these problems… but also the stats of research into sequential decision problems.
1
2
… if anyone thinks they know of a proper model of a stochastic inventory model, send me the title and page number.
1
1
I have been reviewing the top textbooks on supply chains… supply chain management, supply chain theory, supply chain engineering, and production/operations management… not one presents a mathematical model of an inventory problem involving uncertainty…
3
3
18
I am currently writing a new book with tentative title: “Supply Chain Modeling: An information-based approach”. It will be the first book to approach the modeling of supply chain problems using my new universal framework for sequential decision problems.
1
1
12
I have been giving talks emphasizing the need for programs in “sequential Decision analytics” which lays the foundation for technical work in supply chains. Please see tinyurl.com/sdafieldyoutube. I close with a pitch for new academic programs on the topic.
2
I just updated my “what is RL” webpage at tinyurl.com/what-is-rl/. I compare the perspectives of three leaders: Sutton and Barto (2nd edition). Ben van Roy (a leading RL researcher), and Dimitri Bertsekas), then Wikipedia, and finally my new book.

1
11
I am starting a new series of tutorials on teaching how to solve sequential decision problems under uncertainty using my new framework. See tinyurl.com/teachingsda
1
16
Many thanks to @brianlaungaoaeh for highlighting the limitations of "modern" inventory theory. My video at tinyurl.com/sdafield provides an introduction to my new framework, illustrated with examples for real-world supply chain problems.
1
2
For anyone following me on Twitter… I use LinkedIn for all my professional posts on sequential decision analytics.
2
The e-book version of my new book Reinforcement Learning and Stochastic Optimization is now available on the Wiley website at tinyurl.com/RLSOwiley. Looks good! (and weighs a lot less!).
1
7
… For an overview of the book, go to tinyurl.com/RLandSO. For an introduction to a field that I am calling sequential decision analytics, go to tinyurl.com/sdafield/.
3
… My new book is also the first to identify four classes of policies that cover *every* method for making decisions that has been studied in the academic literature or used in practice. It is truly a universal framework for sequential decisions….
My new book: - Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions - is finally out! This is the first book to unify 15 distinct fields that deal with sequential decision problems….
2
6
86
A lot of people talk about doing (or using) "reinforcement learning". In the 1990s, this was a form of ADP (Q-learning). Today, it is a bit of a mess. See my discussion at tinyurl.com/what-is-rl/
6
BTW: If anyone likes these tweets on dynamic programming/sequential decisions, please follow me on LinkedIn. Tweets are just too short! :(
7
Deterministic optimization people understand creating a model with a generic x, *then* design an algorithm to find x. Everyone gets this. For stochastic problems, we create a model using a generic *policy* X^\pi(S_t), and *then* go search for the best policy.
4
I see so many papers that think that the right way to model a dynamic program is to start with Bellman's equation (this is how I learned it as a grad student at MIT). This is just wrong! It makes as much sense as writing a linear program in terms of complementary slackness.
1
7
Even for stochastic problems, you design a policy that provides a decision that you implement. Imagine planning a path through a stochastic transportation network. You still have to decide whether to turn left or right, and you have to live with your decision.
This tweet is unavailable
6
Message from Donald Trump... Come on in! The water is warm!
5
Load more