Distributed Reinforcement Learning for Electricity Market Bidding with Spark - Databricks

Distributed Reinforcement Learning for Electricity Market Bidding with Spark

Download Slides

A majority of the electricity in the U.S. is traded in independent system operator (ISO) based wholesale markets. ISO-based markets typically function in a two-step settlement process with day-ahead (DA) financial settlements followed by physical real-time (spot) market settlements for electricity. In this work, we focus on obtaining equilibrium bidding strategies for electricity generators in DA markets. Electricity prices in DA markets are determined by the ISO, which matches competing supply offers from power generators with demand bids from load serving entities. Since there are multiple generators competing with one another to supply power, this can be modeled as a competitive Markov decision problem, which we solve using a reinforcement learning approach. For power networks of realistic sizes, the state-action space could explode, making the RL procedure computationally intensive. This has motivated us to solve the above problem over Spark. The talk provides the following takeaways:

1. Modeling the day-ahead market as a Markov decision process
2. Code sketches to show the markov decision process solution over Spark and Mahout over Apache Tez
3. Performance results comparing Mahout over Apache Tez and Spark.