First Indian presence at simulated power trading competition; walks away with 2nd place
What happens when machine learning algorithms, game theory and smart electricity markets come together? A IIIT-H team led by Dr. Praveen Paruchuri, Dr. Sujit Gujar, and Dual Degree student Susobhan Ghosh in collaboration with scientists from TCS Research and Innovation, Dr. Sanjay Bhat and Dr. Easwar Subramanian show us by participating in and bagging second place at the annual Power Trading Agent Competition (PowerTAC).
Smart Grids
Before explaining the energy trading game, Susobhan Ghosh, student researcher at the Machine Learning Lab begins by first describing a smart grid. A smart-grid which incidentally is a key component of the EU energy strategy is a power supply network consisting of conventional energy generators, renewable energy generators (e.g., wind turbines and solar panels), local electrical consumers, energy storage devices (e.g., batteries, electric vehicles), and smart infrastructure (e.g., smart meters, smart appliances), all working together to supply energy needs reliably, efficiently, and cheaply. In the real world, energy supply doesn’t always follow demand. Since renewable energy sources will not necessarily be able to produce power on demand, there is a need to simulate market behaviour of all participants in a smart grid to to predict and understand how such a grid will perform in different circumstances.
What is PowerTAC?
PowerTAC is an annual competition that uses an open-source simulation platform based on real-world data and state-of-the-art customer models. The simulation environment models a wholesale market, a regulated distribution utility, and a population of energy customers, situated in a real location on Earth during a specific period for which weather data is available. When different scenarios are created during the game, it helps researchers understand the dynamics of customer and retailer decision making, as well as the robustness of proposed market designs. The simulation also includes competing business entities or “broker agents” who, just like in the real world, offer energy services to retail customers through tariff contracts, and serve those customers by trading in a wholesale market. Brokers are challenged to maximize their profits by buying energy in the wholesale market and selling it in retail markets, subject to costs and constraints. The winner of an individual “game” is the broker with the highest bank balance at the end of a simulation run.
VidyutVanika
As opposed to the other competing agents with generic names such as AgentUDE, etc, the home team stayed true to its Indian roots by creating an autonomous agent which they named VidyutVanika. “Vidyut stands for electricity; Vanika means trader in Sanskrit, and hence VidyutVanika or ‘Power Trader’ ”, explains Dr. Bhat. This smart agent learned and adapted to the simulated smart-grid environment, by heavily relying on online reinforcement learning and prediction methods. According to Susobhan, their broker agent, nicknamed VV, tried to balance its demand and supply portfolio so as to reduce losses incurred from imbalance in the balancing market. “As far as we know, VV is the first agent to utilize weather data and weather forecast in its market decisions,” he says. “The agent that we built uses a Q-learning based approach in combination with a neural net based prediction. And not to take away credit from the humans (yet) with a couple of heuristics mixed in,” adds Dr. Easwar Subramaniam.
The Game
The tournament consists of multiple configurations – each configuration having a fixed number of brokers in the market and multiple “games”. The competition goes on 24/7 for several days. Playing a total of 204 games in the tournament with 6 other teams in various configurations like agent vs. agent, agent vs. 3 other agents, agent vs. all competing agents and so on, VV emerged in the second spot.
Team’s Expertise
The competition is open to all – from individuals to academic institutions. Being the only Indian entrant so far in the competition, it is not without pride that Dr. Paruchuri states that competing in PowerTAC requires a niche knowledge. “You not only need knowledge of game theory and machine learning, but also a good understanding of the energy markets. In addition to this, you need to invest huge amounts of effort and time.” The research team came from diverse backgrounds with each researcher bringing along his own expertise to the sounding board – Dr. Paruchuri with his focus on multi-agent systems and game theory, Dr. Gujar with his background in Micro-Economics and game theory, Dr. Bhat with his core competency in mathematical finance and prior experience in aerospace engineering, and Dr. Subramanian with his combined strengths in Mathematics and Computer Science. “Dr. Subramanian was pretty hands-on with the code along with Susobhan, “ adds Dr. Paruchuri.
The TCS Connection
The scientists from TCS say that TCS Research and Innovation was interested in problems that involved multiple autonomous agents learning to compete with each other in an interactive dynamic environment. The Machine Learning Lab at IIIT-H emerged as a natural partner, since it possessed expertise in the two main areas of research related to such problems, namely game theory and machine learning. “During the initial discussions, PowerTAC emerged as an interesting testbed to try out ideas and build a collaboration around. Hence the Planning and Control Group from TCS Research and Innovation along with the Machine Learning Lab from IIIT-H decided to team up and submit a broker agent to the competition,” says Dr. Bhat.
The team has been working on this over the last year. They submitted VV to PowerTAC’s previous edition but admit that they had worked on it only for a few months. Hence it was a very ‘simple’ agent. Buoyed by the success of this year’s performance, they hope to take a stab at it one more time before venturing into other competitions requiring similar expertise.