Intelligent Systems Internships

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-6 months) in the Intelligent Systems (IS) group at CWI.  Our group has experience and ongoing European projects with industry in the area of smart-grids and energy markets. We host both collaborative applied research visitors and pure research-oriented projects in areas such as multiagent systems, negotiation, game theory and machine learning. Successful applicants will be working under supervision of IS researchers: Michael Kaisers, Tim Baarslag and Pablo Hernandez. The applicant needs to be currently enrolled as a Master's or PhD student*. *If the candidate is registered at a Dutch university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-6 months) in the Intelligent Systems (IS) group at CWI. 

Our group has experience and ongoing European projects with industry in the area of smart-grids and energy markets. We host both collaborative applied research visitors and pure research-oriented projects in areas such as multiagent systems, negotiation, game theory and machine learning. Successful applicants will be working under supervision of IS researchers: Michael KaisersTim Baarslag and Pablo Hernandez. The applicant needs to be currently enrolled as a Master's or PhD student*.

*If the candidate is registered at a Dutch university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

Possible ideas are:


Learning in multiagent-systems

Competitive-Cooperation

First, a survey of example systems from smart energy grids, automated trading (auctions/negotiation), transportation and/or distributed sensor networks shall identify a small set of benchmark systems. Second, the candidate shall extend multi-agent learning techniques for sequential decision making to exploit the coopetitive context, e.g. in the form of tailored opponent models. The topic is tightly connected to the research in our group, and may draw on areas such as game theory, mechanism design or machine learning. 

Mechanisms to foster reciprocity in energy cooperatives:

Energy cooperatives are an emergent and interesting association among several individuals that not only share a geographic location but also aim to share resources such as storage and renewable energy generation systems (e.g., solar panels). These energy cooperatives have the potential of reducing costs for their members in the long run, however there are still many open questions.One idea is to develop intelligent mechanisms to foster cooperation among agents.  Consider a population of agents in the cooperative; each agent has different energy resources and behaviors. At any specific point in time, we can identify two (possibly empty) subsets: those agents with shortage of energy and those with excess of energy. We could envision a central system that select some agents from each set in order to promote cooperation. For example, they can decide whether they want to give/receive energy from the other agent. The motivation for a focal agent that shares (gives) energy to one of its neighbours is that in the future the focal agent may need energy and someone will may share that energy with him. Moreover, another related idea is to propose an insurance/hedging scheme for those users who prefer a standard (and stable) payment and do not like variability that can arise due to smart metering.

Transparency effects in (blockchain) markets:

In multiagent systems the observability of the other agents actions, payoffs and environment variables is one key element to learn optimal behaviors. In the context of markets, it is common to assume anonymity of the participants, this is, the participants are not fully informed on the actions of the rest. However, this limited observability might change with emerging technologies like blockchain (e.g. bitcoin). With blockchain systems all transactions are visible to the general audience ensuring transparency which therefore can have effects on how agents interact in face of this new type of information. For example, it is an open questions how this new information (complete or partial) can affect the market efficiency.

Efficient representation in multiagent interactions:

Many learning techniques need the complete specification of the mixed strategies of the other agents to derive an optimal behavior, i.e., best response. However, this is not tractable when the number of agents increases, like in many real-world domains. Thus, one fundamental problem in the multiagent area is to make an efficient compromise between having a correct yet compact representation of the agents.  One idea is to explore the concept of influence space. For example, assume only another agent in the environment with only two actions. In some scenarios the other opponent strategy could be parametrized with a single value (a threshold) for which the opponent either uses one or the other action. This influence space can then be used to represent the best response in a reduced way. Furthermore, this idea could be generalized with more agents which will provide major improvements in the computation of the best response.

Future energy markets:

Standard markets assume discrete intervals where the clearing takes place, this results in unfair situations for some participants since they are not coordinated with those intervals. Instead of using discrete periods, one idea is to use tools from continuous-time reinforcement learning as a novel way to solve to approach these markets. There are different open questions, for example how this new type of continuous-bids will affect the market clearing.

Flexibility allocation in Smart-Grids

Retailers of energy in current electricity markets are playing the role of balancing responsible parties, procuring electricity in the day-ahead markets, and balancing supply with demand in reserve markets. Supply needs to follow demand. However, demand side management methodologies make the reverse assumption. Current methods for balancing supply with demand, as well as providing incentives for flexibility by customers, include but not limited to dynamic pricing (e.g., real-time, Time of use tariffs). The increasing adoption of flexibility (e.g., Intelligent home appliances, EVs, Home batteries) enables demand side management, and if provided incentives it can further contribute to balancing. Candidates for this position will study and analyze arising problems in this setting, elicitation of flexibility, optimal allocation of flexibility, as well as the optimal procurement of electricity in the ahead markets. The topic is tightly connected to the research in our group, and may draw on areas such as game theory, mechanism design or machine learning.

Interested applicants in these topics may contact Pablo Hernandez for more information.


Cooperation and Negotiation

Negotiation in Energy Cooperatives

An energy cooperative is a group of people who share their energy resources, such as their batteries and renewables (e.g. solar panels). By working together, energy cooperatives can reduce their overall costs: for example, when someone produces excess energy and their battery is fully charged, they can share it with their neighbor instead of selling it back to the grid. Of course, this gesture should be returned at some point, and for this, members of an energy cooperative can use negotiation to make deals about the amount of energy to share with each other, and the best time to do so. The candidate is expected to come up with a negotiation protocol for a simple, hands-on scenario (e.g. day/night transfer) and implement a simulation of such energy exchanges. The negotiation platform Genius can be used as a platform to build negotiation agents that represent the members of the cooperative and study their interactions.

Typical research questions include:

- Every cooperative member can have different characteristics with respect to how 'tough' they are in the negotiation. How does the competitiveness of the members influence the system aggregate (e.g. using measures such as the price of anarchy)? What are essential characteristics of its members for a cooperative to remain viable?

- Members of the cooperative interact on a day-to-day basis, so it is important for them to build a good relationship with their peers. How should the members balance their short-term preferences with their follow-up interactions?

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. An eligible candidate has good programming and writing skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants in these topics may contact Tim Baarslag for more information.

 

Smart Grid Negotiations

Computers that are able to negotiate are already common practice in areas such as high frequency trading, and are now finding applications in domains closer to home, such as the smart grid, which involve not only mere financial optimizations but balanced trade-offs between multiple issues, such as cost and convenience. Such agents can autonomously negotiate and coordinate with others in our stead, to reach outcomes and agreements in our interest.

As a simple example, a smart thermostat controlling a heat pump could provide demand response to the electricity grid if the inconvenience is offset by the grid relieve incentives. In such situations, the agent represents a user with individual and a priori unknown preferences, which are costly to elicit due to the user bother this incurs. Therefore, the agent needs to strike a  balance between increasing the user model accuracy and the inconvenience caused by interacting with the user. To do so, we require a tractable metric for the value of information in an ensuing negotiation, which until now has not been available.

Typical research questions include:

  1. How can we design a negotiation decision model that can capture and reason about uncertain utility information about the user (as well as the opponent) and can query the user for more preference information?
  2. Can we formulate a tractable measure of the estimated expected utility of a negotiation?
  3. What is a good stopping criterion that can weigh the estimated post-query value of information against the elicitation costs?

A (partial) solution to these questions would yield entirely new systems (energy management systems being one example) that can negotiate on behalf of a user and that have the ability to query a user to specify their preferences when needed. This fits in a broader vision of an interactive negotiation agent that has knowledge about what questions can be asked at what costs, and can decide, using our method, which question is worth posing given the utility expected to arise from it.

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. An eligible candidate has good mathematical skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants in these topics may contact Tim Baarslag for more information.


Intelligent Energy Management

Spectral Utilities (www.spectralutilities.com) and the Intelligent Systems research group at CWI (www.cwi.nl/research-groups/Intelligent-Systems) offer a joint internship in Amsterdam on the topic of data-driven intelligent control for sustainable energy supply. The internship will focus on the optimization of an advanced energy management system (EMS) which will enable the aggregation and control of distributed energy systems to automate micro-grid operations. The EMS may be implemented in both temporary (e.g. local festivals and events) or permanent applications (e.g. residential communities with shared infrastructure), and should be adaptable to a wide range of contexts. Applicants should have experience in programming and at least a basic understanding of electricity and electronics. The goal of this 2-3 month placement is to improve existing algorithms for intelligent energy storage control in the face of uncertain future supply from renewable energy sources. This challenge may involve the development of new data collection, analysis, and forecasting techniques to integrate into the existing software platform. Data is available for analysis as the basis of exploring viable improvements, and successful trials will be implemented for further testing at an R&D facility.  

Innovative business case in Smart-Grids

The Intelligent Systems research group at CWI (www.cwi.nl/research-groups/Intelligent-Systems) and its spin-off company SEITA offer a joint internship in Amsterdam on the topic of innovative flexibility services for the smart grid. The intern may contribute to 1. analyzing stakeholder requirements for consultancy and electric vehicle (EV) aggregation services, 2. developing the backbone multi-sided platform addressing energy retailers, EV owners, charging infrastructure operators and distribution system operators; or 3. refining business models and evaluating innovative tariffs of products and services in this context. The successful applicant has an entrepreneurial spirit and seeks experience on the intersection of research and innovation, drawing primarily on economics, informatics and electrical engineering. Depending on the chosen focus area, the outcome could be a detailed use-case report, an improved software package or algorithm, or a business case / business plan document.

Interested applicants in these topics may contact Tim Baarslag and Pablo Hernandez for more information.