Internships in the Intelligent and Autonomous Systems Group

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-6 months) in the Intelligent and Autonomous Systems (IAS) group at CWI.  Our group has experience and ongoing European projects with industry in the area of planning and (multi-agent) learning for negotiations, markets and smart-grids. We host both collaborative applied research visitors and pure research-oriented projects in areas such as multiagent systems, negotiation, game theory and machine learning. Successful applicants will be working under supervision of IAS researchers: Michael Kaisers, Tim Baarslag, and Daan Bloembergen. The applicant needs to be currently enrolled as a Master's or PhD student. If the candidate is registered at a European university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-6 months) in the Intelligent and  Autonomous Systems (IAS) group at CWI. 

Our group has experience and ongoing European projects with industry in the area of planning and (multi-agent) learning for negotiations, markets and smart-grids. We host both collaborative applied research visitors and pure research-oriented projects in areas such as multiagent systems, negotiation, game theory and machine learning. Successful applicants will be working under supervision of IAS researchers: Michael KaisersTim Baarslag, and Daan Bloembergen. The applicant needs to be currently enrolled as a Master's or PhD student. If the candidate is registered at a European university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

Possible ideas are:


Learning in multi-agent systems

Many complex systems in today's society can be effectively modelled as multi-agent systems, in which multiple (autonomous) decision makers interact and jointly determine the system dynamics. Multi-agent systems are on the one hand highly complex from a central decision making perspective, and on the other hand non-deterministic and partially observable from the perspective of a single agent. As a result, planning in advance for all possible scenarios quickly becomes intractable, and instead learning algorithms can be used to find (approximate) solutions.

The typical application domain for research within the IAS group are smart energy systems, ranging from micro-grid interactions between individual households, to larger scale negotiations between energy cooperatives and utility providers. For example, local communities may wish to cooperate by sharing resources, such as solar panels and battery storage, and by competing as a group in external energy and flexibility markets. A typical internship project within this application domain would be centred around a research topic such as:

  • Learning joint solutions, e.g. to allocate flexibility in smart-grids for balancing (local) production, storage, and consumption.
  • Cooperation in multi-agent systems, e.g., by fostering reciprocity within energy cooperatives.
  • Opponent modelling using efficient representation of other agents' strategies in multi-agent learning problems to facilitate best response computation, e.g. using parametrization or influence spaces.

Internship projects on the topic of multi-agent learning may draw on areas such as (multi-agent) reinforcement learning, deep learning, game theory, and mechanism design. Projects can be theoretical, i.e. contributing to fundamental research on multi-agent learning, or more practically oriented by applying existing techniques to problems from the smart energy domain.

Interested applicants in these topics may contact Daan Bloembergen for more information.


Preference modelling and Negotiation

Negotiation in Energy Cooperatives

An energy cooperative is a group of people who share their energy resources, such as their batteries and renewables (e.g. solar panels). By working together, energy cooperatives can reduce their overall costs: for example, when someone produces excess energy and their battery is fully charged, they can share it with their neighbor instead of selling it back to the grid. Of course, this gesture should be returned at some point, and for this, members of an energy cooperative can use negotiation to make deals about the amount of energy to share with each other, and the best time to do so. The candidate is expected to come up with a negotiation protocol for a simple, hands-on scenario (e.g. day/night transfer) and implement a simulation of such energy exchanges. The negotiation platform Genius can be used as a platform to build negotiation agents that represent the members of the cooperative and study their interactions.

Typical research questions include:

- Every cooperative member can have different characteristics with respect to how 'tough' they are in the negotiation. How does the competitiveness of the members influence the system aggregate (e.g. using measures such as the price of anarchy)? What are essential characteristics of its members for a cooperative to remain viable?

- Members of the cooperative interact on a day-to-day basis, so it is important for them to build a good relationship with their peers. How should the members balance their short-term preferences with their follow-up interactions?

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. An eligible candidate has good programming and writing skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants in these topics may contact Tim Baarslag for more information.

Smart Grid Negotiations

Computers that are able to negotiate are already common practice in areas such as high frequency trading, and are now finding applications in domains closer to home, such as the smart grid, which involve not only mere financial optimizations but balanced trade-offs between multiple issues, such as cost and convenience. Such agents can autonomously negotiate and coordinate with others in our stead, to reach outcomes and agreements in our interest.

As a simple example, a smart thermostat controlling a heat pump could provide demand response to the electricity grid if the inconvenience is offset by the grid relieve incentives. In such situations, the agent represents a user with individual and a priori unknown preferences, which are costly to elicit due to the user bother this incurs. Therefore, the agent needs to strike a  balance between increasing the user model accuracy and the inconvenience caused by interacting with the user. To do so, we require a tractable metric for the value of information in an ensuing negotiation, which until now has not been available.

Typical research questions include:

  1. How can we design a negotiation decision model that can capture and reason about uncertain utility information about the user (as well as the opponent) and can query the user for more preference information?
  2. Can we formulate a tractable measure of the estimated expected utility of a negotiation?
  3. What is a good stopping criterion that can weigh the estimated post-query value of information against the elicitation costs?

A (partial) solution to these questions would yield entirely new systems (energy management systems being one example) that can negotiate on behalf of a user and that have the ability to query a user to specify their preferences when needed. This fits in a broader vision of an interactive negotiation agent that has knowledge about what questions can be asked at what costs, and can decide, using our method, which question is worth posing given the utility expected to arise from it.

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. An eligible candidate has good mathematical skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants in these topics may contact Tim Baarslag for more information.


Heuristic Search for Sample-based Planning and Control

Operating large scale complex systems, such as the electrical grid, efficiently requires both scheduling at course time resolutions, and continuous control. Automated planning algorithms face this wide range of time scales, and may need to trade offs (finite) computational resources and solution quality. If a model of the environment is available, the class of sample-based planning algorithms, such as Monte Carlo Tree Search (MCTS), has successfully applied heuristic search techniques to sequential decision-making. This internship will review the state of the art in this field and implement a (variation of) cutting edge algorithms for planning and control.

Interested applicants in these topics may contact Michael Kaisers for more information.


Applications

Energy management for smart grids

The IAS research group at CWI offers joint internships in Amsterdam, building on previous collaborations with energy startups/SMEs Spectral, SEITA, and Senfal on the topic of data-driven intelligent control for sustainable energy supply. The internship will focus on the optimization of an advanced energy management system (EMS) which will enable the aggregation and control of distributed energy systems to either automate micro-grid operations, or refine business models for services in this context. The EMS may be implemented in both temporary (e.g. local festivals and events) or permanent applications (e.g. residential communities with shared infrastructure), and should be adaptable to a wide range of contexts. Applicants should have experience in programming and at least a basic understanding of electricity and electronics. The goal of this 2-6 month placement is to improve existing algorithms for intelligent energy storage control in the face of uncertain future supply from renewable energy sources. This challenge may involve the development of new data collection, analysis, and forecasting techniques to integrate into the existing software platform. Data is available for analysis as the basis of exploring viable improvements, and successful solution prototypes may be further tested at an R&D facility.  

Consensus and cooperation on blockchains

This internship investigates the potential to apply the techniques above (learning, preference elicitation, negotiation) on blockchain-based platforms (in particular, considering blockstack). Building on crypto-based solutions for authentication and payment, the successful applicant will work on building autonomous agents with impact on the real world.

Interested applicants in these topics may contact Michael Kaisers for more information.