Internships in the Intelligent and Autonomous Systems Group

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-9 months) in the Intelligent and Autonomous Systems (IAS) group at CWI.  Our group has experience and ongoing European projects with industry in the area of planning and (multi-agent) learning for negotiations, markets and smart-grids. We host both collaborative applied research visitors and pure research-oriented projects in areas such as multiagent systems, negotiation, game theory and machine learning. Successful applicants will be working under supervision of IAS researchers: Tim Baarslag, Hendrik Baier, and Valentin Robu. The applicant needs to be currently enrolled as a Master's or PhD student. If the candidate is registered at a European university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

We are looking for outstanding candidates who want to pursue an internship or thesis project (2-9 months) in the Intelligent and  Autonomous Systems (IAS) group at CWI. 

Our group has experience and ongoing European projects with industry in the area of planning and (multi-agent) learning for negotiations, markets, and smart grids. We host both visitors for collaborative applied research, as well as pure research oriented projects in areas such as multi-agent systems, negotiation, game theory, game AI, planning, search, and machine learning. Successful applicants will be working under supervision of IAS researchers: Tim Baarslag, Hendrik Baier, and Valentin Robu. The applicant needs to be currently enrolled as a Master's or PhD student. If the candidate is registered at a European university with a performance rate of 8 or higher as evidenced by a list of results provided by the university the candidate can be eligible for an allowance. For non-EU citizens special regulations may apply.

Possible ideas are:


Negotiation and Preference modelling

Negotiation AI

Negotiation has become a major and interesting research area within Artificial Intelligence. Computers that are able to negotiate are already common practice in areas such as high frequency trading, and are now finding applications in domains closer to home, such as the smart grid, real estate, market places, the Internet of Things, and autonomous vehicles. These negotiations involve not just financial optimizations but balanced trade-offs between multiple aspects, such as cost and convenience. Such agents can autonomously negotiate and coordinate with others in our stead, to reach outcomes and agreements in our interest.

Negotiation is one of the main tools an autonomous agent has to agree about a course of action or to resolve conflicts with other agents. As such, you might be interested in doing an internships and/or MSc. thesis about negotiation and there are many angles that you can take, e.g.:

  • Design of negotiation strategies and better opponent models (e.g. using machine learning techniques);
  • Design methods to negotiate with multiple opponents at once, in settings such as procurement and trading. Here, we need to search for smart combinations of multiple deals.
  • Design of a negotiation support system, either advising humans in a particular domain, or even taking over negotiation all together;
  • Design of a negotiation decision model that can capture and reason about uncertain preferences about the user and can query the user for more preference information. In such cases, the agent needs to strike a  balance between increasing the user model accuracy and the inconvenience caused by interacting with the user.
  • Design of a negotiation protocol that can generate better agreements (e.g., multi-agent settings, auctions, use a new form of communication between the agents);
  • Design of a testbed for comparing negotiating agents;
  • Extend the current negotiation setting to more expressive preferences, for instance non-linear utility functions with dependencies between different issues;
  • Design agents in a multi-agent many-to-many setting, for example an auction;
  • Research related to choosing your negotiation partner, for instance trust and the reputation of agents;

These topics are tightly connected with research in AI, including topics such as cooperation & competition, search algorithms, decision making under uncertainty, and learning. An eligible candidate has good mathematical skills, and a passion for iterative research and design: i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion.

Interested applicants may contact Tim Baarslag for more information.

 

Negotiation algorithms and elicitation

Negotiation AI can autonomously negotiate and coordinate with others in our stead, to reach outcomes and agreements in our interest. In such situations, the agent represents a user with individual and a priori unknown preferences, which are costly to elicit due to the user bother this incurs. Therefore, the agent needs to strike a  balance between increasing the user model accuracy and the inconvenience caused by interacting with the user. To do so, we require a tractable metric for the value of information in an ensuing negotiation, which until now has not been available.

Typical research questions include:

  1. How can we design a negotiation decision model that can capture and reason about uncertain utility information about the user (as well as the opponent) and can query the user for more preference information?
  2. What are effective bidding strategies?
  3. Can we formulate a tractable measure of the estimated outcome of a negotiation?
  4. What is a good stopping criterion that can weigh the estimated post-query value of information against the elicitation costs?

A (partial) solution to these questions would yield entirely new systems (procurement platforms being one example) that can negotiate on behalf of a user and that have the ability to query a user to specify their preferences when needed. This fits in a broader vision of an interactive negotiation agent that has knowledge about what questions can be asked at what costs, and can decide, using our method, which question is worth posing given the utility expected to arise from it.

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. An eligible candidate has good mathematical skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants may contact Tim Baarslag for more information.

 

Procurement and Trading

The world of trading is changing rapidly. Whereas in the past, people would contact their vendors personally over phone or e-mail, online peer-to-peer trading platforms now make it possible for vendors to negotiate in a more effective and scalable way. This creates a new opportunity to use artificial intelligence techniques and negotiation agents that can support vendors in their negotiations and eventually make deals autonomously on their behalf. This requires a combination of a state-of-the-art multi-agent and machine learning techniques that can support the user in making efficient offers and learn the preferences of the user as well as of the trading partners.

 We aim to develop fundamental and applicable solutions for agent negotiations and to develop a new online trading framework. Depending on the interests and expertise of the candidate, the project draws on tools and techniques from artificial intelligence, multi-agent learning, combinatorics, preference modeling and elicitation, protocol design, machine learning, automated negotiation, decision support systems, and sequential multi-agent decision-making under uncertainty. One of the scientific challenges in this project will be to formulate negotiation strategies that can strike partial deals with multiple partners at the same time.

The intern or MSc. student will primarily devise algorithms and simulation experiments for concurrent, multi-deal negotiations in face of uncertainty, and will closely collaborate with other project team members and industrial partners for the implementation of experimental tests.

Interested applicants may contact Tim Baarslag for more information.

 

Negotiation in Energy Cooperatives

An energy cooperative is a group of people who share their energy resources, such as their batteries and renewables (e.g. solar panels). By working together, energy cooperatives can reduce their overall costs: for example, when someone produces excess energy and their battery is fully charged, they can share it with their neighbor instead of selling it back to the grid. Of course, this gesture should be returned at some point, and for this, members of an energy cooperative can use negotiation to make deals about the amount of energy to share with each other, and the best time to do so. The candidate is invited to come up with a negotiation protocol for a simple, hands-on scenario (e.g. day/night transfer) and implement a simulation of such energy exchanges. The negotiation platform Genius can be used as a platform to build negotiation agents that represent the members of the cooperative and study their interactions.

Typical research questions include:

- Every cooperative member can have different characteristics with respect to how 'tough' they are in the negotiation. How does the competitiveness of the members influence the system aggregate (e.g. using measures such as the price of anarchy)? What are essential characteristics of its members for a cooperative to remain viable?

- Members of the cooperative interact on a day-to-day basis, so it is important for them to build a good relationship with their peers. How should the members balance their short-term preferences with their follow-up interactions?

These topics are tightly connected to the research in our group, which includes topics such as cooperation & competition, automated negotiation, and learning. The successful applicant can work on building autonomous agents with impact on the real world. An eligible candidate has good programming and writing skills, and a passion for iterative research and design (i.e. come up with of a solution, try it out theoretically or in simulation, and re-assess in an iterative fashion).

Interested applicants in these topics may contact Tim Baarslag for more information.


Playing complex multi-player strategy games

Ever since the idea of artificial intelligence was born, researchers have used games to push its progress. From the conquest of chess by IBM's Deep Blue to that of Go by DeepMind's AlphaGo, and beyond to current cutting-edge research on complex video games like StarCraft 2 or DotA 2, games are full of hard and interesting problems for all core areas of AI. The list of successful applications of game AI techniques to non-game domains is already impressive today, from finance to manufacturing, from transportation to software engineering, and from medicine to space exploration. In the Intelligent and Autonomous Systems group at CWI, applications are currently focused on managing flexibility in future energy markets in order to allow for an increasing share of renewable energy.

As one particular benchmark and test domain for various AI research directions, the IAS group is working with a mobile multi-player turn-based strategy game. It serves as an environment in which many of the techniques that have recently been successful in computer Go can be extended to new challenges: for example partial instead of full observability, multiple players instead of only two, and much larger action and state spaces. Decision-making algorithms developed for this test domain can later be generalized and applied to operating complex systems such as the electrical grid. And of course, developing game AI is also motivating and fun.

A typical internship project within this area could be centred around a research topic such as:

  • Integrating machine learning and search for decision-making in complex domains (see AlphaGo and AlphaZero for an integration of deep reinforcement learning and Monte Carlo Tree Search)
  • Scaling up decision-making (from smaller to larger worlds, lower to higher numbers of units to control, fully observable to partially observable environments, one to many opponents, etc.)
  • Planning and search on multiple levels of abstraction and multiple timescales (strategy vs. tactics, long-term vs. short-term decisions)
  • Opponent modelling (representing and learning the other agents' strategies in order to optimally exploit them)
  • Developing and improving the testing framework.

Internship projects on multi-player strategy games can be theoretical, i.e. contributing to fundamental research on learning, planning, or multi-agent systems; or more practical by applying existing techniques to problems related to multi-player strategy games, or further developing the testing framework. Solid programming and software engineering skills are required. Experience in mobile development, in particular for Flash and through Adobe AIR, are a plus.

Interested applicants can contact Hendrik Baier for more information.