Bayesian-DPOP for continuous distributed constraint optimization problems

In this work, the novel algorithm Bayesian Dynamic Programming Optimization Procedure (B-DPOP) is presented to solve multi-agent problems within the Distributed Constraint Optimization Problem framework. The Bayesian optimization framework is used to prove convergence to the global optimum of the B-DPOP algorithm for Lipschitz-continuous...

Security Games with Restricted Strategies: An Approximate Dynamic Programming Approach

In this chapter we consider a security game between an agent and an intruder to find optimal strategies for patrolling against illegal fishery. When patrolling large areas that consist of multiple cells, several aspects have to be taken into account. First, the current risk of the cells has to be considered such that cells with high risk are...

Optimal control for integrated emission management in diesel engines

Integrated Emission Management (IEM) is a supervisory control strategy that minimises operational costs (consisting of fuel and AdBlue) for diesel engines with an aftertreatment system, while satisfying emission constraints imposed by legislation. In most work on IEM, a suboptimal heuristic real-time implementable solution is used, which is...

Real-Time QoS Control for Service Orchestration

Service orchestration has become the predominant paradigm that enables businesses to combine and integrate services offered by third parties. For the commercial viability of orchestrated services, it is crucial that they are offered at sharp price-quality ratios. A complicating factor is that many attractive third-party services often show...

Energy management with TRIANA on FPAI

The current growth of smart grid capable appliances motivates the development of general and flexible software systems to support these devices. The FlexiblePower Application Infrastructure (FPAI) is such a system, which classifies devices by their type of flexibility. Subsequently, energy applications only have to support these flexibility...

Revenue optimization of service compositions using conditional request retries

Due to the inherent stochastic nature of services execution environment within service oriented systems, a runtime adaptation of the given composition may be required. We address a runtime service adaptation mechanism based on conditional retries for the orchestrated web services. The conditional retry may be issued while a concrete service...

Run-time revenue maximization for composite web services with response time commitments

We investigate dynamic decision mechanisms for composite web services maximizing the expected revenue for the providers of composite services. A composite web service is represented by a (sequential) workflow, and for each task within this workflow, a number of service alternatives may be available. These alternatives offer the same...

Dynamic Profit Optimization of Composite Web Services with SLAs

In this paper we investigate sequential decision mechanisms for composite web services. After executing each sub-service within a sequential workflow, decisions are made whether to terminate or continue the execution of the workflow. These decisions are based on observed response times, expected rewards, and typical Service Level Agreement...

Rate adaptation using acknowledgement feedback in finite-state markov channels with collisions

We investigate packet-by-packet rate adaptation so as to maximize the throughput.We consider a finite-state Markov channel (FSMC) with collisions, which models channel fading as well as collisions due to multi-user interference. To limit the amount of feedback data, we only use past packet acknowledgements (ACKs) and past rates as channel state...

A theoretical and empirical analysis of expected sarsa

This paper presents a theoretical and empirical analysis of Expected Sarsa, a variation on Sarsa, the classic onpolicy temporal-difference method for model-free reinforcement learning. Expected Sarsa exploits knowledge about stochasticity in the behavior policy to perform updates with lower variance. Doing so allows for higher learning rates and...

An investigation into the need of a dynamic coupled well-reservoir simulator

Within the research framework of the "Integrated System Approach Petroleum Production" (ISAPP) knowledge center of TNO, TU Delft and Shell, the necessity of taking the interaction between dynamic reservoir and dynamic well behavior into account when optimizing a producing asset is investigated. To simulate dynamic phenomena in the well and in...

Traffic adaptive control of a single intersection: A taxonomy of approaches

The design of a traffic adaptive control system is as well a science as an art. Along the way compromises have to be made in order to end up with a workable system that is not only able to come up with good signal timings, but is also able to deliver them on time. In this paper we propose a taxonomy of the various traffic adaptive control...