Hybrid Capacity Planning Methodology for Web Search Engines

Capacity planning studies are suitable for supporting decision making in management and operation of Web search engines deployed on large clusters of processors. Among many possibilities, they enable ensuring that a sufficient amount of computational resources are timely provisioned to efficiently deal with the ever changing streams of user queries. In this paper, we present a simulation based methodology devised to perform capacity planning in large scale Web search engines. It combines classical operational analysis formulae with discrete event simulation to significantly reduce the number of deployments that are evaluated to find an optimal assignment for a target workload. We experimentally evaluate our proposal for demanding cases such as service nodes with temporary failures. The results show that the proposed methodology is able to produce good quality results in practical running times. © 2018 Elsevier B.V.