A Combiner is an optional class that acts as a Reducer for the output key–value pairs that are sent by a particular Mapper. A Combiner helps in minimising the data transferred between the Map and Reduce tasks. Thus, it helps in reducing the size of the intermediate data, thereby saving disk and network I/O. The Reducer class can itself be used as the Combiner if the Reduce function is commutative and associative like integer addition and multiplication.
However, if the Reduce function is an average function which is not commutative and associative, then you need to write a separate class for the Combiner.
Comments