MapReduce is a parallel programming model that allows distributed processing on large data sets on a cluster of computers.MapReduce derives its ideas and inspiration from concepts in the world of functional programming.Map and reduce are commonly used functions in the world of functional programming. In functional programming, a map function applies an operation or a function to each element in a list. For example, a multiply-by-two function on a list [1, 2, 3, 4] would generate another list as follows:
[2, 4, 6, 8]. When such functions are applied, the original list is not altered.
Like the map function, functional programming has a concept of a reduce function. Actually, a reduce function in functional programming is more commonly known as a fold function. A reduce or a fold function is also sometimes called an accumulate, compress, or inject function. A reduce or fold function applies a function on all elements of a data structure, such as a list, and produces a single result or output. So applying a reduce function-like summation on the list generated out of the
map function, that is, [2, 4, 6, 8], would generate an output equal to 20.
So map and reduce functions could be used in conjunction to process lists of data, where a function is first applied to each member of a list and then an aggregate function is applied to the transformed and generated list.
Asked In: Many Interviews |