Wait, it takes more then a few minutes to spin up the hadoop mapreduce job. Are you using mapreduce for preprocessing or did you optimize hadoop for this scenario?