hadoop - Hive query stuck at 99% -


i inserting records using left joining in hive.when set limit 1 query works records query stuck @ 99% reduce job.

below query works

   insert overwrite table tablename select a.id , b.name left join b on a.id = b.id limit 1;  

but not

    insert overwrite table tablename select table1.id , table2.name table1 left join table2 on table1.id = table2.id; 

i have increased number of reducers still doesn't work.

hive automatically optimizations when comes joins , loads 1 side of join memory if fits requirements. in cases these jobs stuck @ 99% , never finish.

i have faced multiple times , way have avoided explicitly specifying settings hive. try settings below , see if works you.

  1. hive.auto.convert.join=false
  2. mapred.compress.map.output=true
  3. hive.exec.parallel=true

Comments