我正在处理多个作业，我需要使用一个全局数组值。我在函数设置 (Mapper) 中使用了数组，我需要在函数清理 (Reducer) 中更改它。 在创建作业之前，我读取了一个包含此值的序列文件，然后我使用了 conf.setInt()。 在 Cleanup (Reducer) 中，我用新数组编写了一个序列文件。 我面临这个问题:

13/11/19 10:58:23 INFO mapred.JobClient: Task Id : attempt_201311190929_0005_m_000015_0, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 134.
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

attempt_201311190929_0005_m_000015_0: centers.size() - 10
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: # A fatal error has been detected by the Java Runtime Environment:
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: #  SIGSEGV (0xb) at pc=0x00007fed37941b78, pid=6494, tid=140656104789760
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: # JRE version: 7.0_25-b30
attempt_201311190929_0005_m_000015_0: # Java VM: OpenJDK 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops)
attempt_201311190929_0005_m_000015_0: # Problematic frame:
attempt_201311190929_0005_m_000015_0: # V  [libjvm.so+0x6b2b78]  methodOopDesc::get_c2i_entry()+0x8
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: # An error report file with more information is saved as:
attempt_201311190929_0005_m_000015_0: # /tmp/hadoop-vega/mapred/local/taskTracker/akuma/jobcache/job_201311190929_0005/attempt_201311190929_0005_m_000015_0/work/hs_err_pid6494.log
attempt_201311190929_0005_m_000015_0: #
attempt_201311190929_0005_m_000015_0: # If you would like to submit a bug report, please include
attempt_201311190929_0005_m_000015_0: # instructions on how to reproduce the bug and visit:
attempt_201311190929_0005_m_000015_0: #   https://bugs.launchpad.net/ubuntu/+source/openjdk-7/

我很绝望，不知道该怎么办=/

    public class Teste7 extends Configured implements Tool {
        public static int[] vetor;   
        private static int choosenIteration = 10;

        @Override    
        public int run(String[] strings) throws Exception {

        while(interations < choosenIteration) {

                    interations ++;
                    fs = FileSystem.get(conf);            
                    conf = new Configuration();            
                    conf.setInt("n", n);    

                    //This File was written on Cleanup (Reduce), I need to read it to pass to map 
                    try (SequenceFile.Reader reader = new SequenceFile.Reader(fs, dir, conf)) {            
                        IntWritable key = (IntWritable) reader.getKeyClass().newInstance();
                        IntWritable value = (IntWritable) reader.getValueClass().newInstance();                               
                        int a = 0;
                        while (reader.next(key, value)){        
                            System.out.println("vetor " + vetor[a]);
                            vetor[a] = value.get();                    
                            conf.setInt("vet["+a+"]", vetor[a]);
                            a++;
                        }              
                        reader.close();
                    } catch (InstantiationException | IllegalAccessException ex) {
                        Logger.getLogger(Teste7.class.getName()).log(Level.SEVERE, null, ex);
                    }

                    job = new Job(conf);            
                    job.setJarByClass(Teste7.class);
                    job.setJobName("KMeans Locão!");

                    if(fs.exists(saida)){       
                        fs.delete(saida, true);             
                    }

                    job.setMapperClass(MapTeste7.class);
                    job.setReducerClass(RedTeste7.class);
                    job.setJarByClass(Teste7.class);

                    job.setInputFormatClass(SequenceFileInputFormat.class);       

                    FileInputFormat.addInputPath(job, entrada);
                    FileOutputFormat.setOutputPath(job, saida);        

                    job.setOutputKeyClass(IntWritable.class);
                    job.setOutputValueClass(VectorWritable.class);

                    job.waitForCompletion(true);         
                }
}

    public class RedTeste7 extends Reducer<IntWritable, VectorWritable, IntWritable, VectorWritable> {
     @Override
        protected void cleanup(Context context) throws IOException, InterruptedException {     

            Configuration conf = context.getConfiguration();izado
            fs = FileSystem.get(conf); 
            try (SequenceFile.Writer write = SequenceFile.createWriter(fs, conf, dir, IntWritable.class, IntWritable.class)){
                IntWritable oi;
                IntWritable qtd;
                for(int i = 0; i < _partitonNumber; i++){
                    oi = new IntWritable(i);
                    qtd = new IntWritable(vetor[i]);
                    write.append(oi,qtd);  
                }
                write.close();
            }
        }
}

最佳答案

这表明 JNI 代码中存在错误，或者(更可能的是，给定段错误的框架)运行任务的节点上的内存错误。

关于java - 多重就业和全局值(value)，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/20072772/