我正在尝试以这种方式过滤 HBase 中的结果:

List<Filter> andFilterList = new ArrayList<>();
SingleColumnValueFilter sourceLowerFilter = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("source"), CompareFilter.CompareOp.GREATER, Bytes.toBytes(lowerLimit));
sourceLowerFilter.setFilterIfMissing(true);
SingleColumnValueFilter sourceUpperFilter = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("source"), CompareFilter.CompareOp.LESS_OR_EQUAL, Bytes.toBytes(upperLimit));
sourceUpperFilter.setFilterIfMissing(true);
SingleColumnValueFilter targetLowerFilter = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("target"), CompareFilter.CompareOp.GREATER, Bytes.toBytes(lowerLimit));
targetLowerFilter.setFilterIfMissing(true);
SingleColumnValueFilter targetUpperFilter = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("target"), CompareFilter.CompareOp.LESS_OR_EQUAL, Bytes.toBytes(upperLimit));
targetUpperFilter.setFilterIfMissing(true);

andFilterList.add(sourceUpperFilter);
andFilterList.add(targetUpperFilter);

FilterList andFilter = new FilterList(FilterList.Operator.MUST_PASS_ALL, andFilterList);

List<Filter> orFilterList = new ArrayList<>();
orFilterList.add(sourceLowerFilter);
orFilterList.add(targetLowerFilter);
FilterList orFilter = new FilterList(FilterList.Operator.MUST_PASS_ONE, orFilterList);

FilterList fl = new FilterList(FilterList.Operator.MUST_PASS_ALL);
fl.addFilter(andFilter);
fl.addFilter(orFilter);

Scan edgeScan = new Scan();
edgeScan.setFilter(fl);
ResultScanner edgeScanner = table.getScanner(edgeScan);
Result edgeResult;
logger.info("Writing edges...");
while ((edgeResult = edgeScanner.next()) != null) {
    // Some code
}

此代码启动此错误:

org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
    at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:402)
    at org.deustotech.internet.phd.framework.rdf2subdue.RDF2Subdue.writeFile(RDF2Subdue.java:150)
    at org.deustotech.internet.phd.framework.rdf2subdue.RDF2Subdue.run(RDF2Subdue.java:39)
    at org.deustotech.internet.phd.Main.main(Main.java:32)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: 178 number_of_rows: 100 close_scanner: false next_call_seq: 0
    at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3098)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
    at java.lang.Thread.run(Thread.java:745)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:285)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:59)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
    at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:354)
    ... 9 more
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException): org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: 178 number_of_rows: 100 close_scanner: false next_call_seq: 0
    at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3098)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
    at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
    at java.lang.Thread.run(Thread.java:745)

    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1453)
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:29900)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:174)
    ... 13 more

RPC 超时设置为 600000。鉴于这些结果，我尝试删除一些过滤器:

• sourceUpperFilter && (sourceLowerFilter || targetLowerFilter) --> 成功
• targetUpperFilter && (sourceLowerFilter || targetLowerFilter) --> 成功
• (sourceUpperFilter && targetUpperFilter) && (sourceLowerFilter) --> 失败
• (sourceUpperFilter && targetUpperFilter) && (targetLowerFilter) --> 失败

如有任何帮助，我们将不胜感激。谢谢。

最佳答案

我通过设置hbase.client.scanner.caching解决了这个问题

see also

Client and RS maintain a nextCallSeq number during the scan. Every next() call from client to server will increment this number in both sides. Client passes this number along with the request and at RS side both the incoming nextCallSeq and its nextCallSeq will be matched. In case of a timeout this increment at the client side should not happen. If at the server side fetching of next batch of data was over, there will be mismatch in the nextCallSeq number. Server will throw OutOfOrderScannerNextException and then client will reopen the scanner with startrow as the last successfully retrieved row.

Since the problem is caused by the client-side overtime, then the corresponding reduction in client cache (hbase.client.scanner.caching) size or increase rpc timeout time (hbase.rpc.timeout) can be.

希望这个回答对您有所帮助。

关于java - 在 HBase 中过滤结果时出现 OutOfOrderScannerNextException，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/24801507/