java - mongodb分片集群下,count和聚合統(tǒng)計問題
問題描述
在mongodb分片集群下,直接用count統(tǒng)計會不準(zhǔn)確,用聚合統(tǒng)計則可以@Testpublic void testCount() throws Exception { DynamicSqlParameter dsp = new DynamicSqlParameter(); long sT = System.currentTimeMillis(); MongoDatasource mongoDatasource = MongoDatasource.getInstance(mongoService.getDatasource()); DBCollection dbCollection = mongoDatasource.getDB().getCollection('dayFlow'); List arrayList = new ArrayList<>(); DBObject dbObject1 = new BasicDBObject(); dbObject1.put('usedDayFlow', 2); DBObject dbObject2 = new BasicDBObject(); dbObject2.put('_id', null); dbObject2.put('count', new BasicDBObject('$sum', 1)); arrayList.add(new BasicDBObject('$match', dbObject1)); arrayList.add(new BasicDBObject('$group', dbObject2)); System.out.println(JSON.serialize(arrayList)); AggregationOutput size = dbCollection.aggregate(arrayList); System.out.println(size.results()); System.out.println('運行時間:' + ((System.currentTimeMillis() - sT) /1000) + 's');}執(zhí)行結(jié)果:
[ { '$match' : { 'usedDayFlow' : 2}} , { '$group' : { '_id' : null , 'count' : { '$sum' : 1}}}]
[{ '_id' : null , 'count' : 1002223}]
該統(tǒng)計結(jié)果比實際數(shù)據(jù)量要多一些,請教大神,對于分片集群的聚合統(tǒng)計要如何操作?問題解答
回答1: 該問題已經(jīng)解決,使用的是最新驅(qū)動mongo-java-driver-3.4.0,通過下面的方法可以在分片集群模式下,準(zhǔn)確的統(tǒng)計到記錄數(shù)量,感謝大家的相助!mongo shell >> db.collection.aggregate([{$match:{categories:'Bakery'},{$group:{'_id':null,'count':{$sum:1}}}}])public long getCount() {String user = '用戶名';String database = 'admin';String password = '密碼';MongoCredential credential = MongoCredential.createCredential(user,database, password.toCharArray());MongoClientOptions options = MongoClientOptions.builder().connectionsPerHost(10).threadsAllowedToBlockForConnectionMultiplier(10).socketTimeout(20000).connectTimeout(15000).maxWaitTime(50000).build();MongoClient mongoClient = new MongoClient(new ServerAddress('IP地址', '端口'), Arrays.asList(credential), options);MongoDatabase mongoDatabase = mongoClient.getDatabase('數(shù)據(jù)庫');MongoCollection<Document> collection = mongoDatabase.getCollection('數(shù)據(jù)表');final long[] count = new long[1];Block<Document> printBlock = new Block<Document>() { @Override public void apply(final Document document) { count[0] = (long) document.get('count'); }};Bson bson = Filters.eq('categories', 'Bakery');collection.aggregate(Arrays.asList(Aggregates.match(bson),Aggregates.group(null, Accumulators.sum('count', 1L)))).forEach(printBlock);return count[0];}回答2:
能否補充一些評論中的信息。多謝!
將評論的內(nèi)容轉(zhuǎn)發(fā)在這里,方便查看:
1、count和aggregate的不同:在mongoDB中,count和aggregate是在兩支不同的程序中實現(xiàn)的,aggregate的實現(xiàn)是考慮到了shard的環(huán)境的,所以官方文檔是推薦使用aggregate來進(jìn)行shard環(huán)境下的count。
2、MongoDB shell下使用aggregate和使用Java MongoDB驅(qū)動使用aggregate來進(jìn)行count,結(jié)果應(yīng)該是一樣的,因為兩者都是使用的aggregate。
您提到Issue大概是MongoDB shell和Java MongoDB驅(qū)動進(jìn)行count的結(jié)果不一致。
這種不一致,我覺得可能是:
1)比較的過程有沒有紕漏;2)所用的Java MongoDB驅(qū)動是否有紕漏。
供參考。
Love MongoDB! Have Fun!
今晚8點,MongoDB中文社區(qū)大神在線講座,請大家踴躍參與;此大神常駐本版哦!
請戳此鏈接。
