java - 分頁(yè)查詢中如何使用多線程加快處理速度?
問(wèn)題描述
一個(gè)歷史表中有大量的數(shù)據(jù),現(xiàn)在要通過(guò)分頁(yè)式查詢處理轉(zhuǎn)換數(shù)據(jù)。
現(xiàn)在將處理數(shù)據(jù)的邏輯放在線程池中處理,以加快處理流程。
可是總是出現(xiàn)事務(wù)方面的異常
比如 : SQLNonTransientConnectionException
請(qǐng)問(wèn)該如何解決上述異常,或者有什么好的多線程分頁(yè)查詢處理方案?
原來(lái)問(wèn)題描述不太清楚,現(xiàn)在添加以下代碼(手敲,如果有錯(cuò),請(qǐng)多包含)
分頁(yè)式查詢邏輯:
int pageSize = 100;int currentPageLength = 0;int pageIndex = 0;ExecutorService exe = newFixedThreadPool(Runtime.getRuntime().availableProcessors());do { int offset = pageIndex * pageSize; List<TradeInfo> tradeInfos = tradeInfoService.findTradeInfoBysPage(queryParams,offset,pageSize); if (null != tradeInfos && tradeInfos.size() > 0) {currentPageLength = tradeInfos.size();TradeInfoProcesserTask task = new TradeInfoProcesserTask(tradeInfos );exe.execute(task);pageIndex++; }else{System.out.println('Page Query TradeInfo Got NOTHING! Break query loop!');break; }} while (currentPageLength == pageSize);exe.shutdown();while(true) { if(exe.isTerminated()){doOtherThings();System.out.println('分頁(yè)式多線程處理數(shù)據(jù)完畢!');break; }}
數(shù)據(jù)處理邏輯:
public class TradeInfoProcesserTask implements Runnable{ private volatile List<TradeInfo> tradeInfos;public TradeInfoProcesserTask (List<TradeInfo> _tradeInfos){tradeInfos = _tradeInfos; }@Override public void run() {processTradeInfos(); }private void processTradeInfos(){//do something with tradeInfos ..... }}
問(wèn)題解答
回答1:邏輯先不說(shuō)?,F(xiàn)在沒(méi)有判斷多線程是否全部執(zhí)行完,while循環(huán)完就shutdown。。。將CountDownLatch通過(guò)構(gòu)造器傳入線程
ExecutorService exe = newFixedThreadPool(Runtime.getRuntime().availableProcessors());CountDownLatch latch = new CountDownLatch(?); //?代表開(kāi)啟全部線程的數(shù)量do { int offset = pageIndex * pageSize; List<TradeInfo> tradeInfos = tradeInfoService.findTradeInfoBysPage(queryParams,offset,pageSize); if (null != tradeInfos && tradeInfos.size() > 0) {currentPageLength = tradeInfos.size();TradeInfoProcesserTask task = new TradeInfoProcesserTask(tradeInfos, latch);exe.execute(task);pageIndex++; }else{System.out.println('Page Query TradeInfo Got NOTHING! Break query loop!');break; }} while (currentPageLength == pageSize);latch.await(); //多線程全部執(zhí)行完exe.shutdown();doOtherThings();System.out.println('分頁(yè)式多線程處理數(shù)據(jù)完畢!');
public class TradeInfoProcesserTask implements Runnable{ private volatile List<TradeInfo> tradeInfos; private CountDownLatch latch;public TradeInfoProcesserTask (List<TradeInfo> _tradeInfos, CountDownLatch latch){tradeInfos = _tradeInfos;this.latch = latch; }@Override public void run() {processTradeInfos();latch.countDown(); }private void processTradeInfos(){//do something with tradeInfos ..... }}回答2:
分頁(yè)查詢是不并發(fā)(DAO)的,數(shù)據(jù)處理是并發(fā)(Service),所以你的事務(wù)級(jí)級(jí)別是設(shè)置在哪個(gè)環(huán)節(jié)呢?
相關(guān)文章:
1. 老師,請(qǐng)問(wèn)我打開(kāi)browsersync出現(xiàn)這個(gè)問(wèn)題怎么解決???2. java - Oracle如何獲取去重結(jié)果集中某一條數(shù)據(jù)的下一條數(shù)據(jù)3. python - pyspider的分布式運(yùn)行成功,2臺(tái)slave跑,但是時(shí)間并沒(méi)有縮短問(wèn)題?4. python - 數(shù)據(jù)無(wú)法插入到mysql表里5. npm install -g browser-sync這個(gè)之后出錯(cuò) 還有人嗎 我都感覺(jué)沒(méi)人回答問(wèn)題了6. javascript - 小程序跳轉(zhuǎn)失敗?7. atom開(kāi)始輸入!然后按tab只有空格出現(xiàn)沒(méi)有html格式出現(xiàn)8. ubuntu - elasticsearch-h(huán)ead插件安裝后,啟動(dòng)問(wèn)題!9. mysql scripts提示 /usr/bin/perl: bad interpreter10. javascript - ie11以下單擊打開(kāi)不了file,雙擊可以。求解?
