Skip to content

Commit de88b9b

Browse files
Load: fix memory leak when failed to send chunk data in first phase (#15518) (#15542)
This commit addresses a memory leak issue in the load process by ensuring that memory usage is reduced even when the dispatch of a piece node fails. The key changes include: - Storing the result of the dispatch call in a variable (isDispatchSuccess) before reducing memory usage. - Deducting the memory usage prior to checking the dispatch result to avoid leaks. - Returning false immediately after the reduction when the dispatch fails. (cherry picked from commit 417ddd0)
1 parent 52adf9a commit de88b9b

File tree

1 file changed

+9
-5
lines changed

1 file changed

+9
-5
lines changed

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/scheduler/load/LoadTsFileScheduler.java

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -663,12 +663,8 @@ private boolean addOrSendChunkData(ChunkData chunkData) throws LoadFileException
663663
if (pieceNode.getDataSize() == 0) { // total data size has been reduced to 0
664664
break;
665665
}
666-
if (!scheduler.dispatchOnePieceNode(pieceNode, replicaSet)) {
667-
return false;
668-
}
666+
final boolean isDispatchSuccess = scheduler.dispatchOnePieceNode(pieceNode, replicaSet);
669667

670-
dataSize -= pieceNode.getDataSize();
671-
block.reduceMemoryUsage(pieceNode.getDataSize());
672668
regionId2ReplicaSetAndNode.replace(
673669
sortedRegionId,
674670
new Pair<>(
@@ -678,6 +674,14 @@ private boolean addOrSendChunkData(ChunkData chunkData) throws LoadFileException
678674
singleTsFileNode
679675
.getTsFileResource()
680676
.getTsFile()))); // can not just remove, because of deletion
677+
dataSize -= pieceNode.getDataSize();
678+
block.reduceMemoryUsage(pieceNode.getDataSize());
679+
680+
if (!isDispatchSuccess) {
681+
// Currently there is no retry, so return directly
682+
return false;
683+
}
684+
681685
if (isMemoryEnough()) {
682686
break;
683687
}

0 commit comments

Comments
 (0)