Skip to content

[server] Support generating rack aware bucket assignment when creating table #786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

swuferhong
Copy link
Collaborator

@swuferhong swuferhong commented Apr 23, 2025

Purpose

Linked issue: close #785

Currently, Fluss only support generate bucket assignment info using round-robin strategy. However, in some cases, like deploy on K8s, we need to avoid multi replicas of one bucket was deployed to same host.
So, in this pr I will introduce one rack aware bucket assignment strategy. For example:

If we have a tabletServer set as follow:
ts-id -> machine rack
0 -> "rack1"
1 -> "rack3"
2 -> "rack3"
3 -> "rack2"
4 -> "rack2"
5 -> "rack1"

First we will create a rack alternated tabletServers list:

[0, 3, 1, 5, 4, 2]

the list is in order rack1 -> rack2 -> rack3 -> rack1 -> rack2 -> rack3, and the preview tabletServerId lower than the next tabletServerId for the same rack.

Then an easy round-robin assignment can be applied. Assume 6 buckets with replica factor of 3, the assignment will be:

bucket0 -> 0,3,1
bucket1 -> 3,1,5
bucket2 -> 1,5,4
bucket3 -> 5,4,2
bucket4 -> 4,2,0
bucket5 -> 2,0,3

Once it has completed the first round-robin, if there are more buckets to assign, the algorithm will start shifting the followers. This is to ensure we will not always get the same set of sequences. In this case, if there is another bucket to assign (bucket6, bucket7), the assignment will be:

bucket6 -> 0,4,2 (instead of repeating 0,3,1 as bucket0)
bucket7 -> 3,2,0 (instead of repeating 3,1,5 as bucket1)

Brief change log

Tests

API and Format

Documentation

@swuferhong swuferhong requested review from wuchong, luoyuxia and loserwang1024 and removed request for luoyuxia April 23, 2025 12:42
@swuferhong
Copy link
Collaborator Author

@wuchong @luoyuxia @loserwang1024 , the pr to support generating rack aware bucket assignment, PTAL, thx.

@loserwang1024
Copy link
Collaborator

loserwang1024 commented Apr 27, 2025

. In this case, if there is another bucket to assign (bucket6, bucket7), the assignment will be:
bucket6 -> 0,4,2 (instead of repeating 0,3,1 as bucket0)
bucket7 -> 3,2,0 (instead of repeating 3,1,5 as bucket1)

8 bucket( * 3) to 8 servers:

  • server 0: 5 buckets
  • server 1: 5 buckets
  • server 2: 4 buckets
  • server 3: 4 buckets
  • server 4: 3 buckets
  • server 5: 3 bucket

8 bucket( * 3) to 3 racks:

  • rack 1 (server 0 & 5): 8 bucket
  • rack 2 (server 3 & 4): 7 bucket
  • rack 3( server 1 & 2): 9 bucket

It seems not even enough for servers and racks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support generating rack aware bucket assignment when creating table
2 participants