[File system] Add Google Cloud Storage integration

[gkatzioura]

Purpose

Linked issue: close #630

Brief change log

Added the GSFileSystem based on the given Hadoop Google Cloud Storage file system

Tests

API and Format

No

Documentation

It adds support for the Google Cloud Storage Filesystem

[gkatzioura]

Will add integration tests. Putting temporary to draft

[gkatzioura]

Added an integration test based on FileSystemBehaviorTestSuite.
For the integration test to run it requires a Google Cloud Storage bucket declared on the variable IT_CASE_GS_BUCKET
and a path of the service account in the IT_CASE_GS_ACCESS_KEY. The service account file needs to be mounted during the CI process.

Since the service account key is in a file form and not in a string form I see two options.

1. Changing the ci.yaml to read the service account file (base64) from secrets and store it to a path.
2. During the integration test initialization read the service account file (base64) and store it to a path.

In the pr I followed the second approach.

[wuchong]

@luoyuxia could you help to review it?

[wuchong]

@gkatzioura , could you check how Flink tests the gs-filesystem in CI?

[gkatzioura]

Hi @wuchong , after checking there are not tests for the actual filesystem integration.
Will proceed on adding a mock server thus avoiding the need for integrating with an actual service account.

[wuchong]

Thank you @gkatzioura , please ping me when you have finished the mock server.

[gkatzioura]

Hi @wuchong & @luoyuxia
I implemented a server simulating Google Cloud Storage, thus the now the test runs pointing to that server. No credentials or actual GCP account needed.
The test is similar to the ones found on the FileSystemTest.java.
Should we need to bring back the Google Cloud Storage integration test can do on demand.

[wuchong]

Can we remove this? It breaks the CI build. Currently, we still need to ship artifacts build in Java8.

Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project fluss-fs-gs: Fatal error compiling: invalid target release: 9 -> [Help 1]

https://github.com/alibaba/fluss/actions/runs/14344481121/job/40238125178?pr=631

[wuchong]

checkstyle are failed

2025-04-14T15:08:32.2197637Z [INFO] --- maven-jar-plugin:2.4:jar (default) @ fluss-fs-oss ---
2025-04-14T15:08:32.2261014Z [INFO] There are 2 errors reported by Checkstyle 9.3 with /tools/maven/checkstyle.xml ruleset.
2025-04-14T15:08:32.2283668Z [ERROR] src/test/java/com/alibaba/fluss/fs/gs/GSPathServerHandler.java:[42,1] (javadoc) MissingJavadocType: Missing a Javadoc comment.
2025-04-14T15:08:32.2289030Z [ERROR] src/test/java/com/alibaba/fluss/fs/gs/MockAuthServer.java:[35,1] (javadoc) MissingJavadocType: Missing a Javadoc comment.

[wuchong]

@luoyuxia could you help to review this PR?

[luoyuxia]

@luoyuxia could you help to review this PR?

Sure, now I got some free time. I'll help review these days.

[luoyuxia]

@gkatzioura Thanks for the greate work! Left some comments.

[wuchong]

Hi @gkatzioura , you can "request review" again when you have addressed all the comments, so that the reviewer can get the notification in time.