S3 private buckets feature does not seem to be usable - confusion between s3 endpoints to publish and read

[squalou]

Hi,

I hope I'm wrong but I tried to follow documentation for "private s3 storage" found here

https://developers.plane.so/self-hosting/govern/private-bucket

and I tried to implement it. I'm using s3 buckets that block public access at account level. (No choice here, best practice and security and etc..).

observations

Here is what happens when I try to create a project for instance :

• fill informations
• click create
• get an error message "Could not upload image"

plane-api can upload files (and delete) just fine, Ican find the project's image file in the bucket.

Yet, the page cannot read that back, I get a "CORS missing blabla" Error. (even when CORS are filled in s3 configuration)

Thing is, using "s3 web hosting" AND private buckets ( = will all public access blocked) seems not possible, which makes sense.

On the other hand, I tried to change "AWS_S3_ENDPOINT_URL" to other value for a check, and I noticed errors in plane-api logs :

packages/botocore/args.py\", line 138, in get_client_args\n
    endpoint = endpoint_creator.create_endpoint(\n
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
  File \"/usr/local/lib/python3.12/site-packages/botocore/endpoint.py\", line 400, in create_endpoint\n
    raise ValueError(f\"Invalid endpoint: {endpoint_url}\")\nValueError: Invalid endpoint:

which I understand as : plane-api is using boto3 lib for AWS s3 sdk use pointing to the given "AWS_S3_ENDPOINT_URL", that has nothing to do with re resource url served over http.

conclusion

Once again I may be wrong but my understanding is that AWS_S3_ENDPOINT_URL is both used for Files uploads using SDK AND for frontend serving files.

It would be really a solution to separate the two, imagine a AWS_S3_WEBHOST_URL (facultative) that would be used to serve files.

With that, it's then easy to set up a CDN (Cloudfront for instance) in front of the private s3 bucket, and everything should work.

(for the records, I may try to tweak file server config following these ideas #3278 but I fear I wont go far)