TestContainers and LocalStack S3: Invalid S3 URI: hostname does not appear to be a valid S3 endpoint

I am trying to extract AWS Object URL via the following code:

s3AsyncClient.utilities()
                .getUrl(r -> r.region(awsRegion).bucket(bucketName).key(key));

For Integration tests where I am using TestContainers and LocalStack S3, the returned URL looks like this:

        http://127.0.0.1:51435/my-bucket/games/123.json

The URL looks good to me, but when I try to transform URL to S3Uri:

        S3Uri s3Uri = s3AsyncClient.utilities().parseUri(url.toURI());

I am getting the following error:

java.lang.IllegalArgumentException: Invalid S3 URI: hostname does not appear to be a valid S3 endpoint: http://127.0.0.1:51435/my-bucket/games/123.json

Seems that the aws parser doesn’t accept 127.0.0.1 as a valid s3 endpoint, but what should I do?
Not sure that I can force the AWS parser to accept 127.0.0.1 as a valid host, but maybe it is possible somehow to configure LocalStack S3 to return an acceptable hostname instead of an IP address?

Current Localstack S3 configuration see below:

    private static final LocalStackContainer LOCAL_STACK_CONTAINER = new LocalStackContainer(LOCAL_STACK_IMAGE)
            .withServices(LocalStackContainer.Service.S3);

    static {
        LOCAL_STACK_CONTAINER.start();
    }

    @DynamicPropertySource
    static void s3Properties(final DynamicPropertyRegistry registry) {
        registry.add("ssp.s3.region", LOCAL_STACK_CONTAINER::getRegion);
        registry.add("ssp.s3.endpoint", () -> LOCAL_STACK_CONTAINER.getEndpointOverride(LocalStackContainer.Service.S3));
    }

  • What is s3AsyncClient and where does it come from? I suspect you’re using an unsupported version of the AWS S3 SDK.

    – 

  • @lane.maxwell It comes from here: mvnrepository.com/artifact/software.amazon.awssdk/s3/2.23.3

    – 

  • @lane.maxwell, btw, AmazonS3URI uri = new AmazonS3URI(“127.0.0.1:51435/my-bucket/games/123.json”); that belongs to java sdk v1 also throws exactly the same exception.

    – 

  • It looks like you’ve stumbled on a pseudo bug in LocalStack. I say pseudo because while this does manifest itself in S3 based on the LocalStack configuration, it’s only in the utilities class and doesn’t prevent the S3Client or AsyncS3Client from functioning normally.

    – 

  • @lane.maxwell I guess, you are right, I was able successfully to upload a file to the s3 bucket and successfully received the uploaded file URL, but I wasn’t able to download the file because the AWS parser rejected the URL as invalid.

    – 

Leave a Comment