I am trying to extract AWS
Object URL via the following code:
s3AsyncClient.utilities()
.getUrl(r -> r.region(awsRegion).bucket(bucketName).key(key));
For Integration tests where I am using TestContainers
and LocalStack
S3
, the returned URL looks like this:
http://127.0.0.1:51435/my-bucket/games/123.json
The URL looks good to me, but when I try to transform URL
to S3Uri
:
S3Uri s3Uri = s3AsyncClient.utilities().parseUri(url.toURI());
I am getting the following error:
java.lang.IllegalArgumentException: Invalid S3 URI: hostname does not appear to be a valid S3 endpoint: http://127.0.0.1:51435/my-bucket/games/123.json
Seems that the aws parser doesn’t accept 127.0.0.1
as a valid s3
endpoint, but what should I do?
Not sure that I can force the AWS parser to accept 127.0.0.1
as a valid host, but maybe it is possible somehow to configure LocalStack
S3
to return an acceptable hostname instead of an IP address?
Current Localstack
S3
configuration see below:
private static final LocalStackContainer LOCAL_STACK_CONTAINER = new LocalStackContainer(LOCAL_STACK_IMAGE)
.withServices(LocalStackContainer.Service.S3);
static {
LOCAL_STACK_CONTAINER.start();
}
@DynamicPropertySource
static void s3Properties(final DynamicPropertyRegistry registry) {
registry.add("ssp.s3.region", LOCAL_STACK_CONTAINER::getRegion);
registry.add("ssp.s3.endpoint", () -> LOCAL_STACK_CONTAINER.getEndpointOverride(LocalStackContainer.Service.S3));
}
What is s3AsyncClient and where does it come from? I suspect you’re using an unsupported version of the AWS S3 SDK.
@lane.maxwell It comes from here: mvnrepository.com/artifact/software.amazon.awssdk/s3/2.23.3
@lane.maxwell, btw, AmazonS3URI uri = new AmazonS3URI(“127.0.0.1:51435/my-bucket/games/123.json”); that belongs to java sdk v1 also throws exactly the same exception.
It looks like you’ve stumbled on a pseudo bug in LocalStack. I say pseudo because while this does manifest itself in S3 based on the LocalStack configuration, it’s only in the utilities class and doesn’t prevent the S3Client or AsyncS3Client from functioning normally.
@lane.maxwell I guess, you are right, I was able successfully to upload a file to the s3 bucket and successfully received the uploaded file URL, but I wasn’t able to download the file because the AWS parser rejected the URL as invalid.
Show 1 more comment