Bug: Error (Invalid Path) while attaching external KuzuDB from S3

[rishabh-oswal-opensc]

Kuzu version

v0.7.1

What operating system are you using?

MacOS Sequoia

What happened?

When trying to attach an external Kuzu DB from S3 using the ATTACH statement, the query fails with the error

Runtime exception: Cannot attach a remote kuzu database due to invalid path: s3://<my-bucket>/<my-path>.

I can confirm that the S3 path exists. The S3 bucket is not public. But before that I pass the credentials using the call statement. Also install and load the httpfs extension.

call s3_access_key_id='<my-creds>';
call s3_secret_access_key='<my-creds>';
call s3_region='eu-central-1';

I have received the same error via Kuzu explorer running on Docker and Kuzu sdk via Python.
Via Python I tried setting the credentials by both below ways and received the same error.

1. Call statement
2. Env variables.

Documentation that we have referred:

• https://docs.kuzudb.com/extensions/httpfs/#aws-s3-file-system
• https://docs.kuzudb.com/extensions/attach/kuzu/

What we would like to understand is,

1. It seems that the error message is very generic. What is the exact error message that we are facing here ?

Thank You for the help with this issue.

Are there known steps to reproduce?

No response

Are there known steps to reproduce?

No response

[acquamarin]

Hi @rishabh-oswal-opensc
Can you paste the command that you used to attach s3 hosted kuzu there?
Have you tried attach the same db in duckdb and do they give an error message?

Thanks,

Ziyi

[rishabh-oswal-opensc]

Hi @rishabh-oswal-opensc Can you paste the command that you used to attach s3 hosted kuzu there? Have you tried attach the same db in duckdb and do they give an error message?

Thanks,

Ziyi

Hi @acquamarin ,

Sure, the command that we are using is:

CALL s3_access_key_id='<my-creds>';
CALL s3_secret_access_key='<my-creds>';
CALL s3_region='eu-central-1';
INSTALL httpfs;
LOAD EXTENSION httpfs;
CALL HTTP_CACHE_FILE=TRUE;
ATTACH 's3://my-bucket/my-path-to-kuzu-db' AS uw (dbtype kuzu);

No we have not tried to attach using DuckDb. Our DB in S3 is a KuzuDB which we create from json files.

Thanks for helping us with the issue!
Rishabh

[acquamarin]

Hi @rishabh-oswal-opensc
I think you have to set your s3_acccess_key_id, s3_secret_access_key and s3_region after the httpfs extension is loaded.
Those options are only available after the extension has been loaded.
If the problem still exists, can you try the following to narrow down the issue:

1. Host an arbitrary csv file on your s3 bucket, and scan from duckdb. This is to confirm whether there is something wrong with the access control on your s3 bucket.
2. If duckdb can scan the csv file, then there must be a bug on our side.

Thanks

[prrao87]

Hi @rishabh-oswal-opensc, just checking in again on this - have you been able to try out the steps suggested by @acquamarin above?

[rishabh-oswal-opensc]

Hi @prrao87 , @acquamarin ,

Sorry I wasn't able to in the last few days due to other prios. But I will reply back with an answer by Monday.
Thank you again for the help!