Search before asking
Description
PR #66805 made Lance storage options provider-aware and forwards one Lance-native option map from the FE to the BE. It intentionally left non-S3 providers in pass-through mode: Doris still contributes no static OSS options, and a Lance filesystem catalog rejects an oss:// warehouse.
Add native Alibaba Cloud OSS support for Lance catalogs. Doris should translate its typed oss.* properties to Lance's public OSS option vocabulary:
| Doris property |
Lance storage option |
oss.endpoint |
oss_endpoint |
oss.access_key |
oss_access_key_id |
oss.secret_key |
oss_secret_access_key |
oss.region |
oss_region |
oss.session_token |
oss_security_token |
Lance normalizes these public names internally to endpoint, access_key_id, access_key_secret, region, and security_token. When a Namespace vends either spelling, Doris should collapse only these known aliases before merging so that the vended value deterministically overrides the static catalog value. Unknown Namespace options must continue to pass through unchanged.
The Lance filesystem catalog should also accept an oss:// warehouse.
Use case
- Read Lance datasets directly from Alibaba Cloud OSS through a filesystem catalog configured with Doris
oss.* properties.
- Read
oss:// Lance tables described by a REST Namespace that vends OSS storage options, including a catalog with no static credentials in Doris.
Acceptance requires FE metadata access and a BE scan for both catalog types.
Related issues
Not in scope: role-based credential vending, STS credential refresh during a scan, a new Thrift envelope, or static support for other object-store providers.
Are you willing to submit PR?
Code of Conduct
Search before asking
Description
PR #66805 made Lance storage options provider-aware and forwards one Lance-native option map from the FE to the BE. It intentionally left non-S3 providers in pass-through mode: Doris still contributes no static OSS options, and a Lance filesystem catalog rejects an
oss://warehouse.Add native Alibaba Cloud OSS support for Lance catalogs. Doris should translate its typed
oss.*properties to Lance's public OSS option vocabulary:oss.endpointoss_endpointoss.access_keyoss_access_key_idoss.secret_keyoss_secret_access_keyoss.regionoss_regionoss.session_tokenoss_security_tokenLance normalizes these public names internally to
endpoint,access_key_id,access_key_secret,region, andsecurity_token. When a Namespace vends either spelling, Doris should collapse only these known aliases before merging so that the vended value deterministically overrides the static catalog value. Unknown Namespace options must continue to pass through unchanged.The Lance filesystem catalog should also accept an
oss://warehouse.Use case
oss.*properties.oss://Lance tables described by a REST Namespace that vends OSS storage options, including a catalog with no static credentials in Doris.Acceptance requires FE metadata access and a BE scan for both catalog types.
Related issues
Not in scope: role-based credential vending, STS credential refresh during a scan, a new Thrift envelope, or static support for other object-store providers.
Are you willing to submit PR?
Code of Conduct