Spice v0.17.1-beta (August 5, 2024)
The v0.17.1-beta minor release focuses on enhancing stability, performance, and usability. The Flight interface now supports the GetSchema
API and s3
, ftp
, sftp
, http
, https
, and databricks
data connectors have added support for a client_timeout
parameter.
Highlights in v0.17.1-betaโ
Flight API GetSchema: The GetSchema
API is now supported by the Flight interface. The schema of a dataset can be retrieved using GetSchema
with the PATH
or CMD
FlightDescriptor types. The CMD
FlightDescriptor type is used to get the schema of an arbitrary SQL query as the CMD bytes. The PATH
FlightDescriptor type is used to retrieve the schema of a dataset.
Client Timeout: A client_timeout
parameter has been added for Data Connectors: ftp
, sftp
, http
, https
, and databricks
. When defined, the client timeout configures Spice to stop waiting for a response from the data source after the specified duration. The default timeout is 30 seconds.
datasets:
- from: ftp://remote-ftp-server.com/path/to/folder/
name: my_dataset
params:
file_format: csv
# Example client timeout
client_timeout: 30s
ftp_user: my-ftp-user
ftp_pass: ${secrets:my_ftp_password}