3 posts tagged with "oss"

Announcing the release of Spice v1.0-rc.3 🧊

Spice v1.0.0-rc.3 is the third release candidate for the first major version of Spice.ai OSS. This release continues the focus on production readiness and includes new Iceberg Catalog APIs, DuckDB improvements, and a new Iceberg Catalog Connector.

Highlights in v1.0-rc.3​

• Iceberg Catalog APIs: Spice now functions as an Iceberg Catalog provider, implementing a core subset of the Iceberg Catalog APIs. This enables Iceberg Catalog clients native discovery of datasets and schemas through Spice APIs.

• GET /v1/namespaces - List all catalogs registered in Spice.

• GET /v1/namespaces?parent=catalog - List schemas registered under a given catalog.

• GET /v1/namespaces/:catalog_schema/tables - List tables registered under a given schema.

• GET /v1/namespaces/:catalog_schema/tables/:table - Get the schema of a given table.

• Iceberg Catalog Connector: The Iceberg Catalog Connector is a new integration to discover and query datasets from a remote Iceberg Catalog.

Example connecting to a remote Iceberg Catalog with tables stored in S3:

catalogs:
  - from: iceberg:https://my-iceberg-catalog.com/v1/namespaces
    name: ice
    params:
      iceberg_s3_access_key_id: ${secrets:ICEBERG_S3_ACCESS_KEY_ID}
      iceberg_s3_secret_access_key: ${secrets:ICEBERG_S3_SECRET_ACCESS_KEY}
      iceberg_s3_region: us-east-1

View the Iceberg Catalog Connector documentation for more details.

• DuckDB Improvements: Added cosine_distance support for DuckDB-backed vector search, improved unnest nested type handling for array_element and lists, and optimized query performance.

• SQLite Data Accelerator: Graduated to Release Candidate (RC).

• File Data Accelerator: Graduated to Release Candidate (RC).

Announcing the release of Spice v1.0-rc.2 🔗

Spice v1.0.0-rc.2 is the second release candidate for the first major version of Spice.ai OSS. This release continues to build on the stability of Spice for production use, including key Data Connector graduations, bug fixes, and AI features.

Highlights in v1.0-rc.2​

• MS SQL and File Data Connectors: Graduated from Alpha to Beta.

• GraphQL and Databricks Delta Lake Data Connectors: Graduated from Beta to Release Candidate.

• gospice SDK Release: The Spice Go SDK has updated to v7.0, adding support for refreshing datasets and upgrading dependencies.

• Azure AI Support: Added support for both LLMs and embedding models. Example spicepod.yml configuration:

embeddings:
  - name: azure
    from: azure:text-embedding-3-small
    params:
      endpoint: https://your-resource-name.openai.azure.com
      azure_api_version: 2024-08-01-preview
      azure_deployment_name: text-embedding-3-small
      azure_api_key: ${ secrets:SPICE_AZURE_API_KEY }
models:
  - name: azure
    from: azure:gpt-4o-mini
    params:
      endpoint: https://your-resource-name.openai.azure.com
      azure_api_version: 2024-08-01-preview
      azure_deployment_name: gpt-4o-mini
      azure_api_key: ${ secrets:SPICE_AZURE_TOKEN }

Accelerate subsets of columns: Spice now supports acceleration for specific columns from a federated source. Specify the desired columns directly in the Refresh SQL for more selective and efficient data acceleration.

Example spicepod.yaml configuration:

datasets:
  - from: s3://spiceai-demo-datasets/taxi_trips/2024/
    name: taxi_trips
    params:
      file_format: parquet
    acceleration:
      refresh_sql: SELECT tpep_pickup_datetime, tpep_dropoff_datetime, trip_distance, total_amount FROM taxi_trips

Announcing the release of Spice v1.0-rc.1 🚀

Spice v1.0.0-rc.1 marks the release candidate for the first major version of Spice.ai OSS. This milestone includes key Connector and Accelerator graduations and bug fixes, positioning Spice for a stable and production-ready release.

Highlights in v1.0-rc.1​

API Key Authentication: Spice now supports optional authentication for API endpoints via configurable API keys, for additional security and control over runtime access.

Example Spicepod.yml configuration:

runtime:
  auth:
    api-key:
      enabled: true
      keys:
        - ${ secrets:api_key } # Load from a secret store
        - my-api-key # Or specify directly

Usage:

• HTTP API: Include the API key in the X-API-Key header.
• Flight SQL: Use the API key in the Authorization header as a Bearer token.
• Spice CLI: Provide the --api-key flag for CLI commands.

For more details on using API Key auth, refer to the API Auth documentation.

DuckDB Data Connector: Has graduated from Beta to Release Candidate.

Arrow and DuckDB Data Accelerators: Both have graduated from Beta to Release Candidates.

Debezium Kafka Integration: Spice now supports secure authentication and encryption options for Kafka connections when using Debezium for Change Data Capture (CDC). The previous limitation of PLAINTEXT protocol-only connections has been lifted. Spice now supports the following Kafka security configurations:

• Security protocol: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL
• SASL mechanisms: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512

Example Spicepod.yml configuration:

datasets:
  - from: debezium:my_kafka_topic_with_debezium_changes
    name: my_dataset
    params:
      kafka_security_protocol: SASL_SSL
      kafka_sasl_mechanism: SCRAM-SHA-512
      kafka_sasl_username: kafka
      kafka_sasl_password: ${secrets:kafka_sasl_password}
      kafka_ssl_ca_location: ./certs/kafka_ca_cert.pem