Setup

Before running any of the examples, it is necessary to first setup a warehouse, database and tables for writing in Snowflake. You can create the required Snowflake objects directly through the Snowflake console, or API’s, but the examples below use the Ray Data Snowflake connector.

Create a warehouse

For this example to work, we need to first create a Snowflake Warehouse for processing queries and a database to write results into.

from ray_db.provider.snowflake import SnowflakeConnector

# Snowflake settings
WAREHOUSE_SIZE = 'XSMALL'
WAREHOUSE_NAME = f'ANYSCALE_{WAREHOUSE_SIZE}'

with SnowflakeConnector.create() as connector:
    connector.query(f'''
        CREATE WAREHOUSE IF NOT EXISTS {WAREHOUSE_NAME} 
        WITH WAREHOUSE_SIZE={WAREHOUSE_SIZE}'''
    )

Create a database

The examples and benchmarks will also require a database for writing tables to.

DEST_DATABASE = 'ANYSCALE_SAMPLE'

with SnowflakeConnector.create() as connector:
    connector.query(f'CREATE DATABASE IF NOT EXISTS {DEST_DATABASE}')

Test the Snowflake connection

First, test the connection to be sure everything works prior to executing a ray db read or wrote operation.

SOURCE_TABLE = 'SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.CUSTOMER'

with SnowflakeConnector.create(warehouse='XSMALL') as connector:
    one_row = connector.query_value(f'SELECT count(*) FROM {SOURCE_TABLE}')
    print(one_row)

150000