What Sail supports today
Everything Databasin Sail supports today, from Spark SQL and table maintenance to live sources and Python, and what's coming next, including OLTP mode.
This is the full list of what you can do on Databasin Sail today, followed by what we plan to build next. Each line links to the page with the details. A cluster keeps the Sail build it was created with, so an older cluster may not have everything listed here yet (see Spark compatibility).
Available now
Query and write
| Feature | What you get |
|---|---|
| Spark SQL | Joins, window functions, CTEs, PIVOT and about 500 built-in functions. SQL reference |
| Table writes | INSERT, INSERT OVERWRITE (including dynamic partition overwrite), CREATE TABLE AS, MERGE, UPDATE, DELETE |
| Table formats | Apache Iceberg and Delta Lake, readable by Trino |
| Views | Temporary views, and persistent views in lake catalogs |
| Iceberg maintenance | OPTIMIZE, VACUUM, snapshot expiry, orphan-file removal, manifest rewrites, rollback, ALTER TABLE, metadata tables such as t.snapshots |
| Delta maintenance | OPTIMIZE, VACUUM, RESTORE, DESCRIBE HISTORY and DESCRIBE DETAIL, ALTER TABLE columns |
| Metadata | information_schema on every catalog, SHOW and DESCRIBE, answered even while workers sleep |
| Spark compatibility | Checked against Apache Spark 4.0. Spark compatibility |
Live sources
Make a connection live and it becomes its own catalog on Sail, read-only and joinable with your lakehouse tables:
- Native API connectors, and files in Azure Data Lake Storage or Amazon S3.
- PostgreSQL, MySQL and SQL Server databases that sign in with a username and password.
- Databricks Unity Catalog Delta tables on Azure Databricks.
See Live sources on Sail.
Python
| Feature | What you get |
|---|---|
| Notebooks | Spark - SQL and Spark - Python cells sharing one session, with pandas, NumPy and matplotlib |
| PySpark | The DataFrame API over Spark Connect, from notebooks or your own Python |
| Python UDFs | Python, pandas and Arrow UDFs, UDTFs and Python data sources, on single-user clusters |
| Python packages | A cluster list installed at every start, or %pip install for one notebook session |
| Project files | databasin.files in Python, and Spark reads from /Files/<path> |
See Python and PySpark on Sail.
Connect and operate
| Feature | What you get |
|---|---|
| Outside tools | Tableau, DBeaver and JDBC apps over HiveServer2 (Thrift), Power BI with its Spark connector; PySpark over Spark Connect. Connect outside tools |
| Access rules | The same file-based rules as Trino, checked on every table a statement touches |
| Access modes | Shared (default) or single user |
| Network access | Private, Specific IPs or Open |
| Sizes | XSmall to XLarge, worker count override, coordinator choice. Sail clusters and sizes |
| Sleep and wake | Workers scale to zero when idle; coordinator always on or scaling to zero; scheduled starts with a warm window |
| Query Activity | Statements from every tool, running and in history, with cancel for statements started in Databasin. Members see their own; project admins see everyone's |
Coming soon
OLTP mode
A managed PostgreSQL database beside your Sail cluster, for apps that read and write single rows fast, queryable from Sail as the oltp catalog. The cluster card already shows the OLTP mode switch, marked Coming soon; it can't be turned on yet. See OLTP mode (coming soon).
Planned
These are decided and on our list, without a date yet:
- More live database sources: Snowflake, BigQuery and Oracle, alongside PostgreSQL, MySQL and SQL Server.
- Writes to live sources from Sail. Today live sources are read-only.
- Databricks-style connections for BI tools, so Tableau's Databricks connector and Power BI's Azure Databricks connector can connect to Sail.
- Progress for running statements in Query Activity.
Not supported today
These aren't on Sail now and have no date:
- Row filters and column masks in access rules (a table protected by one is refused).
- Spark ML (MLlib), Structured Streaming, the RDD API and Scala.
- Spark writes to
/Files. Write a table tosandbox, or save a file withdatabasin.files. - Python packages from private indexes or as source packages.
Spark compatibility lists the smaller SQL and DataFrame gaps.