Database driver Parquet

Overview

This guide provides instructions on how to set up and use Parquet files with Team Edition. The Parquet driver allows you to work with Parquet data as if it were in a database. You can retrieve data and apply filters, sorting, and other operations, even combining data from multiple files.

Before you start, you need to create a connection in Team Edition and select the appropriate Parquet driver. If you haven’t done this, see our Database Connection article.

Important: When using the Parquet driver, all connected Parquet files are read-only. To make changes, you need to update the original files outside Team Edition.

Parquet Files driver connection settings

This section describes how to set up a connection using the Parquet driver. The connection settings page requires the following fields:

Field	Description
Connect by (Host/URL)	Choose whether to connect using a local host path or a URL.
File paths	Specify the location of the Parquet file(s). You can:
	- File: Select a single Parquet file.
	- Folder: Choose a directory containing multiple Parquet files.
Driver name	This field will be auto-filled based on your selected driver type.
Driver Settings	If there are any specific driver settings, configure them here.

Tip: When using the Folder option, Team Edition scans the directory up to two levels deep for Parquet files. For more information, see folder structure. If you select a folder, Team Edition organizes files in schemas based on their directory structure.

Features and capabilities

Advanced SQL query capabilities

The Parquet driver supports the full range of SQL queries:

• Simple queries (e.g., SELECT * FROM table): Data is read directly from the Parquet file.

• Complex queries (e.g., using WHERE, JOIN, ORDER BY, GROUP BY): When a complex query is executed for the first time, the driver imports the entire Parquet file into an internal database to enable advanced SQL functions. Subsequent queries run faster because the data is already imported into internal database.

Folder structure

When working with a folder containing multiple Parquet files, Team Edition organizes them as follows:

Folder structure	Schema in Team Edition
Root files	Default schema
Subfolder files	Schema named after the subfolder
Files in deeper folders	Ignored

If your folder looks like this:

Data/
├── employees.parquet
├── sales.parquet
└── Reports/
    └── monthly.parquet
    └── yearly.parquet

Team Edition will create:

• Default schema: employees, sales
• Reports schema: monthly, yearly

Tip: To focus on specific files, consider selecting individual files or folders when configuring the connection.

Internal database

When you execute a complex query (such as WHERE, JOIN, GROUP BY, or ORDER BY.), on a Parquet file for the first time, the Parquet driver processes the data by importing it into a temporary internal SQLite database.

By default, this internal database stores data temporarily on the disk during your session and is cleared when Team Edition restarts. To speed up queries on the same file in future sessions, you can specify the internalDbFilePath option in the Driver properties tab (e.g., C:\User\database.db) to reuse the processed data.

Additional features

Team Edition provides additional features compatible with Parquet driver, but not exclusive to it:

Category	Feature
Data Transfer	Data Export
Data Visualization	Visual Query Builder
	Charts