What is a .parquet file?
Apache Parquet · Data · Apache Software Foundation · 2013
Parquet is a columnar storage format built for analytics on large datasets. Instead of storing rows together, it stores each column together, which makes reading a few columns from a huge table fast and highly compressible.
What is a .parquet file used for?
Parquet is the default format for big-data tools like Apache Spark, and for data lakes and warehouses. Because data is grouped by column, a query that touches three columns of a two-hundred-column table reads only what it needs.
Each file begins and ends with the four bytes PAR1, wrapping compressed column chunks and a footer that holds the schema and statistics. It is a binary format, so you read it with a library such as pandas, DuckDB or Spark rather than a text editor.
What opens a .parquet file?
The .parquet format opens in the following apps, grouped by operating system. If one app does not work, try another from the same list.
Windows
- Python (pandas, pyarrow)
- DuckDB
- Tad viewer
macOS
- Python (pandas, pyarrow)
- DuckDB
- Tad viewer
Linux
- Python (pandas, pyarrow)
- DuckDB
- Apache Spark
Open a .parquet file step by step
- Confirm the file really is a .parquet. Drop it into the WhatFileType identifier, which reads its signature in your browser without uploading anything.
- On Windows, open it with Python (pandas, pyarrow) or DuckDB.
- On Mac, use Python (pandas, pyarrow) or DuckDB.
- On Linux, try Python (pandas, pyarrow) or DuckDB.
- Still stuck? Convert the .parquet to CSV, which opens more widely.
Why can't I open my .parquet file?
Almost always because no installed app understands Parquet. Install one of the apps above, or convert the file to a format your existing software already reads. If double-clicking opens the wrong program instead of nothing at all, the file association is set to the wrong app rather than the file being broken.
The other possibility is that the file is not really a .parquet at all. Drop it into the file identifier to check it by its bytes instead of its name. Nothing is uploaded.
How to convert a .parquet file
You can convert a .parquet file to CSV, JSON, Arrow using a conversion tool or the export menu of an app that opens it.
What is the .parquet file signature (magic bytes)?
A .parquet file starts with the hex bytes 50 41 52 31, which read as PAR1 in ASCII. Programs identify Parquet by those bytes, not by the filename, so a renamed file still gives itself away.
These are the documented signatures for Parquet. For a full breakdown, see the Apache Parquet file signature page.
| Hex signature | Offset | ASCII | Note |
|---|---|---|---|
| 50 41 52 31 | 0 | PAR1 | 'PAR1' header |
| 50 41 52 31 | -4 | PAR1 | 'PAR1' footer at end of file |
| 50 41 52 31 | end of file | PAR1 | PAR1 also closes the file |
Parquet also ends with a trailer. The bytes 50 41 52 31 appear at the end of the file, so a truncated download is spotted by checking the last bytes as well as the first.
See the full magic-numbers reference for every format.
Frequently asked questions
What is a .parquet file?
Parquet is a columnar storage format built for analytics on large datasets. Instead of storing rows together, it stores each column together, which makes reading a few columns from a huge table fast and highly compressible.
How do I open a Parquet file?
Use a data tool: in Python, pandas.read_parquet() or DuckDB's SELECT * FROM 'file.parquet'. Desktop viewers like Tad open it without code.
Why is Parquet smaller than CSV?
Storing each column together lets Parquet compress similar values and skip columns a query does not need, so it is usually a fraction of the size of the same data as CSV.
Is Parquet faster than CSV?
For analytics, yes. Columnar layout and built-in statistics let tools read only the relevant columns and row groups instead of scanning every line.
Comparisons
Related formats
Sources
Details on this page were checked against authoritative references. How sources are ranked and what is deliberately left out is set out in the methodology.