Formats
File formats available for VPNDetection datasets and how to work with them.
Datasets support the following formats:
- CSV (
csvgz) - MMDB (
mmdb) - JSON (
jsonl/ndjson) (Coming Soon) - Parquet (
parquet) (Coming Soon)
Note: Datasets that don't have an IP-based key will not be available in mmdb format, since mmdb is keyed by IP.
CSV Format (csvgz)
csvgz is a CSV file compressed using the gzip (GNU zip) algorithm.
Column names are always included as the first row.
Cells where a value is NULL or the empty string are represented without any value. For example, here the provider is NULL and represented without any cell value:
start_ip,end_ip,provider,confidence,last_seen
0.0.0.0,0.0.0.0,,low,2026-01-01See how to decompress the CSV file.
MMD Format (mmdb)
mmdb is the MMDB file format that allows for a fully in-memory O(1) IP lookup database.
The MMDB files are generally much larger than their CSV equivalents, but can be used for serving API requests at very high throughputs and scale.
The following tools are available to work with MMDBs:
- https://github.com/maxmind/mmdbconvert
- https://github.com/maxmind/mmdbinspect
- https://github.com/ipinfo/mmdbctl
If a dataset does NOT have an IP address as its key (i.e. any one of ip, start_ip, end_ip), it won't be available in MMDB format, as MMDBs are designed only for IP datasets.
JSON Format (jsonl / ndjson)
The JSON files are json-lines (jsonl) and newline-delimited (ndjson) format files compressed using the gzip (GNU zip) algorithm.
Both formats are available but are exactly the same - we recommend using jsonl which is more commonplace.
Each line in the file is an individual self-describing JSON object. For example:
{"start_ip": "0.0.0.0", "end_ip": "0.0.0.0", "provider": "", "confidence": "low", "last_seen": "2026-01-01"}See how to decompress the JSON file.
Parquet
Coming soon.
Decompression
CSV & JSON files are always compressed using the gzip (GNU zip) algorithm. Decompression is necessary to properly read the files.
They be decompressed using any tool that supports decompression of the gzip algorithm. Here are some examples for a Linux terminal environment with different utilities:
# Using `gunzip`
gunzip file.csv.gz
gunzip file.jsonl.gz
# Using `gzip`
gzip -d file.csv.gz
gzip -d file.jsonl.gz
# Using `zcat`
zcat file.csv.gz > file.csv
zcat file.jsonl.gz > file.jsonl