JSONL (JSON Lines): Format, Examples & JSONL vs JSON

Q: Is JSONL the same as NDJSON?

In practice yes. NDJSON (newline-delimited JSON) was specified separately from JSON Lines, but the rules are effectively the same: one JSON value per line, UTF-8, newline separated. Files appear with .jsonl, .ndjson or .jsonlines extensions and are read by the same code. Media types application/x-ndjson and application/jsonl are both in circulation.

JSONL (JSON Lines, also called newline-delimited JSON) is a text format that stores one complete JSON value per line. There is no enclosing array and no comma between records, so every line is a standalone document. That one property is what makes JSONL streamable, appendable and splittable, which is why it has become the default shape for training data, database bulk loads, log pipelines and knowledge graph dumps.

The format rules

The conventions are set out at jsonlines.org and are short enough to state completely.

Each line holds one valid JSON value, almost always an object. The file is UTF-8. Lines are separated by \n; a parser should also tolerate \r\n. A line must not contain a raw newline inside it, which means a record cannot be pretty-printed across several lines. No comma follows a record and no brackets wrap the file. A trailing newline at the end of the final record is conventional and harmless, and a blank line is not a record.

The usual extension is .jsonl. .ndjson (newline-delimited JSON) and .jsonlines are aliases for the same thing, and NDJSON was specified separately with essentially identical rules. Gzipped files are common and normally named .jsonl.gz. There is no registered IANA media type; application/x-ndjson and application/jsonl both circulate, and plain application/json is wrong because the file as a whole is not one JSON document.

Why the format exists

A JSON array is a single document. To read the third object out of [{...},{...},{...}], a parser must consume the opening bracket and the first two records, because record boundaries are only knowable by parsing. To append one record, the closing bracket must be removed, a comma and the record written, and the bracket put back. To validate the file, the whole thing must be well formed: one missing brace anywhere and no record is readable. And a 40 GB array cannot be loaded without 40 GB of memory or a specialized incremental parser.

JSONL removes all of that by moving the record boundary from the syntax to the newline. Four consequences follow, and they are the reason the format is chosen.

Streaming. A reader holds one record in memory at a time regardless of file size, so a 100 GB file processes in a few megabytes of RAM.

Appending. Adding a record is an O(1) write to the end of the file, which is exactly what a logger or a crawler needs. Nothing already written has to be touched.

Splitting. Any newline is a safe split point, so a file can be divided into shards for parallel workers, or read by a distributed system that assigns byte ranges to tasks. This is why cloud warehouses and Spark-style engines prefer it over arrays.

Partial survivability. A corrupt or truncated line invalidates that line only. The other records still parse, so a failed job leaves usable data and a bad record can be logged and skipped rather than aborting the load.

The trade-off is that JSONL cannot express anything above the record level. There is no place for a header, a schema declaration or file-wide metadata, and nothing in the format guarantees that two lines share a shape.

The same data, both ways

Three records as a JSON array:

[
  {"id": "Q243", "name": "Eiffel Tower", "city": "Paris", "height_m": 330},
  {"id": "Q90", "name": "Paris", "country": "France", "population": 2102650},
  {"id": "Q937", "name": "Albert Einstein", "born": 1879}
]

The same three records as JSONL:

{"id": "Q243", "name": "Eiffel Tower", "city": "Paris", "height_m": 330}
{"id": "Q90", "name": "Paris", "country": "France", "population": 2102650}
{"id": "Q937", "name": "Albert Einstein", "born": 1879}

The records are identical. What changed is that the JSONL file has no structure outside its lines, so head -1, wc -l, split, tail -f and grep all work on it, and a reader can stop after the first line without parsing the rest.

JSONL compared to other formats

	JSON	JSONL	CSV	Parquet
Structure	One document	One record per line	Rows and columns	Columnar binary
Streaming reads	Needs an incremental parser	Native, line by line	Native	By row group
Appending	Rewrite the file tail	Append a line	Append a line	Rewrite or add a file
Nested data	Yes	Yes	No (flatten or embed)	Yes
Human readable	Yes	Yes	Yes	No
Compression	Fair	Fair (gzip well)	Fair	Strong, per column
Typed values	JSON types only	JSON types only	Everything is text	Full schema and types
Typical use	APIs, config	Dumps, logs, training data	Spreadsheets, exports	Analytics warehouses

CSV is smaller and simpler but cannot hold nested objects, and its type handling is by convention only. Parquet is far more efficient for analytical scans and carries a real schema, at the cost of being unreadable without tooling and awkward to append to. JSONL sits between them: readable like CSV, nested like JSON, cheap to produce from any language. A common pipeline lands data as JSONL, then converts to Parquet for query.

Where JSONL is encountered

LLM fine-tuning and batch inference. Training files for chat models are conventionally JSONL, one training example per line. The general shape is an object holding a list of messages, each with a role and content:

{"messages": [{"role": "system", "content": "You answer questions about landmarks."}, {"role": "user", "content": "How tall is the Eiffel Tower?"}, {"role": "assistant", "content": "330 metres, including antennas."}]}
{"messages": [{"role": "user", "content": "Where is the Eiffel Tower?"}, {"role": "assistant", "content": "Paris, France."}]}

The exact key names, permitted roles, and optional fields differ by provider and change over time, so the vendor’s current documentation is the only reliable source for the schema. Batch inference endpoints use the same arrangement, with one request object per line and one response object per line in the returned file, matched by a caller-supplied id.

Knowledge graph dumps. Wikidata publishes its full item dump as JSON where the outer container is an array but each entity sits on its own line, so the file is processed line by line in practice by stripping the first and last line. Other projects ship true JSONL. Either way, the reason is the same: the dump is tens of gigabytes, and no consumer wants to parse it as one document. See Wikidata.

Warehouse bulk loads. BigQuery’s NEWLINE_DELIMITED_JSON source format and Snowflake’s JSON loading both expect this layout, because a loader can split the file across slots and report the specific line that failed.

Logs and event pipelines. Structured logging libraries emit one JSON object per event, which is JSONL by construction. Fluent Bit, Vector, Logstash, and most cloud logging exports read and write it.

Datasets and crawlers. Hugging Face datasets are frequently distributed as .jsonl or .jsonl.gz shards. Scrapy writes JSONL with -o out.jsonl (its jsonlines exporter), which matters because a crawl that dies halfway leaves a valid file, while a JSON array export would leave an unclosed bracket.

JSONL and graph data

Two formats do different jobs on this site’s subject. JSON-LD is about meaning: a @context maps keys to IRIs so that a JSON object becomes a set of RDF triples with globally identified subjects and predicates. JSONL is about movement: it says nothing about semantics, only how records sit in a file.

They compose. A JSONL file whose every line is a JSON-LD object is a practical bulk format for graph data, streamable and splittable while each record still expands to triples:

{"@context":"https://schema.org","@id":"http://www.wikidata.org/entity/Q243","@type":"LandmarksOrHistoricalBuildings","name":"Eiffel Tower","location":{"@id":"http://www.wikidata.org/entity/Q90"}}
{"@context":"https://schema.org","@id":"http://www.wikidata.org/entity/Q90","@type":"City","name":"Paris","containedInPlace":{"@id":"http://www.wikidata.org/entity/Q142"}}

This is the same bargain RDF already made with N-Triples and N-Quads: one self-contained statement per line, so the file streams. JSONL gives a JSON-LD pipeline the same property at the entity level rather than the triple level. Repeating @context on every line is the cost, and gzip removes most of it.

Reading and writing JSONL

Line by line in Python, which is the whole technique:

import json

# read
with open("entities.jsonl", encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if not line:
            continue
        record = json.loads(line)
        print(record["name"])

# write (append mode adds to an existing file)
with open("entities.jsonl", "a", encoding="utf-8") as f:
    for record in records:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")

Two details matter. ensure_ascii=False keeps UTF-8 characters readable instead of escaping them, and json.dumps must never be given indent, which would break the one-record-per-line rule.

Gzipped files need only a different opener:

import gzip, json

with gzip.open("dump.jsonl.gz", "rt", encoding="utf-8") as f:
    for line in f:
        record = json.loads(line)

At the shell, jq reads JSONL natively because it consumes a stream of values:

jq -r 'select(.city == "Paris") | .name' entities.jsonl

In pandas, read_json with lines=True loads a JSONL file into a DataFrame, and chunksize returns an iterator when the file is too large for memory. to_json(orient="records", lines=True) writes it back.

import pandas as pd

df = pd.read_json("entities.jsonl", lines=True)
df.to_json("out.jsonl", orient="records", lines=True)

Converting between the two forms is a few lines each way:

import json

# JSON array -> JSONL
with open("in.json", encoding="utf-8") as src, open("out.jsonl", "w", encoding="utf-8") as dst:
    for record in json.load(src):
        dst.write(json.dumps(record, ensure_ascii=False) + "\n")

# JSONL -> JSON array
with open("in.jsonl", encoding="utf-8") as src, open("out.json", "w", encoding="utf-8") as dst:
    records = [json.loads(line) for line in src if line.strip()]
    json.dump(records, dst, ensure_ascii=False, indent=2)

The same conversions in jq are jq -c '.[]' in.json > out.jsonl and jq -s '.' in.jsonl > out.json. Note that the array direction loads everything into memory, so for a large file the streaming Python loop is the safer choice.

For developers

Write JSONL whenever the producer and consumer are decoupled: a crawler, an exporter, a log, an event stream, anything that might be interrupted. Keep every line the same shape and include a stable identifier in every record, so a partial rerun can be deduplicated. Never buffer the whole file into a list before writing; write records as they’re produced. On the read side, wrap json.loads in a try/except that logs the line number and continues, since one bad line should not fail a job. For files that outlive a single script, ship a schema alongside the data (JSON Schema, or a model in the repo), because the format carries none.

For SEOs

JSONL is the format bulk SEO data arrives in, and the one APIs prefer to be fed. Search Console exports, crawl output, and log file samples are all naturally one record per URL per line, which means a 5 million URL crawl can be filtered with grep and jq before anything reaches a spreadsheet. It is also the shape to use when sending an entity list to an API in batches, or preparing pages for entity extraction. What ends up on a page is still JSON-LD; JSONL is the pipeline behind it.

Common mistakes

Pretty-printing a record. json.dumps(record, indent=2) spreads one object across many lines and the file stops being JSONL. Every record must be on exactly one line.

Confusing a trailing newline with a blank line. A final \n after the last record is normal and expected. Two of them create an empty line that json.loads rejects with “Expecting value”, which is why readers should skip empty lines rather than assume they cannot occur.

Mixing schemas between lines. Nothing enforces consistency, so a producer that changes its keys mid-run leaves a file that parses but breaks every consumer. Version the record shape with a field if it must change.

Loading a huge file with json.load. That function expects one JSON document and will fail on the second line, and pointing it at a converted array defeats the purpose. Use json.loads per line, or pandas with lines=True.

FAQ

What is a JSONL file?

A JSONL file is a UTF-8 text file holding one complete JSON value, usually an object, on each line. There is no wrapping array and no comma between records, so every line parses on its own. The extension is .jsonl, with .ndjson and .jsonlines used as aliases, and the format is often shipped gzipped as .jsonl.gz.

What is the difference between JSON and JSONL?

A JSON file is one document, so the whole thing must be parsed to reach any part of it, and appending means rewriting the end of the file. A JSONL file is a sequence of independent documents separated by newlines, so you can read it one record at a time, append to it cheaply, split it across workers, and still use it even if a line is corrupt.

How do I read a JSONL file in Python?

Open the file and call json.loads on each line rather than json.load on the file, skipping blank lines. For gzipped files use gzip.open with mode “rt”. To get a DataFrame instead, use pandas.read_json(path, lines=True), adding chunksize when the file is too large to hold in memory.

Is JSONL the same as NDJSON?

In practice, yes. NDJSON (newline-delimited JSON) was specified separately from JSON Lines, but the rules are effectively the same: one JSON value per line, UTF-8, newline-separated. Files appear with .jsonl, .ndjson or .jsonlines extensions and the same code reads them. Media types application/x-ndjson and application/jsonl are both in circulation.

Sources and further reading

JSON Lines specification: https://jsonlines.org/
NDJSON specification: https://github.com/ndjson/ndjson-spec
RFC 8259, The JavaScript Object Notation (JSON) Data Interchange Format: https://www.rfc-editor.org/rfc/rfc8259
BigQuery, loading JSON data: https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json
pandas.read_json: https://pandas.pydata.org/docs/reference/api/pandas.read_json.html
Wikidata database download: https://www.wikidata.org/wiki/Wikidata:Database_download