GraphCheck: Test Your Neo4j Graph Like You Test Your Code

23 Aug 2026 • Graphora Team

Your application code has tests. Your data warehouse has dbt tests. But the knowledge graph underneath your product, your migration, or your GraphRAG pipeline? For most teams, that graph is verified by hope.

The problem is sharper in 2026 because graphs are increasingly machine-built. LLM extraction pipelines write nodes and edges at scale, and extraction accuracy is good but not perfect. The result is a familiar complaint: "our graph looks healthy but gives wrong answers." Counts look right. Dashboards stay green. And a query that matters returns the wrong thing because an edge went missing three weeks ago.

GraphCheck is our answer: an open-source CLI that tests a Neo4j property graph the way pytest tests code. You declare checks in YAML, GraphCheck runs them read-only, and you get deterministic pass/fail verdicts with evidence — in your terminal and in CI.

This post walks through the whole flow on a real database.

Install

GraphCheck ships on PyPI. It needs Python 3.12+ and a running Neo4j 5.x — the 5 LTS line or a current calendar release. (Neo4j 4.4 is legacy and unsupported.)

1pip install graphcheck

The base install is deliberately lean — a small set of runtime dependencies, installed in seconds. Optional capabilities (AI-assisted check drafting, the MCP agent server) live behind extras: pip install "graphcheck[generate]" or pip install "graphcheck[mcp]".

1$ graphcheck --version
2graphcheck 0.2.0

Scaffold a project

Run init in an empty directory:

1$ graphcheck init
2Wrote graphcheck.yml
3Wrote profiles.yml
4Profile setup help is included in profiles.yml
5Wrote checks/example.yml with 2 sample checks

Three files matter:

graphcheck.yml — project configuration.
profiles.yml — connection details (gitignored; use password_env in CI).
checks/example.yml — a starter suite you will replace with your own checks.

init immediately probes your Neo4j and tells you what it found. If it cannot connect, the message contains the fix, not a stack trace:

1Neo4j was not detected: neo4j.unreachable
2Neo4j is unreachable at the configured Bolt URI.
3Fix: Start Neo4j, verify the host and port in `uri`, then run `graphcheck debug` again.

Point profiles.yml at your database:

1default: local
2profiles:
3  local:
4    uri: bolt://localhost:7687
5    user: neo4j
6    password_env: NEO4J_PASSWORD
7    database: neo4j

Verify the connection

graphcheck debug diagnoses the connection and reports exactly what your credentials can and cannot do:

1$ graphcheck debug
2Profile: local
3GraphCheck version: 0.2.0
4Neo4j Server: 5.26.28
5Cypher: 5
6Edition: community
7APOC: no
8Count store: yes
9Credentials can see: connect, read, procedures
10Blocked checks: none

Two details worth noticing:

Read-only is enforced, not assumed. On Neo4j Enterprise and Developer editions, GraphCheck expects a server-enforced read-only credential (the built-in reader role) and additionally runs a server-side EXPLAIN preflight that rejects any query Neo4j classifies as write-capable. On Community Edition, where roles are limited, the EXPLAIN guard covers every query.
Everything here is also available as stable JSON (graphcheck debug --json) for tooling.

Declare what should be true

Checks live in YAML under checks/. Three patterns cover most of what a graph needs:

1suite: customer-quality
2defaults: { severity: error, tags: [production] }
3
4conformance:
5  # A declarative rule: every Customer should have a name.
6  - id: customer-name-present
7    check: completeness
8    with: { label: Customer, property: name }
9
10competency:
11  # A business question as an executable contract.
12  - id: customers-can-be-counted
13    question: "Can customers be counted?"
14    query: "MATCH (c:Customer) RETURN count(c) AS count"
15    expect: { rows: { min: 1 }, columns: [count] }
16
17# Drift checks require a baseline. Create one with `graphcheck profile`,
18# then uncomment the drift check below.
19#
20# drift:
21#   - id: customer-count-stable
22#     metric: node_count
23#     target: { label: Customer }
24#     tolerance: { max_drop_pct: 10 }
25#     severity: warn

Conformance checks are declarative quality rules from the built-in core pack — completeness, cardinality, orphan detection, and friends. A built-in PII pack flags properties that look like personal data, with clearly labeled confidence.
Competency checks turn the questions your graph exists to answer into executable assertions about shape and cardinality — no hand-pinned expected values required.
Drift checks compare the current graph against a baseline snapshot. They stay commented out here because a first run has no baseline yet: create one with graphcheck profile, then enable the check.

Run

1$ graphcheck run
2Target: neo4j · Neo4j 5.26.28 community · 5 nodes · 0 relationships
3GraphCheck run neo4j_20260820T155958Z: complete
4Score breakdown by check suite:
5
6Suite             Score  Check Coverage  Passed  Failed  Warnings  Errored  Skipped
7customer-quality 50/100             2/2       1       1         -        -        -
8
9Result: 1 failure.
10Results and Report saved to: .graphcheck/runs/latest

One customer in this demo graph is missing a name, and the run caught it. Crucially, the failure comes with evidence: results.json records the exact element IDs that violated the check, alongside the compiled query. A finding you cannot trace is a finding you cannot trust.

Every run writes two artifacts to .graphcheck/runs/latest/:

results.json — the machine-readable contract: verdicts, evidence pointers, coverage, score, and reproducibility metadata (graph fingerprint, server version, suite hash).
report.html — a self-contained report that opens offline. No CDN, no external requests, safe to open in an air-gapped environment.

When the graph is clean, GraphCheck says so honestly — it never manufactures findings:

1Suite             Score  Check Coverage  Passed  Failed
2customer-quality 100/100            2/2       2       -
3
4Result: No failures. All 2 selected checks passed.

Gate your CI on it

The exit codes are a frozen contract:

Exit code	Meaning
`0`	All executed checks passed
`1`	An error-severity finding or an execution error
`2`	Warnings, or an incomplete evaluation
`3`	The run could not be prepared or completed

Which makes CI integration a one-liner in any pipeline:

1# .github/workflows/graphcheck.yml
2- name: Run graph checks
3  env:
4    NEO4J_PASSWORD: ${{ secrets.NEO4J_PASSWORD }}
5  run: |
6    pip install graphcheck
7    graphcheck run --select tag:production

A failing check fails the build, the same way a failing unit test would. Use --suite and --select tag: to scope what runs where, and --fail-fast to stop on the first error-severity failure.

What we deliberately did not build

GraphCheck's trust model is worth stating plainly:

It never writes to your graph. Read-only credential plus EXPLAIN preflight.
Verdicts are deterministic. No LLM ever decides pass or fail. (The optional generate command drafts candidate checks with AI — but they are inert until a human reviews and enables them.)
Estimates are labeled. Sampled checks carry their sample size and confidence.
Nothing leaves your environment. Telemetry is off by default, anonymous when opted in.

Try it

1pip install graphcheck

Documentation: docs.graphora.io/graphcheck
Source: github.com/graphora/graphcheck
Package: pypi.org/project/graphcheck

If you run a production Neo4j graph and want experienced eyes on it first, we also offer a fixed-scope, read-only graph quality audit — you keep the runnable check suite we build for your data.

We would genuinely like to hear where GraphCheck breaks or confuses. Issues and PRs welcome.