Release 2.1.0

What's Changed

Added

• PostgreSQL backend support — DataJoint now supports PostgreSQL as an alternative to MySQL. Use dj.config['database.backend'] = 'postgresql' to connect to PostgreSQL databases. (#1338, #1339, #1340)

• Diagram improvements (#1345)

  • New collapse() method for high-level pipeline views
  • Mermaid output format support via output='mermaid'
  • Schema grouping with module labels
  • Direction control (direction='LR' or direction='TB')
  • Default diagram direction changed from TB to LR

• Singleton tables — Support for tables with empty primary keys (#1341)

Changed

• Performance: Lazy-load deepdiff and tqdm in autopopulate for faster imports (#1349)
• Packaging: Switched from setuptools to hatchling for build system (#1358)

Deprecated

• The migrate module shows deprecation warning (#1373)

Fixed

• Allow table class names with underscores (with warning) (#1375)

Documentation

• Converted all docstrings to NumPy style (#1378)

Full Changelog: v2.0.2...v2.1.0

DataJoint 2.0.2

Bug Fixes

• fix: Support 'KEY' in fetch() for backward compatibility (#1384)

  • Restores fetch('KEY') syntax from DataJoint 0.14
  • Fixes #1381

• fix: Handle inhomogeneous array shapes in to_arrays() (#1382)

  • Correctly handles blob arrays with different shapes
  • Fixes #1380

• fix: Handle semantic_check for job table operations (#1383)

  • Fixes populate(reserve_jobs=True) when keep_completed=True
  • Fixes #1379

• fix: Handle missing SSL context in multiprocess populate (#1377)

  • Prevents errors when SSL context is not available in child processes

Installation

pip install datajoint==2.0.2

DataJoint 0.14.8

Bug Fixes

• fix: Add config option to skip filepath checksum on insert (#1387)
  • New filepath_checksum_size_limit_insert config option
  • Prevents transaction timeouts when inserting large files with filepath attributes in three-part make() methods
  • Fixes #1386

Usage

import datajoint as dj

# Skip checksum on insert for files > 1GB
dj.config['filepath_checksum_size_limit_insert'] = 1024 * 1024 * 1024

Installation

pip install datajoint==0.14.8

Release 2.0.1

⚡️ Enhancements

• fix: Remove setuptools, ipython, matplotlib, faker, and urllib3 from runtime dependencies(#1372)@dimitri-yatsenko

🐛 Bug Fixes

• fix: Allow table class names with underscores (with warning)(#1375)@dimitri-yatsenko
• fix: make fetch a class method of user tables for backward compatibility with pre-v2.0(#1375)@dimitri-yatsenko

Full Changelog: v2.0.0...v2.0.1

Release 2.0.0

DataJoint 2.0 - Computational Foundation for Agentic Data Pipelines

This is a major release representing a complete rewrite of the DataJoint Python library. It introduces a modernized architecture with an extensible type system, object-augmented schemas, semantic matching, and improved developer experience.

Related:

• PR #1311 — Complete rewrite implementation
• Discussion #1235 — DataJoint 2.0 design
• Discussion #1354 — Object-Augmented Schemas (OAS)
• Discussion #1256 — Extensible type system
• Discussion #1243 — Semantic matching and lineage

💥 Breaking Changes

Platform Requirements

• Python 3.10+ required - Dropped support for Python 3.9 and earlier
• MySQL 8.0+ required - Dropped support for MySQL 5.x and pre-8.0 versions

Architecture Changes

• New package structure - Source code moved to src/datajoint/
• Extensible Type/Codec System - New <codec> syntax replaces hardcoded blob/attach handling. Custom codecs extend dj.Codec with encode()/decode() methods
• Object-Augmented Schemas (OAS) - Schema-addressed storage (<object@>, <npy@>) creates browsable paths mirroring database structure
• Semantic Matching with Lineage - ~lineage table tracks attribute origins. Joins/restrictions enforce homologous namesakes must share lineage
• Table-Specific Jobs Tables - Each Computed/Imported table has its own ~~table_name jobs table (replaces shared jobs table)
• New Configuration System - pydantic-settings based config with datajoint.json, .secrets/ directory, and DJ_* environment variables
• New Test Infrastructure - Uses testcontainers for automatic MySQL/MinIO management (no manual docker-compose required)

Removed/Deprecated Features

• dj.conn() interactive prompts - Use environment variables or config file
• dj.kill() and dj.kill_quick() - Use database administration tools
• otumat dependency - S3 credential management simplified
• Positional tuple inserts deprecated - Use dict with explicit field names
• ~log table deprecated - Schema-level logging table no longer used

🚀 Major Features

Core Type System

Scientist-friendly type names with portable semantics:

• Numeric: float32, float64, int64, int32, int16, int8, bool
• Special: uuid (binary(16)), json, bytes (longblob)
• Temporal: date, datetime
• String: char(n), varchar(n), enum(...)
• Fixed-point: decimal(m,n)

Extensible Codec System

class GraphCodec(dj.Codec):
    name = "graph"
    def get_dtype(self, is_store): return "<blob>"
    def encode(self, value, *, key=None, store_name=None): ...
    def decode(self, stored, *, key=None): ...

# Use in definitions: data : <graph>

Built-in codecs: <blob>, <blob@>, <attach>, <attach@>, <hash@>, <object@>, <npy@>, <filepath@>

Object-Augmented Schemas (OAS)

• Hash-addressed (<blob@>, <attach@>, <hash@>): Content-addressed with MD5 deduplication (base32-encoded, 26 chars). Paths: _hash/{hash[:2]}/{hash[2:4]}/{hash}
• Schema-addressed (<object@>, <npy@>): Paths mirror schema structure: {schema}/{table}/{pk}/{attribute}
• Filepath references (<filepath@>): Reference existing files in stores without copying
• Lazy references: NpyRef and ObjectRef provide metadata access without I/O

Semantic Matching

• Lineage tracking identifies attribute origins (schema.table.attribute)
• Binary operations (join, restrict, union, aggr) enforce lineage compatibility
• Use schema.rebuild_lineage() for legacy schema migration

Jobs 2.0

• Per-table job queues with ~~table_name naming pattern
• Composite index (status, priority, scheduled_time) for efficient job fetching
• Improved error tracking and job status management

New Query Operator

• extend(other) - Left-joins a functionally dependent table, preserving primary key and row count

Modernized Output Methods

• keys() - Returns list of primary key dicts
• to_arrays(*attrs) - Returns tuple of numpy arrays
• to_dicts() - Returns list of dictionaries
• to_pandas() - Returns pandas DataFrame
• to_polars() - Returns Polars DataFrame
• to_arrow() - Returns PyArrow Table
• fetch() preserved with deprecation warning for backward compatibility

Configuration Enhancements

• datajoint.json project config with parent directory search
• .secrets/ directory for sensitive values (gitignore this)
• database.database_prefix setting for automatic schema name prefixing
• database.create_tables setting to control automatic table creation
• dj.config.override() context manager for temporary config changes

📚 Documentation

Documentation has been moved to a dedicated repository and completely rewritten using the Diátaxis framework:

• Live site: https://docs.datajoint.com
• Repository: https://github.com/datajoint/datajoint-docs

Structure:

• Tutorials — Learn by building real pipelines (Jupyter notebooks)
• How-To Guides — Practical task-oriented guides
• Explanation — Understanding concepts and design
• Reference — Specifications and API documentation
• Migration Guide — Upgrade from legacy versions

⚖️ License Change

DataJoint 2.0 is released under Apache 2.0 license (previously LGPLv2.1).

0.14.7

🐛 Bug Fixes

• fix: Pass make_kwargs to make_fetch in tripartite pattern (#1360) @dimitri-yatsenko

When using generator-based make (make_fetch, make_compute, make_insert), make_kwargs passed to populate() were not being forwarded to make_fetch. This caused TypeError when using make_kwargs with the tripartite pattern.

Fixes #1350

---

⚠️ End-of-Life Notice

This is the final maintenance release for the 0.14.x branch.

• No further 0.14.x releases are planned
• There will be no v0.15 — the next major version is v2.0
• Security fixes only will be considered on a case-by-case basis

We encourage all users on 0.14.x to plan their migration to v2.0.

---

Full Changelog: v0.14.6...v0.14.7

Release 0.14.6

⚡️ Enhancements

• update documentation and devcontainer(#1250)@dimitri-yatsenko
• Update version 0.14.5(#1249)@kavenk

📝 Documentation

• update documentation and devcontainer(#1250)@dimitri-yatsenko
• Update version 0.14.5(#1249)@kavenk

Full Changelog: v0.14.5...v0.14.6

Release 0.14.5

• [github-actions]Update version.py to Release 0.14.4(#1231)@github-actions[bot]

⚡️ Enhancements

• Fix error message when deleting without delete privilege(#1247)@dimitri-yatsenko
• fix: improve error handling when make_fetch referential integrity fails(#1245)@ttngu207
• document autopopulate.make logic(#1241)@dimitri-yatsenko
• Revert "fix(URL): broken routing and redirects on docs.datajoint.com"(#1239)@dimitri-yatsenko
• fix(URL): broken routing and redirects on docs.datajoint.com(#1238)@MilagrosMarin
• chore: yambottle->drewyangdev(#1236)@drewyangdev
• fix: update home URL from datajoint.com/docs to docs.datajoint.com(#1237)@MilagrosMarin
• Document dj.Top() and add missing pages(#1197)@kushalbakshi

🐛 Bug Fixes

• fix: improve error handling when make_fetch referential integrity fails(#1245)@ttngu207

📝 Documentation

• Fix error message when deleting without delete privilege(#1247)@dimitri-yatsenko
• Revert "fix(URL): broken routing and redirects on docs.datajoint.com"(#1239)@dimitri-yatsenko
• fix(URL): broken routing and redirects on docs.datajoint.com(#1238)@MilagrosMarin
• chore: yambottle->drewyangdev(#1236)@drewyangdev
• fix: update home URL from datajoint.com/docs to docs.datajoint.com(#1237)@MilagrosMarin
• Document dj.Top() and add missing pages(#1197)@kushalbakshi

Full Changelog: v0.14.4...v0.14.5

Release 0.14.4

• fix #1170: Support long make calls (#1204)@dimitri-yatsenko
• Update development.yaml(#1207)@yambottle
• Update development.yaml(#1206)@yambottle
• Test Python 3.13 in CI(#1193)@tabedzki

⚡️ Enhancements

• Update pyproject.toml(#1229)@dimitri-yatsenko
• fix: 📝 update home url(#1227)@yambottle
• fix: 🔥 remove redundant contribution.md | check docs developer guide(#1226)@yambottle
• datajoint-python developer guide(#1225)@yambottle
• fix: 🐛 test/release status badge url typo(#1224)@yambottle
• Dev 846 dj release(#1223)@yambottle
• feat: ✨ dependabot for action version's auto update(#1222)@yambottle
• Dev 846 release ci fix(#1216)@yambottle
• Dev 861 pre commit(#1212)@yambottle
• Dev 861 stale issues(#1208)@yambottle
• DEV-861-auto-label(#1209)@yambottle

🐛 Bug Fixes

• fix: 📝 update home url(#1227)@yambottle
• fix: 🔥 remove redundant contribution.md | check docs developer guide(#1226)@yambottle
• fix: 🐛 test/release status badge url typo(#1224)@yambottle
• Fix #1218(#1219)@yambottle
• fix: 🐛 fix rate limit and rename(#1214)@yambottle

📝 Documentation

• fix: 📝 update home url(#1227)@yambottle
• fix: 🔥 remove redundant contribution.md | check docs developer guide(#1226)@yambottle
• datajoint-python developer guide(#1225)@yambottle
• fix: 🐛 test/release status badge url typo(#1224)@yambottle
• Dev 861 add readme badge(#1221)@yambottle

Full Changelog: v0.14.3...v0.14.4

Release 0.14.3

• Added - dj.Top restriction - PR #1024) PR #1084
• Fixed - Added encapsulating double quotes to comply with DOT language - PR #1177
• Added - Datajoint python CLI (#940) - PR #1095
• Added - Ability to set hidden attributes on a table - PR #1091
• Added - Ability to specify a list of keys to populate - PR #989
• Fixed - fixed topological sort #1057 - PR #1184
• Fixed - .parts() not always returning parts #1103 - PR #1184
• Changed - replace setup.py with pyproject.toml - PR #1183
• Changed - disable add_hidden_timestamp configuration option by default - PR #1188