Skip to content

fix: Add config option to skip filepath checksum on insert#1387

Merged
ttngu207 merged 1 commit intomaint/0.14from
fix/filepath-checksum-bypass
Feb 5, 2026
Merged

fix: Add config option to skip filepath checksum on insert#1387
ttngu207 merged 1 commit intomaint/0.14from
fix/filepath-checksum-bypass

Conversation

@dimitri-yatsenko
Copy link
Member

@dimitri-yatsenko dimitri-yatsenko commented Feb 5, 2026

Summary

Adds filepath_checksum_size_limit_insert config option to skip checksum computation on insert for files larger than the specified limit. This prevents transaction timeouts when inserting large files with filepath attributes in three-part make() methods.

Problem

In three-part make() methods, the transaction can expire while checksums are computed for large files with filepath fields. This causes the entire make() to fail.

Solution

Config Option Purpose
filepath_checksum_size_limit Skip checksum verification on fetch (existing)
filepath_checksum_size_limit_insert Skip checksum computation on insert (new)

When checksum is skipped on insert:

  • A warning is logged
  • contents_hash is stored as NULL
  • Existing file verification is bypassed

Usage

import datajoint as dj

# Skip checksum on insert for files > 1GB
dj.config['filepath_checksum_size_limit_insert'] = 1024 * 1024 * 1024

Or via environment variable:

export DJ_FILEPATH_CHECKSUM_SIZE_LIMIT_INSERT=1073741824

Fixes #1386

🤖 Generated with Claude Code

@dimitri-yatsenko dimitri-yatsenko force-pushed the fix/filepath-checksum-bypass branch from 446277e to 83ade69 Compare February 5, 2026 19:17
@dimitri-yatsenko dimitri-yatsenko changed the title fix: Skip filepath checksum on insert when file exceeds size limit fix: Add separate config options for filepath checksum on insert vs fetch Feb 5, 2026
Adds `filepath_checksum_size_limit_insert` config option to skip checksum
computation on insert for files larger than the specified limit. This
prevents transaction timeouts when inserting large files with filepath
attributes in three-part `make()` methods.

Config options:
- `filepath_checksum_size_limit`: skip checksum verification on fetch (existing)
- `filepath_checksum_size_limit_insert`: skip checksum computation on insert (new)

When checksum is skipped on insert:
- A warning is logged
- contents_hash is stored as NULL
- Existing file verification is bypassed

Fixes #1386

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dimitri-yatsenko dimitri-yatsenko force-pushed the fix/filepath-checksum-bypass branch from 83ade69 to 6d02652 Compare February 5, 2026 19:20
@dimitri-yatsenko dimitri-yatsenko changed the title fix: Add separate config options for filepath checksum on insert vs fetch fix: Add config option to skip filepath checksum on insert Feb 5, 2026
@ttngu207 ttngu207 merged commit 60a8f05 into maint/0.14 Feb 5, 2026
3 checks passed
@ttngu207 ttngu207 deleted the fix/filepath-checksum-bypass branch February 5, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants