feat: add append_atlan_tags to Batch and save_merging_cm#866
Open
mothership-ai[bot] wants to merge 1 commit intomainfrom
Open
feat: add append_atlan_tags to Batch and save_merging_cm#866mothership-ai[bot] wants to merge 1 commit intomainfrom
mothership-ai[bot] wants to merge 1 commit intomainfrom
Conversation
…g_cm, and update_merging_cm The Batch and AsyncBatch classes only supported replace_atlan_tags (True/False), meaning Atlan tags were either fully overwritten or completely ignored during bulk operations. This made it impossible to use the Batch class for incremental tag updates (add/update/remove specific tags without affecting others). This change adds an append_atlan_tags parameter to: - Batch.__init__() and its flush() method - AsyncBatch.__init__() and its flush() method - AssetClient.save_merging_cm() and update_merging_cm() - AsyncAssetClient.save_merging_cm() and update_merging_cm() When append_atlan_tags=True, the bulk API is called with appendTags=true, enabling incremental tag operations via add_or_update_classifications and remove_classifications fields on assets. This is critical for high-volume tag sync workflows (100K+ assets) where customers need to add/remove specific tags without fetching existing state first. Resolves: REQ-717 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Session owner: @Pawan-atlan
Summary
append_atlan_tagsparameter toBatch,AsyncBatch,save_merging_cm, andupdate_merging_cm— enabling incremental Atlan tag operations (add/update/remove) via the bulk API without replacing all existing tagsBatchclass only supportedreplace_atlan_tags(overwrite all or ignore all), making it unusable for high-volume tag sync workflows where specific tags need to be added/removed on 100K+ assetssave()method already supportedappend_atlan_tags, but theBatchclass andsave_merging_cm/update_merging_cmmethods did not pass it throughContext (REQ-717)
A customer running a PyAtlan-based tag sync workflow across ~100K Snowflake assets is experiencing:
add_atlan_tags()/remove_atlan_tag()call does an IndexSearch + individual save (~0.5-1s per asset)_modify_tags()always fetches by qualifiedName even when the customer has the GUIDRecommended approach for the customer (with this fix)
This reduces 100K+ individual API calls (2 per asset) to ~5K bulk API calls (batches of 20), eliminating the IndexSearch overhead entirely.
Test plan
Batch,AsyncBatch,save_merging_cm,update_merging_cmall acceptappend_atlan_tagsparameterflush()passesappend_atlan_tagsthrough tosave()andsave_merging_cm()append_atlan_tags=True🤖 Generated with Claude Code