[Infoblox][Data Connector] - Migrate Checkpoint From File Share to Azure Table#14689
Open
jbhaliya-crest wants to merge 2 commits into
Open
[Infoblox][Data Connector] - Migrate Checkpoint From File Share to Azure Table#14689jbhaliya-crest wants to merge 2 commits into
jbhaliya-crest wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Migrates Infoblox data connector checkpointing from Azure File Share to Azure Table Storage and adds resilience fixes for malformed JSON parsing / chunk stitching.
Changes:
- Introduces an Azure Table-based checkpoint manager and updates multiple functions to read/write checkpoints via tables (with migration from legacy file-share checkpoints).
- Adds JSON parsing recovery and improves raw JSON chunk joining by stripping leading commas.
- Updates ARM deployment/config to include a checkpoint table name and adds the
azure-data-tablesdependency.
Reviewed changes
Copilot reviewed 10 out of 14 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| Solutions/Infoblox/ReleaseNotes.md | Updates release notes entry to mention the checkpoint migration and refreshes the modified date. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/requirements.txt | Adds Azure Tables SDK dependency for the new checkpoint storage backend. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/azuredeploy_Connector_InfoBloxCloud_AzureFunction.json | Adds Checkpoint_Table_Name app setting for the connector functions. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/SharedCode/utils.py | Adds Azure Table checkpoint helpers; adds malformed JSON recovery; caches customer_id in checkpoint table and adds headers. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/SharedCode/table_checkpoint_manager.py | Adds Table Storage wrapper and TableCheckpointManager adapter. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/SharedCode/consts.py | Adds constants and env var for checkpoint table name + customer-id checkpoint key. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/InfobloxParseRawIndicators/parse_json_files.py | Switches parse-progress checkpoints to tables (with migration) and hardens JSON stitching. |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/InfobloxHistoricalToAzureStorage/infoblox_to_azure_storage.py | Switches time-based checkpoints + retry counters to tables (with migration). |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/InfobloxCurrentToAzureStorage/infoblox_to_azure_storage.py | Switches time-based checkpoints + retry counters to tables (with migration). |
| Solutions/Infoblox/Data Connectors/InfobloxCloudDataConnector/AzureStorageToIndicators/create_indicator.py | Migrates indicator_count checkpoint to tables (with migration) and uses table-based helpers. |
Comments suppressed due to low confidence (2)
Solutions/Infoblox/ReleaseNotes.md:1
- This row’s table formatting is inconsistent with the other entries (it introduces
3 ||where prior rows start with||). This likely breaks the Markdown table rendering; align the row prefix/column count with the other version rows.
| **Version** | **Date Modified (DD-MM-YYYY)** | **Change History** |
Solutions/Infoblox/ReleaseNotes.md:1
- Correct spelling of 'Migrataed' to 'Migrated'.
| **Version** | **Date Modified (DD-MM-YYYY)** | **Change History** |
Author
|
@microsoft-github-policy-service agree company="Crest Data" |
Contributor
|
Please provide the testing screenshots validating the changes included in this PR. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change(s):
Reason for Change(s):
Version Updated:
Testing Completed: