Feat/persistency import export#166
Open
viktorbeck98 wants to merge 7 commits into
Open
Conversation
…metadata Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…, fix load() thread safety - Add module-level _load helper that mirrors _save - Restore event_data_class from metadata on load - PersistencySaver.load() now acquires self._lock before mutating persistency - Use fs.pipe() for atomic file writes in _save to eliminate read/write race - Add test_load_restores_event_data_class and TestPersistencySaverThreadSafety Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ctor Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
save(ep) returns a zip archive as bytes; load(ep, bytes) restores from it. export_state() and import_state() on CoreDetector gain the same overload, so callers can transfer state over a network API without touching the filesystem. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
There are merging conflicts and I added some comments in the code. |
ipmach
reviewed
Jun 5, 2026
Contributor
There was a problem hiding this comment.
- This logic is not part of the detector but the persistency itself. The detector should only have to call. persistency.save/load.
- I will just return None and not raise an exception if there is not persistency configure. Because the detector can still work without persistency and we should not crash the program if we dont need to.
- We do not use multithreats detectors so the thread-safe is just added complexity.
Contributor
There was a problem hiding this comment.
Soon we will have more parsers and alert aggregation methods. export_state/import_state methods should be for every class as we will need to save and import from those classes as well. I think it is better if they are in the core component
ipmach
requested changes
Jun 5, 2026
…rary into feat/persistency-import-export
Contributor
I fixed those merge conflicts as they happened because of the renaming of tests into detectmatelibrary_tests. No changes required. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: export and import
EventPersistencystate on demandProblem
There was no way to transfer a detector's trained state between environments (e.g. training → production) or take a manual snapshot outside the auto-save timer cycle, without constructing a full
PersistencySaverwith its threading machinery.What changed
Layer 1 — standalone functions (
detectmatelibrary.utils.persistency)saveandloadare extracted fromPersistencySaver's internals and exposed as module-level functions.PersistencySavernow delegates to them, so the serialisation format lives in exactly one place.Layer 2 — detector-level wrappers (
CoreDetector)export_stateandimport_stateare thin wrappers around the above.import_stateis thread-safe: it acquires the saver lock before loading when aPersistencySaveris running.Bytes overload
Both layers accept an optional
path. When omitted,save/export_statereturn the state as an in-memory zip archive (bytes);load/import_stateaccept either a path string or those bytes. This lets an API send or receive state without touching the filesystem.Metadata
event_data_classis now written to the metadata JSON and restored on load, so a detector loaded into a freshEventPersistencypicks the correct backend for new events automatically. Existing snapshots without the field continue to load fine.Usage
What did not change
Tests