fix(watcher)!: debounce file events, bound concurrency, handle atomic saves. Closes #297#305
Open
rootp1 wants to merge 6 commits intomicrocks:masterfrom
Open
fix(watcher)!: debounce file events, bound concurrency, handle atomic saves. Closes #297#305rootp1 wants to merge 6 commits intomicrocks:masterfrom
rootp1 wants to merge 6 commits intomicrocks:masterfrom
Conversation
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
|
👋 @rootp1 Welcome to the Microcks community! 💖 Thanks and congrats 🎉 for opening your first pull request here! Be sure to follow the pull request template or please update it accordingly. Hope you have a great time there! |
Author
|
@Harsh4902, @yada, @lbroudoux |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
sed -i,cp) into a singleTriggerImportcallgo TriggerImport()with a serial worker reading from a boundedimportQueuechannel (capacity 1), providing natural backpressure and preventing goroutine leaksfsnotify.Create,fsnotify.Rename, andfsnotify.Removeevents in addition tofsnotify.Write, so atomic saves (write temp + rename over target) are no longer silently missedCreate/Rename/Removeevents to track new inodes after atomic file replacementscontext.Contextpropagation throughNewWatchManger,Run(),TriggerImport, andUploadArtifactfor clean shutdown on SIGINT/SIGTERMfmt.Errorfreturn value bug inexecutor.go(line 16 of original)Stop()method toWatchManagerfor programmatic cancellationtriggerFuncfield toWatchManagerfor testability (mockable in tests)Stop()semanticsReproduction results
Before fix - Thundering herd:
5 rapid file saves now 8-10 concurrent goroutines spawned, each firing an unbounded HTTP upload to the Microcks server.
After fix - Debounced:
5 rapid file saves now 1 import triggered (300ms debounce collapses all events into one).
Before fix - Atomic save missed:
write temp + rename over targetemitsCHMOD+REMOVEevents (zeroWriteevents). Old code only handledfsnotify.Write, so atomic saves were completely invisible.After fix - Atomic save handled:
Create/Rename/Removeevents are processed and the file watcher is re-registered on the new inode.Related issue(s)
Fixes thundering herd, race conditions on concurrent uploads, goroutine leaks, and lost file tracking on atomic saves as described in the watcher bug report.
Fixes #297
BREAKING CHANGE
MicrocksClient.UploadArtifactandcmd.MicrocksClientinterface signatures now require acontext.Contextas the first parameter. Any external implementations of these interfaces must be updated accordingly.