Skip to content

fix(watcher)!: debounce file events, bound concurrency, handle atomic saves. Closes #297#305

Open
rootp1 wants to merge 6 commits intomicrocks:masterfrom
rootp1:fix/issue-297
Open

fix(watcher)!: debounce file events, bound concurrency, handle atomic saves. Closes #297#305
rootp1 wants to merge 6 commits intomicrocks:masterfrom
rootp1:fix/issue-297

Conversation

@rootp1
Copy link
Copy Markdown

@rootp1 rootp1 commented May 6, 2026

Description

  • Added 300ms per-file debounce to collapse rapid fsnotify event bursts (IDE saves, sed -i, cp) into a single TriggerImport call
  • Replaced unbounded go TriggerImport() with a serial worker reading from a bounded importQueue channel (capacity 1), providing natural backpressure and preventing goroutine leaks
  • Handled fsnotify.Create, fsnotify.Rename, and fsnotify.Remove events in addition to fsnotify.Write, so atomic saves (write temp + rename over target) are no longer silently missed
  • Re-added the file watcher on Create/Rename/Remove events to track new inodes after atomic file replacements
  • AddED context.Context propagation through NewWatchManger, Run(), TriggerImport, and UploadArtifact for clean shutdown on SIGINT/SIGTERM
  • Fixed discarded fmt.Errorf return value bug in executor.go (line 16 of original)
  • Added Stop() method to WatchManager for programmatic cancellation
  • Added triggerFunc field to WatchManager for testability (mockable in tests)
  • Added comprehensive test suite: debounce collapse, multi-file queueing, context cancellation, event filtering, atomic save handling, timer drain, Stop() semantics

Reproduction results

Before fix - Thundering herd:
5 rapid file saves now 8-10 concurrent goroutines spawned, each firing an unbounded HTTP upload to the Microcks server.

After fix - Debounced:
5 rapid file saves now 1 import triggered (300ms debounce collapses all events into one).

Before fix - Atomic save missed:
write temp + rename over target emits CHMOD + REMOVE events (zero Write events). Old code only handled fsnotify.Write, so atomic saves were completely invisible.

After fix - Atomic save handled:
Create/Rename/Remove events are processed and the file watcher is re-registered on the new inode.

Related issue(s)

Fixes thundering herd, race conditions on concurrent uploads, goroutine leaks, and lost file tracking on atomic saves as described in the watcher bug report.
Fixes #297

BREAKING CHANGE

MicrocksClient.UploadArtifact and cmd.MicrocksClient interface signatures now require a context.Context as the first parameter. Any external implementations of these interfaces must be updated accordingly.

rootp1 added 6 commits May 6, 2026 15:36
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
Signed-off-by: rootp1 <arnav.iitr@gmail.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

👋 @rootp1

Welcome to the Microcks community! 💖

Thanks and congrats 🎉 for opening your first pull request here! Be sure to follow the pull request template or please update it accordingly.

Hope you have a great time there!

@rootp1
Copy link
Copy Markdown
Author

rootp1 commented May 6, 2026

@Harsh4902, @yada, @lbroudoux
Here is my work for the issue #297
I have tried to align with most of the contribution guidelines, Please review this PR whenever you get a chance to. Thankyou

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Watcher File Change Events Trigger Unbounded Concurrent Reimports, without any Debounce or Limit

1 participant