Feat/meta/sn by carpawell · Pull Request #3991 · nspcc-dev/neofs-node

carpawell · 2026-05-14T16:33:54Z

No description provided.

codecov · 2026-05-26T19:36:31Z

Codecov Report

❌ Patch coverage is 15.88629% with 503 lines in your changes missing coverage. Please review.
✅ Project coverage is 26.97%. Comparing base (c3ae25d) to head (688521e).

Files with missing lines	Patch %	Lines
cmd/neofs-node/meta.go	6.40%	117 Missing ⚠️
pkg/services/meta/meta.go	25.43%	81 Missing and 4 partials ⚠️
pkg/services/sidechain/sidechain.go	0.00%	80 Missing ⚠️
pkg/services/meta/storage.go	3.63%	53 Missing ⚠️
pkg/services/object/put/distributed.go	14.03%	48 Missing and 1 partial ⚠️
pkg/services/meta/blocks.go	0.00%	36 Missing ⚠️
pkg/core/object/replicate.go	13.15%	33 Missing ⚠️
pkg/services/object/put/streamer.go	31.81%	14 Missing and 1 partial ⚠️
pkg/services/meta/notifications.go	0.00%	14 Missing ⚠️
pkg/network/address.go	0.00%	8 Missing ⚠️
... and 5 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3991      +/-   ##
==========================================
- Coverage   27.70%   26.97%   -0.73%     
==========================================
  Files         682      684       +2     
  Lines       47029    46513     -516     
==========================================
- Hits        13029    12547     -482     
- Misses      32756    32780      +24     
+ Partials     1244     1186      -58

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cthulhu-rider

not finished

cthulhu-rider

finished. Lets fix everything and merge

@AnnaShaleva may u pls check chain configuration?

AnnaShaleva · 2026-06-02T15:30:16Z

 	// notification work; it is required to always read notifications without
 	// any blocking or making additional RPC.
-	notificationBuffSize = 100
+	notificationBuffSize = 10000


Is it approved by @roman-khimov?

this is an address + pointer buff channel, what cons may we have? ~703KB of memory

i can change it to any value. do you have a suggestion? i just had an experience with deadlock with neo-go subscription, so this is why this big value was chosen

That's why I'm asking. Increasing buffer size is not a solution to the deadlock problem. It will make the problem more hidden, that's it.

If there's a deadlock, it should be fixed in some other (explicit) way.

Increasing buffer size is not a solution to the deadlock problem

we are balancing b/w notification handling speed and storage speed. i guess it is another thing for exploring in load testing (yet not answered)

AnnaShaleva

Tests are failing. Other than that looks legit as a prototype.

carpawell · 2026-06-04T16:29:44Z

Tests are failing. Other than that looks legit as a prototype.

They should be adopted, yes. And there are other problems. Config check and changelog are not accepted to the experimental things.

cthulhu-rider

seems good to me overall

roman-khimov · 2026-06-11T14:43:41Z

+
+func (m *Meta) notificationHandler(ctx context.Context, buff <-chan *state.ContainedNotificationEvent) {
+	for {
+		if len(buff) >= notificationBuffSize-1 {


roman-khimov · 2026-06-11T15:11:52Z

+	on.m.Unlock()
+
+	if ok {
+		on.metaSvc.taskQueue <- storageTask{addr: addr, o: &sub.objHeader}


Does it make any difference? Local HEAD should be pretty fast.

why HEAD smth, when a node already has header in memory?

it also may be non-local, why not?

Code is more complicated for sure. Whether it's worth this complication or not is an open question.

8 nodes, REP 3: we read (block writing at the same time) our own metabase, OR in 62% go to a different node for a header we had in memory just about ~25ms ago. why?

I don't see any difference in chances, in 3 out of 8 cases it's local, in 5 out of 8 it's networked. Irrespective of this structure. Let's keep it as is for now, this can be checked in future as well.

5 out of 8 it's networked

and it means this node collected header, has it already locally in memory, collected every signature, sent tx to the chain, expects the results in ~25ms, and... decided to drop this header to request it soon again from another node. to me, this is "read header from network" vs "read header from memory". i think i do not understand smth

Are you talking about forwarding in-container node? That'd be 1 out of 5 not storing the object.

roman-khimov · 2026-06-11T15:14:12Z

+	wg.Go(func() { m.storager(ctx, m.taskQueue) })
+
+	m.chain.SubscribeForBlocks(m.bCh)
+	m.chain.SubscribeForNotifications(m.evsCh)


Seems like this can be done via JSON-RPC as well. Can we have this mode (non-local chain) for nodes with IR nearby?

Seems like this can be done via JSON-RPC as well

why is it better in any way?

Can we have this mode (non-local chain) for nodes with IR nearby?

what mode would it be? SN that does not have a local chain? what chain does it put it in then?

this is an interface, so not problem to have any chain source there, but i do not fully inderstand your intention

Pretty easy and already discussed a number of times, IR+SN combo has two chains onboard while it could only have one. We can also experiment with non-local chains, but let's move this to another issue, #4022

10_000_000 meant nothing, GAS factor is 1_0000_0000. Also, new value means something at least. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

It is available starting from v0.119.0. This should speed broadcast up, which is critical for metadata chain. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Just drop the old one, use new instead, no functional changed in the new one. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell mentioned this pull request May 15, 2026

ir: deploy IR nodes with new metadata chain nspcc-dev/neofs-testcases#1367

Merged

carpawell force-pushed the feat/meta/sn branch from 10a909b to 12a8fe6 Compare May 21, 2026 21:37

carpawell marked this pull request as ready for review May 21, 2026 21:37

carpawell requested review from End-rey, cthulhu-rider and roman-khimov as code owners May 21, 2026 21:37

carpawell force-pushed the feat/meta/sn branch 2 times, most recently from d26ab18 to f9b5259 Compare May 26, 2026 19:32

cthulhu-rider reviewed May 27, 2026

View reviewed changes

Comment thread pkg/services/meta_new/meta.go Outdated

Comment thread pkg/services/meta_new/meta.go Outdated

Comment thread pkg/services/meta_new/blocks.go Outdated

Comment thread pkg/services/meta_new/blocks.go Outdated

cthulhu-rider reviewed Jun 2, 2026

View reviewed changes

AnnaShaleva reviewed Jun 2, 2026

View reviewed changes

carpawell mentioned this pull request Jun 3, 2026

Find optimal metadata chain parameters #4009

Open

carpawell force-pushed the feat/meta/sn branch from f9b5259 to bf49760 Compare June 3, 2026 19:49

carpawell requested review from AnnaShaleva and cthulhu-rider June 3, 2026 19:50

carpawell force-pushed the feat/meta/sn branch from bf49760 to d0d5071 Compare June 3, 2026 19:51

AnnaShaleva reviewed Jun 4, 2026

View reviewed changes

Comment thread cmd/neofs-node/meta.go Outdated

Comment thread pkg/innerring/innerring.go Outdated

Comment thread pkg/util/network.go Outdated

carpawell force-pushed the feat/meta/sn branch from d0d5071 to f84dcfa Compare June 4, 2026 16:28

carpawell force-pushed the feat/meta/sn branch from f84dcfa to 18d4a4f Compare June 4, 2026 16:30

carpawell requested a review from AnnaShaleva June 4, 2026 16:30

carpawell mentioned this pull request Jun 4, 2026

Adapt tests to changed metaconfig nspcc-dev/neofs-testcases#1373

Closed

carpawell force-pushed the feat/meta/sn branch from 18d4a4f to c4414b6 Compare June 8, 2026 07:35

cthulhu-rider approved these changes Jun 8, 2026

View reviewed changes

carpawell force-pushed the feat/meta/sn branch from c4414b6 to 19291f2 Compare June 11, 2026 10:39

roman-khimov reviewed Jun 11, 2026

View reviewed changes

carpawell force-pushed the feat/meta/sn branch 2 times, most recently from 7618a13 to e21f8b1 Compare June 16, 2026 19:50

roman-khimov reviewed Jun 17, 2026

View reviewed changes

Comment thread pkg/services/meta/storage.go Outdated

meta_test: change test network fee

33bf801

10_000_000 meant nothing, GAS factor is 1_0000_0000. Also, new value means something at least. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell force-pushed the feat/meta/sn branch from e21f8b1 to d55c448 Compare June 17, 2026 12:57

carpawell and others added 5 commits June 17, 2026 19:24

sn: support new meta flow

68144ee

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

meta: add service metrics

7dce458

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

meta: optimize signature verifying

8d98e0b

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

[ir,sn]/meta: enable neo-go's DirectRelay application configuration

a95804a

It is available starting from v0.119.0. This should speed broadcast up, which is critical for metadata chain. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

[ir,sn]/meta: add test not to miss new NEO hard forks

d161a8a

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell force-pushed the feat/meta/sn branch from d55c448 to 5fa8ebf Compare June 17, 2026 16:34

sn: eliminate old meta service

688521e

Just drop the old one, use new instead, no functional changed in the new one. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell force-pushed the feat/meta/sn branch from 5fa8ebf to 688521e Compare June 17, 2026 16:36

roman-khimov mentioned this pull request Jun 17, 2026

Manage meta hardforks automatically #4025

Open

roman-khimov approved these changes Jun 17, 2026

View reviewed changes

roman-khimov merged commit 1f65db8 into master Jun 17, 2026

roman-khimov deleted the feat/meta/sn branch June 17, 2026 19:50

carpawell mentioned this pull request Jun 21, 2026

Meta search works incorrect #4032

Closed

Conversation

carpawell commented May 14, 2026

Uh oh!

codecov Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cthulhu-rider left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cthulhu-rider left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnnaShaleva Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AnnaShaleva left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carpawell commented Jun 4, 2026

Uh oh!

cthulhu-rider left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented May 26, 2026 •

edited

Loading

AnnaShaleva Jun 4, 2026 •

edited

Loading