asciilifeform: shinohai: iirc at least 2nd one this month
kiwi_44: Uploaded file: https://uploads.kiwiirc.com/files/d62a5572a9d840bb896703786bfdba46/ICMEC-FCACPCryptocurrencyPaperFINAL5-17.pdf
cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020499 << i recently showed up on #asciilifeform thinking i had discovered something re trb sync, and have now been studying the subject some more. i've got some further questions, and thought maybe this channel was more on topic.
snsabot: (asciilifeform) 2020-08-26 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020442 and http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020444 << yeah, no objection. i ended up making hasty conclusions.
shinohai: cgra: Have ya managed to build own trb yet?
cgra: shinohai: yeah, used the mod6's package. been modifying that for myself to experiment. my other env is a toy box though, intel, ubuntu etc
cgra: it appears per the trb code, that actual "block" or "tx" messages never occur without node specifically requesting them, only "inv" adverts pass ~constantly around.
cgra: based on this, i've got currently two distinct avenues on my table, both under the same name, "initial sync mode": 1) stop all the tx activity until done with the catch-up, and 2) sync from one peer at a time. after one set of "inv", cycle to the next peer
shinohai: Nice. There is plenty to do under hood, nice of you to experiment.
cgra: these two things appear to reduce the amount of bastard blocks significantly (and improve sync equally), but i could be mistaken, i've got a bunch of if's i'd like to hear comments on:
cgra: a) i reverted the malleus_mikehearnificarum.vpatch for the unknown commands part
cgra: b) i'm behind nat, so no "inbound" peers
cgra: some numbers follow, but could be garbage due to some of my mistake. suitable anyway for estimating if even remotely realistic:
cgra: trb with tx off, sending blocks off, ~25h runtime, starting height around 288k: 563 accepted blocks, 1764652 bastards
cgra: trb with above "2)" enabled, tx off, sending blocks off, ~18h runtime, starting height around: 228k: 60060 accepted blocks, 1088 bastards
cgra: first one also produced 32GB of debug.log, and the second one only 64MB
cgra: more points to consider re above data:
cgra: c) done on a ~5ghz cpu (intel)
cgra: d) ~50-100mti link (variable LTE, didn't measure exactly)
cgra: e) -debug flag was enabled, debug.log truncation code commented out
cgra: "mti" above, meant "Mbit"
cgra: above, "after one set of "inv"", i meant: after one set of "getblocks" query - "inv" response - "getdata" query - x "block" responses -dialog
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001063 << it's actually mostly ~same folx in both chans. ( ftr asciilifeform would actually prefer if all trbism happened here . but tbh cares less nao that log www has working all-chans search. )
snsabot: Logged on 2020-08-30 10:37:42 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020499 << i recently showed up on #asciilifeform thinking i had discovered something re trb sync, and have now been studying the subject some more. i've got some further questions, and thought maybe this channel was more on topic.
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001067 << this almost correct -- 'latest' block is sent unsolicited.
snsabot: Logged on 2020-08-30 10:44:13 cgra: it appears per the trb code, that actual "block" or "tx" messages never occur without node specifically requesting them, only "inv" adverts pass ~constantly around.
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001068 << (1) would theoretically save some bw. (2) de-facto happens already, as trb is effectively single-threaded.
snsabot: Logged on 2020-08-30 10:44:19 cgra: based on this, i've got currently two distinct avenues on my table, both under the same name, "initial sync mode": 1) stop all the tx activity until done with the catch-up, and 2) sync from one peer at a time. after one set of "inv", cycle to the next peer
asciilifeform: note that w/out something like 'cement', a node has no way of mechanically discovering that it is 'done with catchup'.
snsabot: (asciilifeform) 2020-08-25 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020435 << thus far the only potential pill i've come up with is 'cement'. (i.e. noad eats a wot-signed list of block hashes, to then use during sync.)
cgra: asciilifeform: there's that "mapAlreadyAskedFor" that guards against repeatedly requesting items. it can throw things off re "the sync from one node at a time"
cgra: two min cooldown for each item
asciilifeform: cgra: tru
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001071 << that patch is there to reduce time wasted talking to pseudo-nodes that dun actually store blocks, and other wholly incompat. pieces of shit. in asciilifeform's (and other folx's) tests, proved quite effective. which is why part of tbf flagship vtree.
snsabot: Logged on 2020-08-30 10:48:46 cgra: a) i reverted the malleus_mikehearnificarum.vpatch for the unknown commands part
cgra: asciilifeform: re the "done with catchup", i was thinking of perhaps a cmdline argument, at operator's discretion
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001075 << the problem there is that a node configured that way is of 0 use to the network. if yer gonna do that, makes considerably moar sense to simply copy db from an existing node, if all you care about is 'stand up node asap'
snsabot: Logged on 2020-08-30 10:57:14 cgra: trb with above "2)" enabled, tx off, sending blocks off, ~18h runtime, starting height around: 228k: 60060 accepted blocks, 1088 bastards
asciilifeform bbl
cgra: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001097 << because can avoid touching trb codebase? do you think similarly of the "cement" approach?
snsabot: Logged on 2020-08-30 12:11:48 asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001075 << the problem there is that a node configured that way is of 0 use to the network. if yer gonna do that, makes considerably moar sense to simply copy db from an existing node, if all you care about is 'stand up node asap'
cgra: asciilifeform: or just think that being half-functional for the catch-up period is too much of waste?
cgra is out, but will follow logs.
cgra: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001085 << could you point me to the code where this happens? i found only "inv"'s
snsabot: Logged on 2020-08-30 12:05:07 asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001067 << this almost correct -- 'latest' block is sent unsolicited.
cgra: though, the code says any unsolicited items are to be processed nevertheless.
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001100 << imho 'cement' would be useful if implemented compactly; then could feed a node <100MB signed list of hashes and it'll download correct blox through N from the net. but indeed 'less mutilation is best' imho when comes to trb; and if thinking about it, ANY trb node that aint synced to ~current height, will in fact refuse to relay most tx (and will be mostly busy
snsabot: Logged on 2020-08-30 12:20:31 cgra: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001097 << because can avoid touching trb codebase? do you think similarly of the "cement" approach?
asciilifeform: verifying)
asciilifeform actually doesn't have any objection to 'sync mode' where node doesn't even attempt to relay tx, etc -- aside from the fact that one would have to manually switch the thing to standard mode by hand when decide that 'synced'
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001104 << actually cgra is technically correct re trb -- blocks get req'd only via inv (unless transmitted by my 'feeder.py' or similar kludge)
snsabot: Logged on 2020-08-30 12:54:47 cgra: http://logs.nosuchlabs.com/log/therealbitcoin/2020-08-30#1001085 << could you point me to the code where this happens? i found only "inv"'s
asciilifeform: afaik there's no possible mechanism for knowing 'not to ask for' an inv-advertised block that aint the 'next wanted' tho, unless you have sumthing like the 'cement' cheat .
asciilifeform: ( elementarily -- a traditional trb node has no idea what is the hash of 'next' block until receives it )
snsabot: Logged on 2020-08-30 13:01:23 cgra: though, the code says any unsolicited items are to be processed nevertheless.
asciilifeform bbl.