cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020442 and http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020444 << yeah, no objection. i ended up making hasty conclusions.
snsabot: Logged on 2020-08-25 10:57:29 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020426 << except it ~does~ work, albeit slowly.
snsabot: Logged on 2020-08-25 10:58:54 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020428 << first of all, PushGetBlocks is still there. and , after 'aggression' patch, also here .
cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020446 << i meant the block ~advert~ (an "inv" message), it was a convoluted, pre-trb way of resuming the sync chain where it left off last cycle, and what the 'hashContinue' variable was part of. just clarifying myself, i still must finish my homework.
snsabot: Logged on 2020-08-25 10:59:45 asciilifeform: 2nd of all, noshit it doesn't give a damn re 'current' block, because there is no way to verify it ! it is only possible to verify a block for which there exists an antecedent.
cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020449 << to educate a noob: what's wrong with the headers-first sync?
snsabot: Logged on 2020-08-25 11:02:51 shinohai: https://github.com/bitcoin/bitcoin/pull/4468 <<< lest you wind up with?
cgra: or maybe having a bounded block buffer?
shinohai: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020504 <<< I posted wrong link, was meaning to post the PR that added "pruning"
snsabot: Logged on 2020-08-26 03:51:18 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020449 << to educate a noob: what's wrong with the headers-first sync?
shinohai: *mimisbrunnr (~mimisbrun@51.15.42.105) has joined #asciilifeform <<< neato I hope bv brings this back to life.
shinohai: $vwap
btcinfobot: The 24-Hour VWAP for BTC is $ 11363.63 USD
cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020507 << ah, ok. though still hoping to hear comments re the q, from whoever has any
snsabot: Logged on 2020-08-26 07:29:51 shinohai: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020504 <<< I posted wrong link, was meaning to post the PR that added "pruning"
asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020506 << asciilifeform answered this yesterday, plox to reread and ask q's if didn't grasp
snsabot: Logged on 2020-08-26 03:51:28 cgra: or maybe having a bounded block buffer?
snsabot: Logged on 2020-08-25 11:01:25 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020429 << i abolished the buffers quite deliberately, because they are a trivial attack vector. if yer node accepts unverifiable blocks, i can stuff its buffer full of liquishit, and effect on your end will be quite the same as having no buffer at all, but simply eats moar ram. (in early trb -- would in fact exhaust machine ram, because no mechanism to limit the buffers)
asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020504 << plenty of things. it's a tall pile of moving parts that never has been and probably never could be implemented correctly (i.e. while preserving 100% the semantics of the original client as currently represented by trb.)
snsabot: Logged on 2020-08-26 03:51:18 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020449 << to educate a noob: what's wrong with the headers-first sync?
snsabot: (trilema) 2015-06-06 asciilifeform: 'Because release 0.10.0 and later makes use of headers-first synchronization and parallel block download (see further), the block files and databases are not backwards-compatible with pre-0.10 versions of Bitcoin Core or other software. Blocks will be stored on disk out of order (in the order they are received, really), which makes it incompatible with some tools or other programs... The block index database will now hold head
asciilifeform: a large % of the questionable liquishit in prb is in fact there originally on acct of the introduction of headers1stism.
asciilifeform: !w poll
watchglass: Polling 12 nodes...
watchglass: 205.134.172.4:8333 : (172-4.core.ai.net) Alive: (0.022s) V=70001 (/therealbitcoin.org:0.7.0.1/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 205.134.172.27:8333 : Alive: (0.084s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440 (Operator: asciilifeform)
watchglass: 205.134.172.26:8333 : Alive: (0.111s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 205.134.172.6:8333 : (172-6.core.ai.net) Alive: (0.110s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 108.31.170.3:8333 : (pool-108-31-170-3.washdc.fios.verizon.net) Alive: (0.159s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440 (Operator: asciilifeform)
watchglass: 192.151.158.26:8333 : Alive: (0.084s) V=70001 (/therealbitcoin.org:0.7.0.1/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 208.94.240.42:8333 : Alive: (0.165s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 143.202.160.10:8333 : Alive: (0.281s) V=70001 (/therealbitcoin.org:0.7.0.1/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 213.109.238.156:8333 : Alive: (0.387s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 188.121.168.69:8333 : (rev-188-121-168-69.radiolan.sk) Alive: (0.380s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 103.36.92.112:8333 : (terebe.ns01.net) Alive: (0.523s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=645440
watchglass: 176.9.59.199:8333 : Busy? (No answer in 20 sec.) (Operator: jurov)
asciilifeform: $ticker btc usd
btcinfobot: Current BTC price in USD: $11503.03
cgra: asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020514 << can't i have a small enough buffer that doesn't bother me being full if under attack, but help when syncing from friendly nodes?
snsabot: Logged on 2020-08-26 10:48:05 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020506 << asciilifeform answered this yesterday, plox to reread and ask q's if didn't grasp
asciilifeform: cgra: how exactly would a small buffer help ?
asciilifeform: cgra: the bulk of 'bastard' blox received by a syncing trb node, consists simply of 'latest block' push-propagated around the net. which have nowhere to go if the node aint anywhere near 'top of chain' in its sync, with any realistically-sized buffer
asciilifeform: the hypothetical scenario where e.g. 10 block wide buffer could help, would be e.g. 'i have 645441, but someone just sent 10 unknown ???s and guess what o hey i just got 645442 and realized those were 645443 .. 634553'. and this ~never actually happens.
cgra: asciilifeform: sounds like i need here too, a better picture of what the cause of the current slowness exactly is. i might assume too much
cgra: it seemed to me the advertised series of blocks i get in response to "getblocks" is in correct order
cgra: and assumed you could perhaps interleave that across multiple sources
asciilifeform: cgra: there are 3 afaik main headaches in sync w/ current trb. (1) unsolicited 'latest' blocks occupy bandwidth, and are sent by ~all peers at ~all times. (2) peers do not reliably continue feeding next-needed-block in response to 'aggressive' PushGetBlocks (3) trb is effectively single-threaded; gets $block, then can pause for up to 3min during which cannot do anything, inc
asciilifeform: l. receive any other blocks, and connection to peer often breaks.
asciilifeform: (2) may be wholly caused by (3), or there may be other causes as well.
cgra: right... i'm still playing around with heights between 160 and 200k. highly unrepresentative
asciilifeform: after 400k or so, moar representative, i.e. mostly full blox
cgra: asciilifeform: thanks for sharing. i'll be off again, but hopefully return sometime later, wiser
asciilifeform: cgra: ok
asciilifeform: ftr imho the correct solution to (3) is to replace the idiot bdb w/ a fast db that'll give verification that eats seconds, rather than minutes.
snsabot: (therealbitcoin) 2020-05-15 asciilifeform: idea is, when you eat a block, you store the tx in such a way that they actually point to the ones being spent. in mmap'd array. the respective scripts are kept in another, so that they don't fuck your page alignments. (ea. tx points back to the orig. script. so can reconstitute orig. tx / block on demand )
asciilifeform: the solution to (2) is 'cement', and regular updates of the cement snapshot from trusted wot folx, i.e. for 99+% of the init noad bringup, the noad knows already the hash of $nextblock and will request it directly
snsabot: Logged on 2020-08-25 11:05:15 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-25#1020435 << thus far the only potential pill i've come up with is 'cement'. (i.e. noad eats a wot-signed list of block hashes, to then use during sync.)
asciilifeform: to (1) there is no solution, it is a standard aspect of the protocol, from bitcoin 0.1 to even current prb, it is simply how 'top of the chain' block propagates.
asciilifeform: btw 'store all incoming maybe-blocks somewhere' is a nonstarter, because any size buffer you could possibly have will be filled work forkolade liquishit within hours .
snsabot: (trilema) 2019-07-18 a111: Logged on 2018-10-30 18:16 asciilifeform: more interestingly, there was even 1 of 10/30/18 17:05:41 ERROR: ProcessBlock() : CheckBlock FAILED from peer 213.148.193.153
asciilifeform: there are at all times 'over 9000' machines pushing out entirely non-bitcoinistic blocks to nodes on standard bitcoin net.
asciilifeform: this makes ~any attempt at buffering, 100% useless, the buffer will at all times be filled with nominally format-conformant blocks for which a predecessor will ~never~ be seen (unless yer actually operating a fork node on that particular crackpot fork)
asciilifeform: * filled with forkolade liquishit
asciilifeform: btw must add that 'cement' ~alone~ won't make for the fastest possible sync: in '17 i determined that verification delay is substantial and consists largely of waiting for bdb to do its thing.
asciilifeform: the prb folx 'fixed' this simply by forgoing verification for large mass of blocks. which is patently insane. ('bbut!! fast sync!111 how couldja not want fast sync!11')
asciilifeform: imho if yer even ~considering~ 'maybe it's ok not to verify certain blocks', you've made fatal mistake in reasoning that will invariably lead you to recreating prb.
asciilifeform: ~all~ blocks received by machine must be verified, this is how you even know that yer noad's verification rules are compatible with the historic ones.
asciilifeform: likewise must note, if you want to be able to verify received 'cemented' block out of order and/or in parallel, would have to actually make the db thread-safe. which presently it aint (nuffin in trb is thread-safe, there are just about 'over 9000' locks in there, which make the thing effectively single-threaded)
asciilifeform: ( and in particular, if want 'receive out of order and buffer', they gotta also be stored somewhere ~other than the main db~ , and then shuttled into the latter after verification, cuz the classical trb block storage mechanism presumes in-order storage on disk)
asciilifeform: the reason why asciilifeform never attempted to implement any of the above, is that imho adding thousands of lines of moving parts, and potentially introducing fatally exploitable bugs, is not acceptable price to pay for 'new nodes syncs in a week instead of 2months'. esp. since you can bake new noad in ~hours~ if yer willing to simply copy the db from an existing noad that belongs to you.
asciilifeform: trb is small not by accident of fate, but because asciilifeform et al sweated for yrs to remove tumour mass from the original 0.5.3 and carefully testing . and refrained from ~adding~ mass whenever possible !
snsabot: Logged on 2020-06-16 13:51:58 asciilifeform: ( for comparison : trb src weighs ~700kB, consisting of ~22k loc . )
asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020537 << must highlight -- 'under attack', in trb world, is 24/7/365. at no point is a node ~not~ under attack. (unless, lol, cord is pulled outta the wall!)
snsabot: Logged on 2020-08-26 13:17:56 cgra: asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-26#1020514 << can't i have a small enough buffer that doesn't bother me being full if under attack, but help when syncing from friendly nodes?
asciilifeform: so any proposed mechanism that 'will work, if not under attack' is equivalent to 'will never work at all'.
asciilifeform: wb PeterL
PeterL: oh hi
asciilifeform: PeterL: what've you been up to ? ( aside from tiles... btw on asciilifeform's homeworld tile-cutting was a thing one did with a hand-held little diamond wheel thing + straightedge. worked ok )
snsabot: Logged on 2020-08-25 11:14:45 feedbot: http://peterl.xyz/2020/08/things-i-have-learned-about-setting-tiles/ << Blog of Peter Lambert -- Things I have learned about setting tiles
PeterL: re: getting a bunch of the same "latest blocks" mentioned above, does the client try to verify each block it gets, or only the one that would be next in the chain?
asciilifeform: PeterL: tries to verify. if block is from bizarre parallel universe, usually fails very quickly when tries to find antecedent
asciilifeform: if block is 'latest' and yer noad is 'behind' , ditto
PeterL: Yes, my tile cutter is basically a diamond wheel in a frame
PeterL: so would it save time to have a list of hashes of recent blocks checked so it would not try verifying the same new block from all the peers? Or would that not save anything?
asciilifeform: PeterL: this already happens : "ProcessBlock() : already have block %d %s from peer %s"
asciilifeform: the decision is made after seeing block's header. (i.e. quickly)
asciilifeform: trb only proceeds to do the 'slow' part of the verification (i.e. where gotta fetch each & erry single antecedent tx for the tx's in the block) if satisfied that header is for $nextblock, pow is correct, etc
PeterL: http://logs.nosuchlabs.com/log/asciilifeform/2020-08-20#1019898 << Didn't you end up selling all of the FG's you made?
snsabot: Logged on 2020-08-20 18:23:57 asciilifeform: for comparison, unit cost of FG , in materiel, was <50 $ . and still not 'bestseller'.
asciilifeform: PeterL: virtually all of'em went to l1 folx
asciilifeform: ( iirc the only non-l1 -- at the time -- buyers, were jfw (who at the time was lurking) , and someone in argentina who in fact failed to pick up parcel from the post, and it came back to usa )
asciilifeform: the last batch formally handled by asciilifeform , went to piz.
asciilifeform: ( the very last, arguably -- the graveyard auction, of spare parts , jfw & jurov winners. )
asciilifeform: PeterL: arguably it was dumb idea from asciilifeform's pov to bake fg. was heavily -ev, in that asciilifeform had to put considerable time into the development, and then testing, when already realized that would not see any coin from snsa 'like own ears w/out a mirror'(tm)(r)(stalin), on acct of the uncapped cost of mpex accts
asciilifeform: agreed to it because had already sworn the oath; and because 'can demonstrate that sane and usable trng can exist'
asciilifeform: for that matter, it aint as if any of asciilifeform's current public work is bringing in dough.
asciilifeform: 'interesting' is 9000x moar motivation for asciilifeform re a proj, and 'dough' distant secondary; esp. given as ~impossible, per my lights, to make any palpable dough via 'interesting'.
snsabot: Logged on 2020-08-20 19:16:49 asciilifeform: Aerthean: related reading. it is ~very~ difficult to profit by 'actually Do Right Thing' in engineering, vs. 'convincingly pretend to Do Right Thing' , 'i can't believe it's not butter!' atrocities.
asciilifeform: ... adding to earlier pt -- imho would be a Good Thing to bake a zoo of in-the-wild blox which ~nontrivially~ failed verification on trb, to use in tests of any future attempt to rewrite the client (as well as in evaluating any serious changes to trb as it presently exists)
snsabot: Logged on 2020-08-26 13:54:53 asciilifeform: ~all~ blocks received by machine must be verified, this is how you even know that yer noad's verification rules are compatible with the historic ones.
asciilifeform: ( blox which fail trivially -- e.g. wrong size, bad hash, etc -- are very easy to hand-craft from valid ones )
asciilifeform: ideally in such a zoo would turn up blox which represented attempted logic attacks, rather than simply crapola from poorly-written altclients or random bitflips etc
asciilifeform: imho best place for the tripwire would be here.
asciilifeform: ( or, for much broader selection of duds -- here . )
asciilifeform: while on subj of trbism, imho a patch to give separable logs -- 1 for mempool events, other -- for block events -- would be Right Thing.
asciilifeform: ( the mempool log is almost never interesting, in asciilifeform's experience , and massively wears ssd )
feedbot: http://mvdstandard.net/2020/08/us-unrest-17-year-old-kyle-rittenhouse-of-antioch-illinois-charged-with-murder-after-using-rifle-to-defend-self-from-moltov-throwing-vandals/ << The Montevideo Standard -- US Unrest: 17 Year Old Kyle Rittenhouse Of Antioch, Illinois Charged With Murder After Using Rifle To Defend Self From Moltov Throwing Vandals
shinohai: "Black reparations collection activists" <<< gold
BingoBoingo: Well, maybe it turns out solving the school shooting thing by ending schools mean... whatever shooter game the kids play now happens irl.
shinohai: BingoBoingo: The plot thickens: https://twitter.com/RealJamesWoods/status/1298693105617534976/photo/1
BingoBoingo: shinohai: Not quite thickened. Photo's floating around. Clearly self defense.
asciilifeform: BingoBoingo, shinohai : tbh i expect it'll go like the charlottesville thing.
snsabot: (trilema) 2017-10-06 BingoBoingo: In other ???, Heather Heyer, the fat Charlottesville "victim" apparently died of a heart attack and not Dodge Challenger induced injuries http://www.vdare.com/articles/anarcho-tyranny-update-mounting-proof-that-the-charlottesville-five-are-political-prisoners
BingoBoingo: Well depends on if it stays just this fellow or if these start happening at the rate those schools were being shot up
asciilifeform: the fact that (afaik) there ain't bulldozers w/ turrets hastily welded on demolishing the clink where $d00d is kept prisoner -- already is hint, imho, of outcome.
asciilifeform: ( it dun take all that much firepower to break somebody outta city clink. as demonstrated in e.g. afghan, where was routine event during usg occupation. )
asciilifeform: 1-2 rpg rounds to the walls, the terrified polizei run as fast as their legs can carry, etc
BingoBoingo: I expect the outcome here will be disappointing, but...