jurov: great summary! i've only meddled a bit with trb mempool and then tuned out
jfw: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-15#1000235 - from mod6's article I got the idea that his was still a halfway-fix, and yours still flagged experimental. I thought I'd wait for a finished thing before digging into it. Did I miss such a thing?
snsabot: Logged on 2020-05-15 15:26:19 asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-15#1000138 << btw i notice the wedge pill aint in there. may be 1 reason why jfw's box can't sync .
jfw: don't think the 'wedge' is afflicting me though, RPC remains responsive.
jfw: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-15#1000239 - this is the right direction imo; the builtin wallet is awful in many ways (for starters: insists on spilling pregenerated keys to disk *before allowing you to encrypt*)
snsabot: Logged on 2020-05-15 16:45:20 trinque: I have a patch on my desk that cleaves off the entire wallet and stuff only used by wallet
jfw: and lots of pointless db abstractions and machinations, and some of the locking horrors, go into supporting it
asciilifeform: hm jfw for some reason i thought you already baked a trb w/ walletism cut ( and replaced with lispy wallet thing )
jfw: I made a fully-external and airgap-friendy wallet, yep; didn't yet go cutting on the cpp.
asciilifeform: ah
trinque: happy to dust this thing off then, will do sometime this weekend.
jfw: I think some export mechanism for old wallet.dat's would be important before fully ditching it though.
trinque: carved off key generation and address generation into a keyutil proggie too, but might be irrelevant with your thing.
jfw: because of the 'serialized blobs dumped in a key-value store' approach to database, doesn't yield to commodity tools.
trinque: sure, or just the "I found a wallet.dat, wat dis" scenario
asciilifeform: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-16#1000278 << this is half-correct : the patch asciilifeform & mod6 baked, caps the mass of reply to 'getdata' for blox, but not tx. but thus far no one has observed a wedge powered by requesting mempool tx ( both asciilifeform and mod6 tried to make one artificially, and mod6 iirc wrote a debug log parser to look for 'heavy' tx, to throw in 'wedger' . )
snsabot: Logged on 2020-05-16 14:12:41 jfw: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-15#1000235 - from mod6's article I got the idea that his was still a halfway-fix, and yours still flagged experimental. I thought I'd wait for a finished thing before digging into it. Did I miss such a thing?
trinque: always going to want to be able to extract those.
shinohai: asciilifeform: do u haz signed keccak genesis for trb ?
asciilifeform: ( 100% proper patch would cap both. but neither of us at the time was immediately able to figure out how to use the internal knobs to determine tx mass )
trinque: shinohai: iirc mod6 reground the whole tree
asciilifeform: shinohai: it's on mod6's www
asciilifeform: shinohai: currently both of my nodes are using it ( + the wedge pill draft )
jfw: asciilifeform: hm, ok.
jfw: trinque: if you don't mind my asking, what do you use to create transactions?
trinque: pycoin
trinque: not that I'm endorsing it
trinque: would actually be happy to migrate to your thing.
jfw: cool. It's on my list to properly introduce all that
jfw: got genesis patches already, unfortunately noticed they don't follow the subdirs convention (I didn't quite see the point of those but was won over after discussion with diana_coman)
asciilifeform: jfw: the 'one main dir' thing, i deliberately baked into 1st vtron, idea being that i oughta be able to visually identify what proj a vpatch belongs to
shinohai: ok thx. trinque i haz mod6's keccak seals and my own on www
shinohai: just no sigs for anyone else
asciilifeform: afaik not all later vtrons required it tho
asciilifeform: shinohai: i actually signed his regrind, but iirc not yet posted. (held off thinking 'will bake phf-like v-displayer w/ automatic hopper') ; will post after wrap up ch21 (what i'm doing atm)
jfw: jfw's recipe for diffing against old sha512 genesis (though it ate my <pre>)
asciilifeform: there in fact was a utf turd in the genesis. iirc mod6 removed in his.
shinohai also removed
asciilifeform: shinohai: what do you mean 'also removed' ? i thought yours was mirror of mod6's
trinque: phf ever publish his viewer?
asciilifeform: trinque: nope. i requested many times, was frustrating
trinque: weak
asciilifeform: erry time (while fella was still answering) heard 'soon!'
trinque: every node of the wot needs all pieces of infrastructure, none of this "hurr I'm lord of pulling this lever, and he's lord of pulling that"
asciilifeform: he aint dead; snarfed up e.g. ch20d not so long ago. but not answering mailz.
trinque: obvious butthurt joke is too easy.
asciilifeform: trinque: possibly he saw how asciilifeform's publishing of irc logotron was received, lol
trinque: I published "cuntoo"; how was that received?
asciilifeform: iirc he admitted once that his item was even worse kludge, ran from save-lisp-and-die snapshot and only loosely corresponds, lol, to the nominal source
trinque: doesn't matter. it has informed every step of the current item I'm working on
trinque: ahahah oic
trinque: man lisp systems have truly chthonic failure modes.
asciilifeform: trinque: i've nuffin against folx publishing their kludges, erryone is adult and knows not to plug it into missile battery
trinque: sure, same.
trinque: no items made by bags of shit and saltwater are holy.
trinque bbl, gonna get some sunshine
asciilifeform also
jfw: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-16#1000278 - oh and yes, I had a question about the change.
snsabot: Logged on 2020-05-16 14:12:41 jfw: http://logs.nosuchlabs.com/log/therealbitcoin/2020-05-15#1000235 - from mod6's article I got the idea that his was still a halfway-fix, and yours still flagged experimental. I thought I'd wait for a finished thing before digging into it. Did I miss such a thing?
asciilifeform: jfw: he dun appear to be tuned in. if you wanna pose q re item, i might be able to answer
asciilifeform rereads link
jfw: the patch seems to say "if I'm sending you a message that ends up greater than $limit bytes, you're misbehaving." but how am I to know the good-behavior limit when asking for the blocks, since I don't know how big they are?
asciilifeform answrd on mod6's www since q was asked there.
asciilifeform: jfw: in modern era, can assume 1e6 bytes per.
asciilifeform: i neglected to put this in the comment, but the reason why prb-type clients ask for blox with 'getdata', is that they do the 'headers-first sync' heresy.
asciilifeform: i.e. they ask a random noad for as many headers as possible, and then assume these represent the actual chain
jfw: figured. which *does* make for fast download at least in fair weather...
asciilifeform: at the quite obvious cost of '1stcomer tells you what the block hashes are' .
jfw: even with trb, 1stcomer could stick you with a long bogus chain due to the historically low difficulty, no?
asciilifeform: jfw: if 1stcomer is your only peer, than yes. was 1 of the reasons i proposed cement .
snsabot: Logged on 2020-05-15 16:55:06 asciilifeform: ( iirc i described this in the past. cement, not a la prb's 'dun need to verify these', but rather 'oh and btw the 1st N blox have THESE hashes, and it someone brings in a 599,999 that doesn't, he's trying to fuck you )
asciilifeform: in real life tho, it aint a very useful attack, would only work on some n00b who has 1 node and no idea of anything outside it
jfw: seems the "headers-first" could be done similarly, though I'd readily believe it's not.
asciilifeform: jfw: 1 of the afaik yet-uninvestigated atlantises of trbism, btw, is what the max practical reorg depth is. it aint 100% apparent from the coad, and needs fairly gnarly experiment, which no one's yet done
asciilifeform: jfw: it could. could ask for headers from 'cemented' hashes (and even why not store the factual sizes of the blox in the cement, so as to ask for 20MB at a time etc) .
asciilifeform: at one pt i wanted to implement, but iirc it was mp who objected, the notion struck him as too prbesque for comfort
asciilifeform: 'who are you to say what the 1st n blocks were!' or similar, it was
jfw: I mean get headers from multiple peers and go with highest difficulty
asciilifeform: my orig scheme didn't even go that far. only was 'eat .txt that was put in by hand, node keeper verified pgp sig on, then can request 1st n blox by hashes'
asciilifeform: standard trb is stuck asking for 'gimme next blox after H' . this makes for slow sync even in good weather.
jfw: mhm, esp. since it apparently only asks the one time on connect.
asciilifeform: jfw: interestingly, ben's attempt (ask erry N min of no-new-block) didn't seem to make palpable diff
asciilifeform: afaik no one ever found out for certain just why
asciilifeform: ( shitoshi's -- issued pushgetblocks on boot; asciilifeform's 'aggressive' -- on connect; ben's sequel to the latter -- after N min of stall )
jfw tempted to schedule a 'kill -9' and restart every N hours, apparently the nessary size of hammer
asciilifeform: jfw: kill then 10s later kill -9 , is the traditional recipe
asciilifeform: (otherwise risk nuking the db)
jfw: bdb can take it supposedly; meanwhile I learned that when "asked politely to shutdown", it moronically rewrites entire blkindex.dat.
asciilifeform: aha, that knob never worked properly
asciilifeform: ( again on acct of locking idiocy . it waits ~forever for the threads )
jfw: no, it's a particular self-inflicted thing, "flushing" the db which means not just checkpointing but resetting LSNs
asciilifeform: this also. but i never use that knob because the thing doesn't factually shut down in finite time when you turn it.
asciilifeform: 'windows 95' quality standard.
jfw: myeah.
asciilifeform wonders whether jfw has already eaten sufficient trb to have found out that it's factually a single-threaded proggy, i.e. can't do ~anything~ with the actual blox for >1 peer at a time , and only invokes pthreads because author had nfi how to sanely queue commands
asciilifeform: hell, even can't speak on >1 socket at 1 time (polls the sockets!)
asciilifeform: ^ was the direct mechanism for latest 'wedge'
jfw: I'd quite suspected as much from observation, and saw the RPC server is fully criticalsection'd
asciilifeform: damned near erry other set of {} is 'critical section' in trb
asciilifeform: author had nfi what is to queue, or what a thread-safe data structure might look like, so he sprinkled locks like little girl sprinkles glitter on clothes
asciilifeform: jfw: looking at my notes, seems like it's been 3+yrs since i last stood up a node the 'natural' way ( rather than via 'eatblock' , 'lighting it up' from existing noad )
asciilifeform: sorta like how stone age men worked their fires
jfw: that's one advantage of having students - forces you to notice the pain points you've grown to ignore
asciilifeform: indeed
asciilifeform: 'v', for instance, came up with because folx eventually barfed at sorting out signed-patch sequences by hand
asciilifeform: (orig. in trb we had simply signed patches.)
asciilifeform cannot claim to have discovered notion of 'signed patches', linus et al were doin' it in 1990s
trinque: yep, was a properly from-cause improvement.