vex: I like the masks, strolling through the city like a fucking bankrobber. What a time to be alive.
vex: Fishwraps headlined with "unmasked protestors"... that bit flipped quickly
punkman: rent is fine, if you can also afford not to rent.
punkman: re:usury, sorta funny how the "islamic banking" folks work around it
punkman: http://logs.nosuchlabs.com/log/asciilifeform/2021-09-22#1059400 << before I clicked, thought it was gonna be site dedicated to end of Bretton Woods
dulapbot: Logged on 2021-09-22 22:08:42 billymg: asciilifeform: ah, yeah, the whole wtf happened in 1971
punkman: cool colleciton of graphs, but kinda hard to build argument on top of fake GDP/Productivity/CPI/whatever data
punkman: the more you move to the right of 1971, faker the data
punkman: http://logs.nosuchlabs.com/log/asciilifeform/2021-09-22#1059297 << never heard the man talk. so he puts of fake accent on his podcasts? pretty funny
dulapbot: Logged on 2021-09-22 20:29:48 thimbronion: asciilifeform: also has fake slavic accent, if so.
punkman: I read BAM few months ago, liked it.
punkman: he's romanian apparently https://twitter.com/costinalamariu
punkman: "Costin Vlad Alamariu moved to the United States from Romania at the age of ten; he has an undergraduate degree in mathematics from MIT, studied at Columbia and the Sorbonne, and has recently completed his doctorate at Yale University in the political science department"
punkman: "he has taught at Yale and Emory, and is currently traveling in Europe and writing a book about the lives of tyrants in ancient Greece and Renaissance Italy."
punkman: looking at my naive implementation, evil peer can rebroadcast all messages timestamped within 15min, always setting bounces=1, causing other peers to also keep rebroadcasting same messages,
punkman: as receiver of these rebroadcasted messages, you can't tell who the evil peer is
punkman: so I guess my Station needs to keep track of what message it has rebroadcast to which peer, and also what rebroadcast message Station has received from which peer
punkman: first part for Station not to misbehave, second part to know which peers are misbehaving and warn operator
PeterL: punkman: two things: 1) when you see the flood, you can drop the max_bounces to 1, then either it stops or $Bad_Peer is the only one sending and it is obvious
PeterL: and 2) the messages will all appear as hearsay of the form $any_user($bad_peer):message , which will make it pretty obvious who is misbehaving
PeterL: although, this brings up a question, iiuc the peers displayed are only the ones with 1 bounce? if you have your bounce setting high, what would get displayed for a message from your L3, the first peer you see a message from? or anybody with bounces = 2?
punkman: PeterL: yes maxbounces=1 stops my Station from rebroadcasting anything that is not immediate msg. Must also not allow hearsay messages with bounces=0 of course.
punkman: still not obvious who is doing it though
PeterL: the name of who is doing it is right there in the parentheses
PeterL: the maxbounces affects what you accept, not what you rebroadcast
PeterL: (if I understand correctly?)
punkman: no because A sends m1_bounce=0 to B. B starts sending m1_bounce=1 at line rate to a subset of peers, the subset of peers starts rebroadcasting m1_bounce=2 to everyone else
punkman: only subset of peers knows that it's B
punkman: and everyone else thinks it's someone in subset of peers
punkman: and it multiplies, B sends one packet to C, C will rebroadcast N packets
PeterL: right, that is how it is supposed to work, and people de-dupe as they receive the same packet from multiple peers
punkman: yes dedupe saves us from storage ballooning, but not from bandwidth/cpu ballooning
PeterL: so in your exacmple who is the bad peer?
punkman: B
PeterL: well, once everybody has seen the message once they stop resend it
punkman: you just arrived at my "solution"
PeterL: I don't understand what you are trying to solve?
punkman: it seems to me that we should "dedupe" red packets, in addition to dedupe of messages
PeterL: isn't that the same thing?
punkman: message is subset of red packet
PeterL: right, and we expect roughly our entire peerset to each send us a red packet with the same message
PeterL: but with the nonce and bounces, we should not get the same red packet twice
PeterL: if a peer sends the same message twice in two different red packets, the second one gets dropped when you see you already have that message
punkman: so we would only rebroadcast if it's first time we see message
punkman: hmm guess that makes sense, will rethink
PeterL: right (I thought that was obvious from having deduping?)
PeterL: I thought you were describing a malicious peer that was rebroadcasting messages with altered timestamps to make it look like other people were trying to flood the network
asciilifeform: punkman, PeterL : we only rebroadcast the first time we see a message !!
asciilifeform thought this was 100% obvious from the doc -- apparently not ?
asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2021-09-23#1059441 << and again, rebroadcasting takes place ~only~ if accepted (as validly signed AND not stale AND not seen previously message)
dulapbot: Logged on 2021-09-23 09:46:33 PeterL: the maxbounces affects what you accept, not what you rebroadcast
asciilifeform: i.e. any packet containing a previously-seen message is treated exactly the same as a martian (with the possible exception of incrementing optional 'dupes' counter instead of 'martians')
punkman: yes got confused looking at my code, although if we stop at 4.2.1 "7. The deduplication queue..." we don't get to the part where we count hearsay copies
asciilifeform: in current draft, hearsay dupes are mandatorily counted and the count is displayed.
asciilifeform: punkman: i wrote this part recently, so possibly you haven't seen it if were working from the 0xFE piece
asciilifeform: (the current one is linked above)
asciilifeform: and yes the text aint 100% consistent yet, i wrote new sections w/out much updating the old, much remains to be done there
asciilifeform: ( and sadly atm not time to do it, grr )
punkman: I'm not only counting, I also have counter carry a "got_immediate" flag
asciilifeform: anyways dupes counted, but a given peer only counts as '1' for this purpose no matter how many times he sent the dupe (if he is broken somehow. an in-spec peer will never send same msg twice under any circumstances.)
punkman: thinking how to implement the embargo now
punkman: perhaps better to keep counter with set of peers instead of integer per message
punkman: asciilifeform: if shalmaneser changes his handle to ShalmaneserTheGreat, does he keep same selfchain?
asciilifeform: punkman: of course he does. chains are stored per-peer
asciilifeform: handles are simply a thing a peer can have N of
asciilifeform: (N >= 1)
punkman: and a guest of shalmaneser would start his own selfchain, right?
asciilifeform: correct
punkman: so must track selfchain for off-wot hearsay guests
asciilifeform: this is why i suggested that guests, bots, etc. run on separate instances of the station proggy
asciilifeform: punkman: this is a good point actually, i haven't specified proper handling of chains in hearsay case
asciilifeform: (in out-of-wot hearsay, concretely)
asciilifeform: imho is not essential, one could live happily without attempting to process it at all
asciilifeform: and in fact processing it is a potential storage exhaustion attack
asciilifeform: ( imagine an l3 bozo sets up a lolscript to generate 1e9 handles and emit a msg with each )
punkman: I guess we would expect stations with guests to rate limit their guests
asciilifeform also not properly treated the subj. of rate limiting yet
asciilifeform brb
punkman: and perhaps also not let them spam different handles
dulapbot: Logged on 2021-09-10 15:34:04 asciilifeform: thinking about rate-limit -- a limit of e.g. 3 msg/min ~as displayed in console~ (importantly; rather than in general, which would impede multipath propagation) -- would prolly be reasonable. but not sure whether this belongs in algo spec.
dulapbot: Logged on 2021-09-10 15:50:45 asciilifeform: the important thing to note is that a net of say 100 people will contain quite a bit moar (largely dupe) traffic than you'd ever want in your console -- but still useful for multipath propagation/resilience. so any per-peer rate limit must affect only console imho.
asciilifeform: to asciilifeform seems like this is actually easier to implement than seems at first: rate limit ~outgoing~ spew ; erry noad already can reject crapola at line rate (or ~oughta~ be able to, if operator is not a miser for hardware) so no special spew-rejection logic is req'd aside from for console
punkman: 1 message per 20 seconds can be miserable
asciilifeform: probably the Right Thing would be e.g. 100msg per 10min, 'chess-style'
asciilifeform: permits fast responses, bot output, etc. while limiting 'burst pipe' puddle
PeterL: asciilifeform: what would you estimate the minimum hardware required for processing packets at line rate?
asciilifeform: PeterL: let's assume ethernet. so, in bytes, let's calculate 1 packet's mass: 5 (gap) + 4 (preamble) + 14 (eth header) + 20 (ipv4 hdr) + 8 (udp header) + 496 (pest) + 4 (ethernet crc) == 551
asciilifeform: standard gap is 12 iirc, and preamble is 8
asciilifeform: but let's go with the 'tight' ones
asciilifeform: 1e9 / 8*551 ~= 226860 packets/s, if all received packets match the 446byte length (ones which do not, you throw out immediately, they do not cost very much except in bw terms)
asciilifeform: a single cpu would need to process each packet inside ~4.41uS.
asciilifeform: on e.g. 3GHz cpu, that's ~13230 ticks.
asciilifeform: rather tight, tho not patently impossible. but where the fuck are you even gonna find a 1core cpu nowadays.
asciilifeform: i expect you'd have at least 8cores on reasonable box, which gives you ~105840 ticks to process a packet, imho entirely doable.
asciilifeform: PeterL: makes sense ?
asciilifeform: i expect that the hardest piece to fit in this constraint will be the dupe check
PeterL: makes sense, yeah
asciilifeform: PeterL: if pestism catches on, will be very easy to bake a fpga prefilter, much cheaper (and lower mains current usage) than a beefy x86
dulapbot: Logged on 2021-09-19 16:04:04 asciilifeform: i also specifically like the idea of a non-numbertheoretical basis for the signatures -- will make it considerably easier to implement in fpga for baking dedicated (hardware) line-rate prefilters.
asciilifeform specifically aiming from the start to make this cleanly doable
asciilifeform: grr did i say 446
asciilifeform: the figures are correct ( for 496 , which is what they weight in current draft ).
asciilifeform: typo.
asciilifeform: anyways this is the method to estimate 'can eat at line rate?' on given iron.
asciilifeform: in practice of course you will not have 100% of avail. cpu cycles usable for pest!
asciilifeform: ( not to mention, some parallel process has to actually carry on with the packets which passed the filter )
asciilifeform: goes without saying that the thing will need to be implemented ~not~ in a interpreted lang, or 1 with gc, if you want line-rate performance !!
PeterL: I think the library thimbronion found specifically came with a caveat "do not use this if you need it to be fast"
asciilifeform: PeterL: prototype aha
PeterL: I guess it only needs to go that fast if you actually saturating the line
asciilifeform: for use in-anger will need not only compiled lang but i suspect to shave cycles (like this)
asciilifeform: PeterL: the cure for ddos as offered in pest is not simply 'can process at nic line rate' but also the ease of switching to fallback line in flight (all you gotta do is to start sending from it, and peers will start replying there as soon as each receives even 1 valid packet from said new line)
asciilifeform: and, as corrolary, the ease of spreading legitimate traffic over a block of N ips you control
asciilifeform: the human operators do not need to be woken up and asked to reconfig anyffin
asciilifeform: aaaand there's the fact that an outsider won't even know where to direct ddos if you have strong opsec.
asciilifeform: so in fact you have three simultaneous nukes, ~any 1 of which~ imho readily cures ddosism.
asciilifeform: will also add that a machine which cannot keep up with the line rate, is still usable if you simply constrict the line. it isn't as if packets won't be dropped in a ddos scenario anyway. you have 'getdata' for retransmission requests, thing will simply behave as a slow modem.
asciilifeform: so you have too few cycles on given iron? ok, simply means that you need to throw away certain % of 496b packets without eating'em. not fatal.
asciilifeform: peer will do the getdata cycle, wait his turn, etc
asciilifeform bbl
PeterL: also, if your n peers each send you the message, and you drop a %, you will still get most messages. direct messages will be affected though
asciilifeform: PeterL: direct messages will go through same getdata cycle , if 1 goes missing, as others
asciilifeform genuinely bbl
punkman: if we detect a fork, do we embargo (for 1 sec, more?) and try to resolve by getdata missing messages
asciilifeform: punkman: you're thinking of a break in the chain (rather than fork per se) -- and i oughta treat the cases separately in the doc
asciilifeform: ( a fork is when your station picks up ~2 or more~ known continuations of a chain )
asciilifeform: a break otoh is simply when the item pointed to by selfchain or netchain aint in the cache
asciilifeform: ( incl. the zero case )
asciilifeform: upstack, http://logs.nosuchlabs.com/log/asciilifeform/2021-09-23#1059507 << note that this example is for a gigabit line. for e.g. 100Mbit, divide by 10, and so forth
dulapbot: Logged on 2021-09-23 15:16:44 asciilifeform: 1e9 / 8*551 ~= 226860 packets/s, if all received packets match the 446byte length (ones which do not, you throw out immediately, they do not cost very much except in bw terms)
thimbronion: PeterL: there is are also mcrypt bindings for python. For now it seems like a hassle to import that dependency.
asciilifeform: meanwhile, coupla thoughts re pest cpu cycle shaving :
asciilifeform: sha384 (and 224 for that matter) don't expose their full internal state from the output, and so don't actually need the double-pass hmac construction.
asciilifeform: in fact, with these, you can compute e.g. S = SHA384(K + Message) and cache the state for SHA384(K) per peer, and farm the remainder out to worker threads
asciilifeform: i suspect there's no actual risk to 224, for that matter -- K aint a long-lived seekrit (we're doing rekeys at reasonable intervals) , and on top of this enemy has ~15min to usefully collide the hash
asciilifeform: faster hash -- moar ddos resistance, with the obv. tradeoff.
asciilifeform: upstack, must remind folx that whether $box keeps up 'at line rate' will depend also on # of ~keys~ in the wot (and not strictly # of peers, recall that at various times there may be >1 key per)
dulapbot: Logged on 2021-09-23 15:21:07 asciilifeform: i expect you'd have at least 8cores on reasonable box, which gives you ~105840 ticks to process a packet, imho entirely doable.