#pest | 2023-02-09

← 2023-02-08 | 2023-02-10 → ↓

phf: it's still code some corner issues and the code needs to be cleaned up, but pest.lisp now incrementally places messages based on their netchain/selfchain and timestamps

phf: *still got some

asciilifeform: phf: neato

phf: test

phf[asciilifeform]: i'm by the way getting a constant stream of addresscasts http://paste.deedbot.org/?id=fEJ2

asciilifeform: phf: per spec, these only get sent to 'cold' peers; likely bug sumwhere

asciilifeform: ( unless these are actually 'cold' vs. phf, impossible to tell from here )

asciilifeform: phf: does your prototype eat these (and unable to 'warm' $peer on acct of need for nat hammer..?) or notyet ?

phf: well, my counterparties have a bunch of cold peers, so they are just spamming everyone with addresscast requests

asciilifeform: a! that'd make sense

phf: i'm sure if you were to turn on or hack in packet logging in blatta you're going to get the same dilluge of addresscast requests

asciilifeform: probably

phf: i suspect pest traffic is addresscasts by weight at present

phf: *bulk of

asciilifeform not devised any way to avoid this, given a seekritkey-based pest, that'd be compatible with 'only peers have any biz knowing yer addr' dictum

asciilifeform: phf: nearly certainly tru

phf: well, i've proposed that "who you're looking for" should be part of the message, so at the very least peers can coordinate broadcast

asciilifeform recalls thrd. key problem there is that it leaks info publicly that perhaps the sender would not want (identities of 'cold' peers) leaked

asciilifeform also not convinced that it'd cut down the bw guzzle substantially -- you still gotta fwd the broadcast even if marked peer aint in your l1, he could be elsewhere in l3+

asciilifeform: notion behind addrcast was that the peer oughta receive it if he's anywhere (within max bounce) on the net.

phf: well my experiment showed 2/3 reduction

phf[asciilifeform]: sure, but without very explicit algorithm, that's not at present in the spec, you're going to get blatta implementations, where peers spam network with constant addresscast requests

phf: i mean "did peer X appear on the network within the last few seconds?" asked every few seconds is definitely not the way to go about it

phf: and the "cold" peers leak can be delt with in other ways, for example you have to scrub the name if you're rebroadcasting. this at least allows storngly connected peers to "coordinate" addresscasts

phf: at present you have, and i'm directly peered to dulapbot, deedbot and awt, a handful of stations yelling at each other about a small number of peers that have probably been down since forever, and will probably not come back anytime soon

phf: of course the main problem though is that blatta is a on backburner, so an iteration cycle on issues like that, that need to be explored and tuned, counts in weeks if not months.

asciilifeform: phf: i recall sumbody (possibly phf) suggested 'only send addrcast if saw the peer alive via hearsay'. may be worth considering

asciilifeform: (the tradeoffs, imho, obv.)

asciilifeform: would cut down on the 'shouting to people who aint coming any time soon' thing.

asciilifeform: possibly would not work well for bots, however, which rarely speak

asciilifeform still not satisfied that anyffin other than the current bw-guzzling algo would work reliably in a setting where folx connect through short-lived/unreliable pipes ('wardriving' etc)

asciilifeform: then again, it remains to be seen what, if anyffin, worx there.

asciilifeform: ideal is that links to peers oughta heal, after ip change, asap.

asciilifeform: ( and whether or not you / the peer are particularly 'talkative' )

phf: i don't think current algorithm is related to wardriving, though

phf: you have to types of connection nated and direct, when a wardriver goes online he attempts to establish direct connections, and he always have to have at least one

phf: *two types

phf: having established that direct connection, he can send addresscast to all peers, ~and at that point he's done~

phf: because whoever's online and nated, will respond to addresscast and go into nat handshake

phf: but anybody who's offline, will do similar algorithm once he's online

phf: so purely to reestablish connectivity to all available peers, there's no need to spam network

phf: current algorithm prods for peers that are explicitly offline, with the goal of catching them as soon as they appear online

phf: but if you're peered with someone, they have as much an incentive in finding you, as you're in finding them, so more likely than not, when they are ready to talk, they'll either connect to you directly, or send an addresscast

phf: so if the question is ~just~ network self-healing, then you can addresscast at specific events, and then very rarely, and achieve desired result

phf: e.g. addresscast when you hear from peer on hearsay, when peer's "last packet" timeout goes beyond some threshold, when you come online yourself and can't direct connect to peer. there might be one or two more events like that, but that'll give you effective coverage

phf: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and then not par

phf: 􏿽ticularly care that some of my l1 are getting hearsay

phf: (to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)

phf: http://logs.bitdash.io/pest/2023-02-09#1022809 << *whoever's online, nated, or had his ip change since wardrive went offline

bitbot[asciilifeform]: Logged on 2023-02-09 12:21:36 phf[awt|deedbot]: because whoever's online and nated, will respond to addresscast and go into nat handshake

asciilifeform: phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed

asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken

bitbot[asciilifeform]: Logged on 2023-02-09 12:33:08 phf[awt|deedbot]: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and th

asciilifeform: current algo is indeed braindamaged in that it makes no attempt to determine whether 'i can't see $peer' is because ~own~ ip changed, or the latter wandered away.

asciilifeform: indeed in the former case oughta refrain from spamming addrcast.

asciilifeform will amend draft spec re subj

asciilifeform: ( concretely: if you know that $peer was able to reach you at $addr, and prods indicate that $addr not changed since -- no reason to spam )

asciilifeform: kudos to phf for pointing this out .

asciilifeform: the 1 possible case not handled is if $peer somehow lost his AT cache. but not sure why he would w/out also losing keys

asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022818 << this also imho good idea (tho would need to think about when precisely oughta do it)

bitbot[asciilifeform]: Logged on 2023-02-09 12:40:22 phf[awt|deedbot]: (to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)

asciilifeform: ( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )

asciilifeform: ... and does blatta even currently store 'yes, i sent this' mark ? ( cuz if not, buncha stuff will get marked bogus. maybe this behaviour will need to be version-gated, then )

asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022816 << hammer prolly oughta default to off, aha

phf: http://logs.bitdash.io/pest/2023-02-09#1022821 << yeah my pest instance is "mobile", so there's a clear point when need to do bookkeeping after suspend or network rotation, since there's a brief period when socket starts throwing system level exceptions

bitbot[asciilifeform]: Logged on 2023-02-09 13:18:30 asciilifeform[4]: phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed

phf: but you're right can have similarlogic in prod receipt

phf: 􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to green, whi

phf: 􏿽ch will initiate some elaborate discovery mechanism

bitbot[asciilifeform]: Logged on 2023-02-09 13:19:54 asciilifeform[4]: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken

phf: http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"

phf: because if you don't see them, they most likely don't see you either, which will initiate discovery on their end also

phf: so if a peer "went silent" is aggressive discovery ever necessary?

phf: that was re http://logs.bitdash.io/pest/2023-02-09#1022832

bitbot[asciilifeform]: Logged on 2023-02-09 13:55:19 asciilifeform[awt|jonsykkel|deedbot]: ( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )

phf: asciilifeform, signpost, crtdaydreams, in related is any of you running a station that's doing addresscasts? because all the addresscast packets that i'm getting have ("awt" "unpx" "jonsykkel") as speakers

phf: theoretically i should be peered with all three of you, but i don't have dc established

phf: actually running a historgram for a few minutes, gives me (("jonsykkel" 56) ("awt" 179) ("unpx" 24))

phf: (("jonsykkel" 98) ("awt" 265) ("unpx" 42))

phf: (("jonsykkel" 126) ("awt" 326) ("unpx" 54))

phf: (("jonsykkel" 168) ("awt" 446) ("unpx" 72))

phf: (("jonsykkel" 210) ("awt" 584) ("unpx" 90))

phf: (("signpost" 79) ("jonsykkel" 504) ("awt" 1066) ("unpx" 215))

asciilifeform: phf: can't speak for the rest, but currently still on blatta 9973 (where, iirc, not yet addrcast)

asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022842 << ( excluding the obv 'switched off box' ) afaik would just about always be one of a) at least one of you got new dynamic ip, or b) at least one of you is under nat and lost ephemeral port state

bitbot[asciilifeform]: Logged on 2023-02-09 19:37:11 phf[awt|deedbot]: http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"

asciilifeform: ( if anyone can think of an unrelated 'c', plox to write in )

asciilifeform: ( i suppose 'one or both of you moved box to new static ip' could count as 'c', but let's count it as 'a', lol )

asciilifeform: and imho it would make sense not to spam addrcast if 'you know it wasn't you' and $peer ought still have valid addr for you

asciilifeform: the remaining open q afaik is wat-do if you know it was you, but sending 1 addrcast not results in peer returning ( because, say, his station down just then ). how often to spam.

phf: in that last case why not wait till peer comes online and starts sending his own addresscast to find you?

asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022839 << imho requiring manual intervention for addrcast is an ugh -- some stations are automated/headless, e.g. dulapbot & similar

bitbot[asciilifeform]: Logged on 2023-02-09 19:35:34 phf[awt|deedbot]: 􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to

asciilifeform: phf: maybe asciilifeform misunderstands, but sounds uncomfortably like 'if two trains meet on the tracks, neither shall move until the other has started'(tm)

phf: yah i can see that

phf: in the ideal scenario, you have two stations A and B, A changed ip address and B went offline. A sends out addresscast on account of changing ip, but B is offline. eventually B comes online, sends addresscast on account of coming, and not being able to connect to A.

phf: but

phf: if B is "offline" due to network issues in A->B direction, then B would never know anything was amis

phf: actually no, B will know that A's offline, because A eventually will stop sending garbage, etc.

asciilifeform: aha! B may be 'offline' only with respect to A.

asciilifeform: correct, but neither will be in 'i just booted, so let's send addrcast' state

phf: right

phf: this needs more thought, but at least "my ip is still the same, i'm not going to spam" is a solid change

asciilifeform: ( 1 common, asciilifeform suspects, cause -- ephemeral nat state lost for 1 particular peer )

asciilifeform nods

asciilifeform: defo belongs in spec.

phf: (("signpost" 161) ("jonsykkel" 1401) ("awt" 2659) ("unpx" 599))

asciilifeform: ( for thread-completeness -- the ~other~ purpose of addrcast is to make it so that a new peering where both sides are present on same pestnet not requires exchanging AT entries at all. so same q, how much to spam )

phf: the idea is that if i connected through signpost only, but i have peering with awt and unpx, i should be able to get later's ATs without explicit negotiation?

asciilifeform: supposing 1 and/or the other peers with signpost -- correct

phf: by explicit negotiaton i mean operator asking on channel, etc.

phf: right

asciilifeform: aha

asciilifeform: is mechanical (and confidential) substitute for 'ask him on pestnet what is his ip'

phf: just use fibonacci :D

asciilifeform in similar vein once suggested 'just send an 'ignore' to whole ipv4 space' lel

asciilifeform: on decent pipe, not even takes so long

asciilifeform: won't cure nat tho (quite the opposite..)

asciilifeform: iirc 'spam all of ipv4' takes ~40min on a gb/s pipe (if you have a decent map of the lacunae in the space, that is)

phf: also calling the person over phone and asking him for his new ip

asciilifeform: fwiw initial pest bringup already requires an out-of-band talk, so yes

asciilifeform: getting an addr, tho, unlike key exchange, oughn't require talking to $peer directly, is the point tho.

asciilifeform: merely talking to someone he's already able to talk to.

asciilifeform: ( apropos, in the 'spam ipv4 thread', iirc noted that if it suffices to find ~one~ of your peers, then if you have n of'em, bringup of AT-less but properly keyed station is suddenly practical. only need to hit 1. )

asciilifeform: may seem idiotic, but can think of at least 1 scenario where it'd win -- e.g. sumbody who switches on station very infrequently

asciilifeform: 'i was at sea for a year and where did erryone go'

asciilifeform: in principle, pest oughta take ~erry~ possible advantage of the fact that you only need to get 1 packet through to establish a link w/ peer.

phf: i like the idea of leaving pest bots in various corners of the internet

asciilifeform: phf: a la deedbot on dulapnet ? wainot

asciilifeform recalls thrd re www doors etc

asciilifeform: intended use there, tho, is drawing in n00bs

phf: right, you still need an addresscast. you can e.g. get a temp peer with deedbot, and then broadcast through it

asciilifeform: correct. simply pointing out that in principle not necessary for existing (i.e. keyed) peer to find such a corner, even in worst case where both sides ip-drifted in such a way that can reach no one

asciilifeform: ~40min of spam cannon and yer back in biz

asciilifeform: (on acct of 'bday paradox', likely much less )

asciilifeform: i.e. hypothetically, pest letsya treat the net as a (sloow) broadcast medium, if necessary.

asciilifeform: possibly could even have client where omit manual AT entry entirely...

asciilifeform: sounds like a lul, but think, yer on 'mobile' station, ip changes maybe multiple times each day, and you meet somebody and want to exchange keys. not even makes sense to give him an at entry.

asciilifeform: a fully grown pest client oughta include the cannon, and then can peer even if yer both 'vagrants' with 'no fixed place of residence'(tm)

asciilifeform: ( tho at least 1 side oughta be un-nat'd, obv )

phf: yeah, that nat problem makes it not as elegant as it could be

asciilifeform: for so long as pestnet not consists entirely of hobos , still worx

phf: in reality will probably really on a handful of heavily connected nodes, like deedbot and dulapbot

phf: like once i work out the kinks(tm), will stand up a logging station like btcbase, which will probably end up being one of just a handful of peers for the roaming station

asciilifeform considered a 'slave mode' which merely fwds packets, but this is tricky for the obv. reasons

asciilifeform: in 0xfb, instead proposed 'slave mode' where msgs coming through $peer (who you know to be one of your personal bots) are treated as 0bounce. but this afaik not implemented anywhere atm.

asciilifeform: (and not handles direct msgs in any way)

phf: it's fairly trivial to cook up a relay bot out of the bits in pest.lisp

asciilifeform: ( from e.g. asciilifeform's pov, a bounce-1 via dulapbot is precisely as authentic as a bounce-0 )

phf: right

asciilifeform: the elegant pill is to make it simply a one turn of knob setting, with stock clients, rather than requiring a dedicated bot proggy

asciilifeform: then -- set up bot in proper dc, and is as good as sitting there yerself

asciilifeform: ( the obv. downside is that if said box is stolen, you gotta issue new keys out of band. )

phf: i remember mp at some point objected to everyone idling on bouncers, after having voiced themselves, on account of opsec. he wasn't too insistent on it, so it didn't go anywhere, but the problem remains.

← 2023-02-08 | 2023-02-10 → ↑