Hide Idle (>14 d.) Chans


← 2023-02-08 | 2023-02-10 →
phf: it's still code some corner issues and the code needs to be cleaned up, but pest.lisp now incrementally places messages based on their netchain/selfchain and timestamps
phf: *still got some
asciilifeform: phf: neato
phf: test
phf[asciilifeform]: i'm by the way getting a constant stream of addresscasts http://paste.deedbot.org/?id=fEJ2
asciilifeform: phf: per spec, these only get sent to 'cold' peers; likely bug sumwhere
asciilifeform: ( unless these are actually 'cold' vs. phf, impossible to tell from here )
asciilifeform: phf: does your prototype eat these (and unable to 'warm' $peer on acct of need for nat hammer..?) or notyet ?
phf: well, my counterparties have a bunch of cold peers, so they are just spamming everyone with addresscast requests
asciilifeform: a! that'd make sense
phf: i'm sure if you were to turn on or hack in packet logging in blatta you're going to get the same dilluge of addresscast requests
asciilifeform: probably
phf: i suspect pest traffic is addresscasts by weight at present
phf: *bulk of
asciilifeform not devised any way to avoid this, given a seekritkey-based pest, that'd be compatible with 'only peers have any biz knowing yer addr' dictum
asciilifeform: phf: nearly certainly tru
phf: well, i've proposed that "who you're looking for" should be part of the message, so at the very least peers can coordinate broadcast
asciilifeform recalls thrd. key problem there is that it leaks info publicly that perhaps the sender would not want (identities of 'cold' peers) leaked
asciilifeform also not convinced that it'd cut down the bw guzzle substantially -- you still gotta fwd the broadcast even if marked peer aint in your l1, he could be elsewhere in l3+
asciilifeform: notion behind addrcast was that the peer oughta receive it if he's anywhere (within max bounce) on the net.
phf: well my experiment showed 2/3 reduction
phf[asciilifeform]: sure, but without very explicit algorithm, that's not at present in the spec, you're going to get blatta implementations, where peers spam network with constant addresscast requests
phf: i mean "did peer X appear on the network within the last few seconds?" asked every few seconds is definitely not the way to go about it
phf: and the "cold" peers leak can be delt with in other ways, for example you have to scrub the name if you're rebroadcasting. this at least allows storngly connected peers to "coordinate" addresscasts
phf: at present you have, and i'm directly peered to dulapbot, deedbot and awt, a handful of stations yelling at each other about a small number of peers that have probably been down since forever, and will probably not come back anytime soon
phf: of course the main problem though is that blatta is a on backburner, so an iteration cycle on issues like that, that need to be explored and tuned, counts in weeks if not months.
asciilifeform: phf: i recall sumbody (possibly phf) suggested 'only send addrcast if saw the peer alive via hearsay'. may be worth considering
asciilifeform: (the tradeoffs, imho, obv.)
asciilifeform: would cut down on the 'shouting to people who aint coming any time soon' thing.
asciilifeform: possibly would not work well for bots, however, which rarely speak
asciilifeform still not satisfied that anyffin other than the current bw-guzzling algo would work reliably in a setting where folx connect through short-lived/unreliable pipes ('wardriving' etc)
asciilifeform: then again, it remains to be seen what, if anyffin, worx there.
asciilifeform: ideal is that links to peers oughta heal, after ip change, asap.
asciilifeform: ( and whether or not you / the peer are particularly 'talkative' )
phf: i don't think current algorithm is related to wardriving, though
phf: you have to types of connection nated and direct, when a wardriver goes online he attempts to establish direct connections, and he always have to have at least one
phf: *two types
phf: having established that direct connection, he can send addresscast to all peers, ~and at that point he's done~
phf: because whoever's online and nated, will respond to addresscast and go into nat handshake
phf: but anybody who's offline, will do similar algorithm once he's online
phf: so purely to reestablish connectivity to all available peers, there's no need to spam network
phf: current algorithm prods for peers that are explicitly offline, with the goal of catching them as soon as they appear online
phf: but if you're peered with someone, they have as much an incentive in finding you, as you're in finding them, so more likely than not, when they are ready to talk, they'll either connect to you directly, or send an addresscast
phf: so if the question is ~just~ network self-healing, then you can addresscast at specific events, and then very rarely, and achieve desired result
phf: e.g. addresscast when you hear from peer on hearsay, when peer's "last packet" timeout goes beyond some threshold, when you come online yourself and can't direct connect to peer. there might be one or two more events like that, but that'll give you effective coverage
phf: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and then not par
phf: 􏿽ticularly care that some of my l1 are getting hearsay
phf: (to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)
phf: http://logs.bitdash.io/pest/2023-02-09#1022809 << *whoever's online, nated, or had his ip change since wardrive went offline
bitbot[asciilifeform]: Logged on 2023-02-09 12:21:36 phf[awt|deedbot]: because whoever's online and nated, will respond to addresscast and go into nat handshake
asciilifeform: phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed
asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
bitbot[asciilifeform]: Logged on 2023-02-09 12:33:08 phf[awt|deedbot]: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and th
asciilifeform: current algo is indeed braindamaged in that it makes no attempt to determine whether 'i can't see $peer' is because ~own~ ip changed, or the latter wandered away.
asciilifeform: indeed in the former case oughta refrain from spamming addrcast.
asciilifeform will amend draft spec re subj
asciilifeform: ( concretely: if you know that $peer was able to reach you at $addr, and prods indicate that $addr not changed since -- no reason to spam )
asciilifeform: kudos to phf for pointing this out .
asciilifeform: the 1 possible case not handled is if $peer somehow lost his AT cache. but not sure why he would w/out also losing keys
asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022818 << this also imho good idea (tho would need to think about when precisely oughta do it)
bitbot[asciilifeform]: Logged on 2023-02-09 12:40:22 phf[awt|deedbot]: (to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)
asciilifeform: ( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )
asciilifeform: ... and does blatta even currently store 'yes, i sent this' mark ? ( cuz if not, buncha stuff will get marked bogus. maybe this behaviour will need to be version-gated, then )
asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022816 << hammer prolly oughta default to off, aha
bitbot[asciilifeform]: Logged on 2023-02-09 12:33:08 phf[awt|deedbot]: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and th
phf: http://logs.bitdash.io/pest/2023-02-09#1022821 << yeah my pest instance is "mobile", so there's a clear point when need to do bookkeeping after suspend or network rotation, since there's a brief period when socket starts throwing system level exceptions
bitbot[asciilifeform]: Logged on 2023-02-09 13:18:30 asciilifeform[4]: phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed
phf: but you're right can have similarlogic in prod receipt
phf: 􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to green, whi
phf: 􏿽ch will initiate some elaborate discovery mechanism
bitbot[asciilifeform]: Logged on 2023-02-09 13:19:54 asciilifeform[4]: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
phf: http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"
bitbot[asciilifeform]: Logged on 2023-02-09 13:19:54 asciilifeform[4]: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
phf: because if you don't see them, they most likely don't see you either, which will initiate discovery on their end also
phf: so if a peer "went silent" is aggressive discovery ever necessary?
bitbot[asciilifeform]: Logged on 2023-02-09 13:55:19 asciilifeform[awt|jonsykkel|deedbot]: ( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )
phf: asciilifeform, signpost, crtdaydreams, in related is any of you running a station that's doing addresscasts? because all the addresscast packets that i'm getting have ("awt" "unpx" "jonsykkel") as speakers
phf: theoretically i should be peered with all three of you, but i don't have dc established
phf: actually running a historgram for a few minutes, gives me (("jonsykkel" 56) ("awt" 179) ("unpx" 24))
phf: (("jonsykkel" 98) ("awt" 265) ("unpx" 42))
phf: (("jonsykkel" 126) ("awt" 326) ("unpx" 54))
phf: (("jonsykkel" 168) ("awt" 446) ("unpx" 72))
phf: (("jonsykkel" 210) ("awt" 584) ("unpx" 90))
phf: (("signpost" 79) ("jonsykkel" 504) ("awt" 1066) ("unpx" 215))
asciilifeform: phf: can't speak for the rest, but currently still on blatta 9973 (where, iirc, not yet addrcast)
asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022842 << ( excluding the obv 'switched off box' ) afaik would just about always be one of a) at least one of you got new dynamic ip, or b) at least one of you is under nat and lost ephemeral port state
bitbot[asciilifeform]: Logged on 2023-02-09 19:37:11 phf[awt|deedbot]: http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"
asciilifeform: ( if anyone can think of an unrelated 'c', plox to write in )
asciilifeform: ( i suppose 'one or both of you moved box to new static ip' could count as 'c', but let's count it as 'a', lol )
asciilifeform: and imho it would make sense not to spam addrcast if 'you know it wasn't you' and $peer ought still have valid addr for you
asciilifeform: the remaining open q afaik is wat-do if you know it was you, but sending 1 addrcast not results in peer returning ( because, say, his station down just then ). how often to spam.
phf: in that last case why not wait till peer comes online and starts sending his own addresscast to find you?
asciilifeform: http://logs.bitdash.io/pest/2023-02-09#1022839 << imho requiring manual intervention for addrcast is an ugh -- some stations are automated/headless, e.g. dulapbot & similar
bitbot[asciilifeform]: Logged on 2023-02-09 19:35:34 phf[awt|deedbot]: 􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to
asciilifeform: phf: maybe asciilifeform misunderstands, but sounds uncomfortably like 'if two trains meet on the tracks, neither shall move until the other has started'(tm)
phf: yah i can see that
phf: in the ideal scenario, you have two stations A and B, A changed ip address and B went offline. A sends out addresscast on account of changing ip, but B is offline. eventually B comes online, sends addresscast on account of coming, and not being able to connect to A.
phf: but
phf: if B is "offline" due to network issues in A->B direction, then B would never know anything was amis
phf: actually no, B will know that A's offline, because A eventually will stop sending garbage, etc.
asciilifeform: aha! B may be 'offline' only with respect to A.
asciilifeform: correct, but neither will be in 'i just booted, so let's send addrcast' state
phf: right
phf: this needs more thought, but at least "my ip is still the same, i'm not going to spam" is a solid change
asciilifeform: ( 1 common, asciilifeform suspects, cause -- ephemeral nat state lost for 1 particular peer )
asciilifeform: defo belongs in spec.
phf: (("signpost" 161) ("jonsykkel" 1401) ("awt" 2659) ("unpx" 599))
asciilifeform: ( for thread-completeness -- the ~other~ purpose of addrcast is to make it so that a new peering where both sides are present on same pestnet not requires exchanging AT entries at all. so same q, how much to spam )
phf: the idea is that if i connected through signpost only, but i have peering with awt and unpx, i should be able to get later's ATs without explicit negotiation?
asciilifeform: supposing 1 and/or the other peers with signpost -- correct
phf: by explicit negotiaton i mean operator asking on channel, etc.
phf: right
asciilifeform: is mechanical (and confidential) substitute for 'ask him on pestnet what is his ip'
phf: just use fibonacci :D
asciilifeform in similar vein once suggested 'just send an 'ignore' to whole ipv4 space' lel
asciilifeform: on decent pipe, not even takes so long
asciilifeform: won't cure nat tho (quite the opposite..)
asciilifeform: iirc 'spam all of ipv4' takes ~40min on a gb/s pipe (if you have a decent map of the lacunae in the space, that is)
phf: also calling the person over phone and asking him for his new ip
asciilifeform: fwiw initial pest bringup already requires an out-of-band talk, so yes
asciilifeform: getting an addr, tho, unlike key exchange, oughn't require talking to $peer directly, is the point tho.
asciilifeform: merely talking to someone he's already able to talk to.
asciilifeform: ( apropos, in the 'spam ipv4 thread', iirc noted that if it suffices to find ~one~ of your peers, then if you have n of'em, bringup of AT-less but properly keyed station is suddenly practical. only need to hit 1. )
asciilifeform: may seem idiotic, but can think of at least 1 scenario where it'd win -- e.g. sumbody who switches on station very infrequently
asciilifeform: 'i was at sea for a year and where did erryone go'
asciilifeform: in principle, pest oughta take ~erry~ possible advantage of the fact that you only need to get 1 packet through to establish a link w/ peer.
phf: i like the idea of leaving pest bots in various corners of the internet
asciilifeform: phf: a la deedbot on dulapnet ? wainot
asciilifeform recalls thrd re www doors etc
asciilifeform: intended use there, tho, is drawing in n00bs
phf: right, you still need an addresscast. you can e.g. get a temp peer with deedbot, and then broadcast through it
asciilifeform: correct. simply pointing out that in principle not necessary for existing (i.e. keyed) peer to find such a corner, even in worst case where both sides ip-drifted in such a way that can reach no one
asciilifeform: ~40min of spam cannon and yer back in biz
asciilifeform: (on acct of 'bday paradox', likely much less )
asciilifeform: i.e. hypothetically, pest letsya treat the net as a (sloow) broadcast medium, if necessary.
asciilifeform: possibly could even have client where omit manual AT entry entirely...
asciilifeform: sounds like a lul, but think, yer on 'mobile' station, ip changes maybe multiple times each day, and you meet somebody and want to exchange keys. not even makes sense to give him an at entry.
asciilifeform: a fully grown pest client oughta include the cannon, and then can peer even if yer both 'vagrants' with 'no fixed place of residence'(tm)
asciilifeform: ( tho at least 1 side oughta be un-nat'd, obv )
phf: yeah, that nat problem makes it not as elegant as it could be
asciilifeform: for so long as pestnet not consists entirely of hobos , still worx
phf: in reality will probably really on a handful of heavily connected nodes, like deedbot and dulapbot
phf: like once i work out the kinks(tm), will stand up a logging station like btcbase, which will probably end up being one of just a handful of peers for the roaming station
asciilifeform considered a 'slave mode' which merely fwds packets, but this is tricky for the obv. reasons
asciilifeform: in 0xfb, instead proposed 'slave mode' where msgs coming through $peer (who you know to be one of your personal bots) are treated as 0bounce. but this afaik not implemented anywhere atm.
asciilifeform: (and not handles direct msgs in any way)
phf: it's fairly trivial to cook up a relay bot out of the bits in pest.lisp
asciilifeform: ( from e.g. asciilifeform's pov, a bounce-1 via dulapbot is precisely as authentic as a bounce-0 )
phf: right
asciilifeform: the elegant pill is to make it simply a one turn of knob setting, with stock clients, rather than requiring a dedicated bot proggy
asciilifeform: then -- set up bot in proper dc, and is as good as sitting there yerself
asciilifeform: ( the obv. downside is that if said box is stolen, you gotta issue new keys out of band. )
phf: i remember mp at some point objected to everyone idling on bouncers, after having voiced themselves, on account of opsec. he wasn't too insistent on it, so it didn't go anywhere, but the problem remains.
← 2023-02-08 | 2023-02-10 →