PeterL: hello from blatta-9988 !
PeterL: http://paste.deedbot.org/?id=_Lid << had it crash, not sure why?
PeterL: this was just as I tried to connect with my IRC client
jonsykkel: hello
PeterL: shinohai: looks like it works
jonsykkel: 9988 test
PeterL: shinohai, billymg: it looks like messages from shinohai are not ending up in the logs.bitdash.io log
awt: PeterL: the crash you saw is likely due to "gamma ray" connection to your irc port. Blatta will try to use some info from the client that get's created and then crash because the nick, for example, not there.
awt: I made a small effort to only allow one client connection but ran into some issues.
awt: shinohai: yes this is possible
awt: I make sure to do a test press before releasing
awt: So once billymg updates the logger station, we shouldn't be missing any logs due to any known issues.
awt: Hopefully my logs being public will help with debugging various NAT'd stations not receiving packets from each other.
PeterL: I'm getting a 404 when I tried to load your logs?
PeterL: ok, that one works
awt: PeterL: updated the link in the 9988 release article.
asciilifeform aboutta try 9988, will prolly end up with new ephemeral port tho, grr
asciilifeform: helloworld
awt: I see you
asciilifeform sees timestamps from shinohai and awt in AT; but notyet billymg or PeterL
asciilifeform: PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725
asciilifeform: hmm lemme try the bot nao...
bitbot: Logged on 2021-11-17 15:32:01 asciilifeform: PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725
awt: asciilifeform: that's the port I have for you now in my AT
asciilifeform: soo awt the bot silence wasn't, turns out, on account of broken deduper
asciilifeform: (given as bot gets asciilifeform's hearsays, but asciilifeform does not get bot's)
awt: asciilifeform: I received the bot echo
awt: possibly there's a record in my log of my station forwarding that on
asciilifeform: i'd assume it's a bounce limit thing, but that would not explain why it indeed does work -- in asciilifeform->bitbot direction strictly
awt: asciilifeform: in my log I can see where I received the echo from billymg, but I don't see it being repacked and forwarded as I do with other surrounding messages: http://share.alethepedia.com/blatta/logs/@40000000619521a2069255bc.s
awt: lemme restart with a ridiculously high max bounce limit
bitbot: Logged on 2021-11-17 15:44:21 awt: lemme restart with a ridiculously high max bounce limit
awt: asciilifeform: you should have received it this time: 2021-11-17 10:50:13.108495 [71.191.220.241:50725] <- 8aff65d3cb7554f8
awt: also note that afaik billymg has not updated to 9988, so quite possibly still the timestamp/dedupe issue.
billymg: running 9988
PeterL: asciilifeform: that is the port that I am sending to, but still appears nothing is going through
awt: nice!
billymg: the bot is also on 9988
billymg: as of a minute ago
billymg: shinohai: weird, looks like your messages still aren't getting logged
awt: Ok no dropped logs so far.
awt: billymg: perhaps try setting MAX_BOUNCES to something way higher like 10
billymg: btw, if i look at my /at now, jonsykkel, bitbot, and awt all have timestamps. no one else does though
PeterL: so shinohai-peterl-jonskkel-billymg-bot is four bounces?
billymg: awt: those MAX_BOUNCES would be on my billymg station, correct?
awt: My current at: http://paste.deedbot.org/?id=TUAS
awt: billymg: yes
billymg: my current at: http://paste.deedbot.org/?id=lXRD
awt: My understanding of the code is that the original message goes out with bounces=0. The count is then incremented by 1 and rebroadcast by each recipient. So by the time it reaches the bot it should be=3.
asciilifeform: btw asciilifeform's AT still as it was ~hr ago
asciilifeform: (well, with fresh timestamps for awt and shinohai , but otherwise)
asciilifeform testing bot : http://logs.bitdash.io/pest/2021-11-17#1000745
bitbot: Logged on 2021-11-17 15:32:01 asciilifeform: PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725
asciilifeform: *bounces
PeterL: and I just received my own message back as an echo...
billymg: ok restarted my station with MAX_BOUNCES=9
asciilifeform: PeterL: per spec, hearsays have an embargo interval, so that a station has a chance to receive immediate msg (if one is on its way) and failing that, to bounce the copy with the fewest bounces (did i put this in spec? possibly not?) but iirc we don't have the embargo logic yet
billymg: shinohai: wanna try the logging?
asciilifeform testing echo: http://logs.bitdash.io/pest/2021-11-17#1000783
bitbot: Logged on 2021-11-17 16:29:04 asciilifeform: PeterL: per spec, hearsays have an embargo interval, so that a station has a chance to receive immediate msg (if one is on its way) and failing that, to bounce the copy with the fewest bounces (did i put this in spec? possibly not?) but iirc we don't have the embargo logic yet
awt: PeterL, then I sent the message back to you after I got it from shinohai: 2021-11-17 11:30:18.055166 [162.247.151.243:55565] <- b3e3bf6cba250cd8
awt: WTF
billymg: !. uptime
bitbot: billymg: time since my last reconnect : 0d 0h 1m
billymg: ^ that command crashes the blatta instance the bot is connected to
awt: My station should have discarded the one from shinohai as a dupe.
awt: billymg: stack trace?
billymg: one sec, gonna try again
billymg: test
billymg: ok, logging
billymg: !. uptime
bitbot: billymg: time since my last reconnect : 0d 0h 0m
billymg: ok, restarted the logging instance, nobody !. the bot!
shinohai: >.>
billymg: ^ hey, the increased MAX_BOUNCES seem to have fixed shinohai's logging
shinohai: sweet
asciilifeform: billymg: this aint actually a fix tho. (see http://logs.nosuchlabs.com/log/asciilifeform/2021-11-17#1066384 )
awt: Other outstanding issue is why some peer's can't directly communicate, while others can.
asciilifeform: aha
billymg: awt: would it make sense to have 'max-bounces' as a startup flag?
billymg: since spec calls it out as being user configurable
asciilifeform: billymg: per spec, 'CUT' cmd sets bounces. but i dun recall if implemented yet in awt's proggy
billymg: asciilifeform: ah, gotcha
asciilifeform: billymg: thing is, 3 bounces oughta work, if the proper embargo logic were there
asciilifeform: (while 10 or even 9000 will actually fail some % of the time given the current situation)
asciilifeform: ... and produce massive bw guzzle
asciilifeform: remember that bounce cut only affects ~your~ station's acceptance of incomings
PeterL: does it accept acceptance of incoming or decision to pass along farther?
PeterL: s/accept/affect
awt: Also, might be helpful to log messages dropped due to bounce cut off. Currently blatta just silently doesn't rebroadcast.
asciilifeform: PeterL: acceptance of incoming (with the latter gated by the former naturally)
shinohai: awt: mirror of blatta vpatches now live: http://btc.info.gf/devel/irc/pest/blatta/
awt: ty shinohai
asciilifeform: awt: i suspect that if you log bounce counts (of both rejected and accepted msgs) will find the effect discussed earlier
awt: awt: yes highly likely
asciilifeform: (i.e. msgs accumulating bounce much faster than they oughta per the net's topology, on account of deduper zapping the low-bounce and even immediate copies in favour of high-bounce ones)
asciilifeform: it's why asciilifeform specified the embargo thing
awt: asciilifeform: I'm getting there lol
asciilifeform: aite
asciilifeform genuinely must bbl
PeterL: so, regardin gtheephemeral ports thing, what happens if you point something at the port I specified to blatta instead of the ephemeral port it is sending from?
shinohai: now that i have own mirror the beauty of kelvin versioning is apparent.
awt: PeterL: the packet will never reach that port because it's behind a NAT.
PeterL: shinohai: but doesn't that just list everything in reverse order? Wouldn't it make more sense to list it the other way around?
shinohai: just as easy to map $LATEST to mean lowest integer found in `patches/` neh ?
shinohai: dupes!
awt: saw it too
shinohai: `test`
awt: Weird. Completely swapped out dupe detection -- behaves almost exactly the same.
awt: Some how some messages are being packed in such a way that their hashes differ, while the message body is the same.
awt: possibly
asciilifeform: awt: do you have proper zero padding for unfilled portions of fields?
awt: asciilifeform: messages and handles are padded with the " " character using .ljust
asciilifeform: mnope, per spec gotta be 0
awt: unused fields are padded with the null character
awt: er, 0
asciilifeform: messages and handles too, per spec, gotta be
awt: asciilifeform: noted
asciilifeform: tho the lack of this wouldn't explain if indeed you got multiple hashes for same msg
awt: also why intermittent?
asciilifeform: awt: is it possible you included a field in the hash which oughtn't to be included ?
asciilifeform: the other possible explanation is that the proggy modifies fields prior to rebroadcast somewhere (per spec, nuffin in above list ought to ever be modified)
awt: asciilifeform: this is what is hashed: struct.pack(MESSAGE_PACKET_FORMAT, int_ts, self_chain, net_chain, speaker, message.body)
awt: net_chain currently always 000
asciilifeform: hm why is it message.body ?
asciilifeform: what else is in 'message' ?
awt: timestamp, speaker, etc..
asciilifeform: so why not message.self_chain, message_net.chain, ... etc ?
awt: command
asciilifeform: hm?
asciilifeform: awt: i rec to put an assert (or equiv.) that outgoing rebroadcast has same hash it came in with, or similar. then will likely find where it gets diddled.
awt: asciilifeform: check out get_message_bytes() in infosec.py -- will possibly make more sense.
asciilifeform prolly oughta reread whole thing before commenting further
awt: asciilifeform: there is also the possiblity that the messages are being originated with different contents somehow.
asciilifeform: awt: how do you mean? origination is a 1-time thing
asciilifeform: (per spec, it's what happens when you enter a line in console)
awt: asciilifeform: as implemented the message is packed for each peer.
awt: results should be identical but possibly not
awt: certainly were not before 9988
asciilifeform: if this is the case, i suspect you end up with different timestamps periodically
asciilifeform: rly oughta pack 1ce
asciilifeform: pack once, then cipher/sign to ea. peer
awt: attempted to address the multiple timestamp issue in latest patch by using the same timestamp for each pack() call.
awt: But yes the message should just be packed one time
awt: for the sake of sanity
asciilifeform expects this bug will go away when cleaned up as above
asciilifeform: and likewise you oughtnta be repacking when rebroadcasting (can't recall if doing this tho)
awt: asciilifeform: am, because currently message packing is in the same func as encryption
asciilifeform: heh then prolly also mutilates when rebroadcasting, lol
awt: quite possibly
PeterL: http://logs.bitdash.io/pest/2021-11-17#1000799 << *Magic packet* !
bitbot: Logged on 2021-11-17 16:41:39 billymg: ok, restarted the logging instance, nobody !. the bot!