Hide Idle (>14 d.) Chans
← 2022-11-27 | 2022-11-29 → ↓
bitbot[asciilifeform|busybot]: Logged on 2022-11-27 18:00:19 phf[awt]: http://logs.bitdash.io/pest/2022-11-25#1016874 << this line lands particularly well, because i'm reading unixhaters guide at the moment
crtdaydreams[busybot]: http://logs.bitdash.io/pest/2022-11-27#1016937 << A workshop experiment was done where grease-gun cartridges were replaced with transparent ones, allowing workers to see exactly how much grease was in them at a glance. After a few days of observation it appeared that workers would simply go look for another grease gun rath
crtdaydreams[busybot]: er than fill the ones that they could see were empty.
bitbot[busybot]: Logged on 2022-11-27 21:06:57 asciilifeform[5]: not infrequently uses (raw, rather than via mc) sftp
crtdaydreams[busybot|asciilifeform]: asciilifeform: haven't had time to read latest draft spec, is this line-break issue implementation or spec based?
crtdaydreams[busybot|asciilifeform]: actually, that's a terrible question.
asciilifeform: crtdaydreams: rec to read the latest draft.
bitbot[asciilifeform|busybot]: Logged on 2022-11-10 11:51:20 asciilifeform[6]: meanwhile: signpost , awt , jonsykkel , phf, billymg , et al : preview of 0xfa. a number of sections not done, and prolly over9000 typos but fwiw.
asciilifeform: $ticker btc usd
busybot: Current BTC price in USD: $16241.76
awt[asciilifeform]: asciilifeform: I'm still completely bogged down with fiat work and relocation. Likely won't be able to give it attention until the new year.
signpost[asciilifeform]: asciilifeform: awesome, will re-read soon.
phf[asciilifeform]: unfortunately until there's a great switchover, can't really do multipart reliably, since it'll spit noise into log
phf[asciilifeform]: also multipart message not being it's own command type makes me think that asciilifeform has discovered some quality prince george's county dope.
asciilifeform: phf: notion was that old clients will eat'em as standard msgs. lemme know if dope (not tried experimentally, and jonsykkel suggested that his may not, tho can't presently recall why)
asciilifeform originally had'em as separate msg coades. then concocted the kludge pictured.
asciilifeform: fwiw if blatta & smalpest indeed would choke on the proposed multipart -- there's then 0 point in not making'em separate codes and dispensing with the kludge
asciilifeform admits that not read either prototype in sufficient detail to be certain of the answer
phf[asciilifeform]: it's got a big of a makefile effect: you'll have a kludge that's relevant for about a month, and then it becomes legacy kludge forever
asciilifeform: (there's a reason it's a 'pre-draft' lol)
phf[asciilifeform]: *a makefile effect
asciilifeform: phf: tru phakt
asciilifeform was thinking 'can avoid fragmenting an already small pestnet?'
jonsykkel[asciilifeform|busybot]: http://logs.bitdash.io/pest/2022-11-28#1016953 << this was re the initial version of new pdf spec, where no explicit 0terminator between messsage Chunk and N+Of
bitbot[busybot|asciilifeform]: Logged on 2022-11-28 11:19:38 asciilifeform[4]: phf: notion was that old clients will eat'em as standard msgs. lemme know if dope (not tried experimentally, and jonsykkel suggested that his may not, tho can't presently recall why)
asciilifeform: jonsykkel: aah
bitbot[asciilifeform|busybot]: Logged on 2022-11-15 23:02:17 asciilifeform[6]: updated pre-draft pest spec with jonsykkel's correction & coupla others.
asciilifeform: currently nfi re behaviour of blatta re subj, tho
asciilifeform: phf does have strong point, if we carry on with this kind of thing, will end up with 'x86' pile of ???
asciilifeform: alternative tho is also quite ugh, in other direction
asciilifeform: worth discussing imho.
phf[asciilifeform]: asciilifeform, the reason why its a kludge though, is that you have a declarative wire format, driven by (case command (#x00 ...)) for both packing and unpacking. now you have to kludge a new mechanism of differentiation where also check "but wait, does that region have special value?"
asciilifeform: phf: is the exact reason it's a kludge, yes
shinohai[busybot]: *asciilifeform is now known as gabriel_ladell ...
asciilifeform: lol notyet
phf[asciilifeform]: that's how the protocol can potentially look right now, in declerative lisp form http://paste.deedbot.org/?id=jnZx
asciilifeform: neato phf, is this from your current pestron ?
phf[asciilifeform]: gives me, (MAKE-RED-PACKET :COMMAND 0 :SPEAKER NICK :TIMESTAMP TIMESTAMP :SELFCHAIN SELFCHAIN :NETCHAIN NETCHAIN :TEXT TEXT)
phf[busybot]: or … (string-equal (red-packet-speaker p) nick) …
asciilifeform as presently understands, there's only 2 possible answers to 'how to multipart' -- a) the pictured one, where kludge; b) the clean one, where new codes, and old pestrons end up ignoring long msgs
phf[asciilifeform]: in fact (make-red-packet :command 0 :speaker "joe" :timestamp (get-universal-time) :selfchain 0 :netchain 0 :text "hello, world" :bounces 1)
joe[asciilifeform]: hello, world
jonsykkel[asciilifeform|busybot]: i agre no good reason to introduce permanent kludges for bakcwards compatibilty with early prototypes
jonsykkel[asciilifeform|busybot]: royal decree 48bit ipv4 in 1995 and ignore short term consequences wwould have saved a lot of truble
phf[busybot]: “According to legend, Stu Feldman didn’t fix make’s syntax, after he real- ized that the syntax was broken, because he already had 10 users.
phf[busybot]: 􏿽test
phf[asciilifeform]: 􏿽well, if nothing else the kludge works. this is a valid "multipart" message with n=1 of of=1
asciilifeform: hm the 'nonprintable' turd shows in log
asciilifeform: (maybe need a diff turd..)
bitbot[busybot]: Logged on 2022-11-28 11:58:12 phf[awt]: 􏿽test
phf[asciilifeform]: the siren song of inbandism
jonsykkel[asciilifeform|busybot]: if gonna klugde might as well dispense with the uniturd and make 1byte Z and immediately 1byte turd after
asciilifeform: jonsykkel: how would that be compat with old pestrons?
asciilifeform: yea it wont choke, but won't print either
jonsykkel[busybot|asciilifeform]: asciilifeform: i mean u simply begin the chunk where Mark is now
jonsykkel[asciilifeform|busybot]: then newer trons will just look for special value immediately after the 0terminator
phf[asciilifeform]: one could do Chunk Z N Of TextHash Mark, and you only need mark in case "old pestrons" don't pad their text with zero
asciilifeform: jonsykkel: problem with 'mark after Z' is that random padding after Z may end up == to the mark
asciilifeform: (on ordinary non-multi packet of the correct length)
jonsykkel[asciilifeform|busybot]: asciilifeform: indeed but iirc old specs had 0padding
shinohai[busybot]: phf `MAKE-RED-PACKET` isn't being very inclusive to all the other colors out there.
asciilifeform: jonsykkel: ~new~ proposed spec has random padding tho, and may issue an unintended multipart mark if sending with the correct length, under your scheme
phf[asciilifeform]: shinohai, red packet ethnostate when
asciilifeform strongly inclined to throw out the kludge, but will wait for folx to beat subj to death
asciilifeform: the obv cost is fragging of the net; bots will need swapping, folx on ancient versions (mod6?) will see some msgs and not others, etc
asciilifeform: thinking about it, theoretically mark ~could~ go after Z, and new-style pestron emitting classical single-part msg would then need to terminate'em with (0)(anything-but-mark)(random..) but still ugh
asciilifeform: inbandism is inescapably ugly
jonsykkel[asciilifeform|busybot]: put new comand for broadcasts with multipartism, ignore old 0x00s, plus emit "ur pestron is old plz run windows update" 0x00 every 5 sec until nobody in net uses old progs
phf[asciilifeform]: well, i added support for your kludge to my defwire http://paste.deedbot.org/?id=Ifgb it's possibly better even without kludge
phf[busybot]: can now say (make-red-packet :type :broadcast-text :text "hello, world" …)
phf[asciilifeform]: but i still lost the declerative mapping between `command` and its payload, through the "type" indirection. not sure if it's better or worse. i guess this fix makes it very explicit that there's no declerative mapping between type and command with this kludge.
asciilifeform not disputes that it's ugly as sin
asciilifeform: phf: worx. what was it test of ?
phf[asciilifeform]: patched defwire
phf[busybot]: hmm, this is going to fail misserably…
phf[busybot]: 􏿽Never before had Brother Francis actually seen a pilgrim with girded loins, but that this one was the bona fide article he was convinced as soon as he had recovered from the spine-chilling effect of the pilgrim’s advent on the far horizon, as a wiggling iota of black caught in a
phf[asciilifeform|busybot]: 􏿽shimmering haze of heat. Legless, but wearing a tiny head, the iota materialized out of the mirror glaze on the broken roadway and seemed more to writhe than to walk into view, causing Brother Francis to clutch the crucifix of his rosary and mutter an Ave or two. The iota suggested
phf[asciilifeform]: 􏿽 a tiny apparition spawned by the heat demons who tortured the land at high noon, when any creature capable of motion on the desert (except the buzzards and a few monastic hermits such as Francis) lay motionless in its burrow
phf[asciilifeform]: imho texthash is a waste of space. there's already a mechanism for chain reconstruction, which is netchain/selfchain. this one just eats up 32 bytess without adding any extra guarantees (except testing that the counterparty's client is not broken)
phf[asciilifeform]: asciilifeform, have you given a thought to ACTION? that's something that's not address in the spec, and if you're going to drop irc specific stuff probably should be
asciilifeform: phf: asciilifeform's rationale for texthash : 1) makes over9000 easier to organize incoming out-of-order chunks 2) provides at least modicum of resistance against buggy or otherwise misbehaving clients corrupting hearsayed chunks
asciilifeform: phf: not considered ACTION. (will need to remember how it was encoded traditionally...) may need explicit mention in spec
asciilifeform: esp. considering that irc frontends may be with us for rather long time
phf[asciilifeform] ~a~c" #\Soh text #\Soh)
asciilifeform: oughta work without explicit knobs then
asciilifeform not defined wat-do if conflicting hearsay chunks in fact received (i.e. for some N, Of, and TextHash, different payloads.) if only couple -- can iterate over'em, reconstruct. but when to try? at any rate, oughta alert operator
asciilifeform: notion is, if 1st chunk is immediate, and some number of subsequent chunks -- hearsays, ought not to have to wonder re authenticity of reconstructed payload
asciilifeform: (i.e. if it reassembles and hashes to TextHash, whole thing can be displayed as 'immediate')
asciilifeform: *if any chunk if immediate
asciilifeform: (rather than 1st)
phf[asciilifeform]: i guess that makes authenticity of whole text as authorative as the most authorative chunk
asciilifeform: correct
asciilifeform: and then not need to display thing in chunks, marking relayers for each
asciilifeform: (display 'immediate' if any chunk immediate, or min-bounce-relayers of min-bounce chunk if none of it was)
asciilifeform: ( if above not makes sense to anyone tuned in, plox to write in! )
phf[asciilifeform]: doesn't simplify the mechanism though, has to be an extra mechanism on top of existing mechanism.
phf[asciilifeform]: like you build your missing packets the usual way, holes and all, chain validation, etc. and then once the whole chain is there need to get an aggregate, merge text, hash, etc.
asciilifeform: phf: simplest mechanism is that you dun need to walk back over received msgs to know when it's time to reassemble a multi -- instead you spawn a data structure when see new texthash and fill the thing in
phf[asciilifeform]: and now your datastructure is "magic", when it comes to lost packets, etc.
asciilifeform: 'magic' in that you gotta specify when to give up if somehow not got 1 or moar chunks, yes. but that problem would have even w/out texthash
phf[asciilifeform]: but that problem is solved in a generic way already, the tradeoff of your "just stuff it all into datastructure" is that datastructure is now a kind of subgraph, need to have all the ops special handled
asciilifeform: you still gotta store reassembled multis' payloads in db in some reasonably o(1) way. so may as well allocate the space when see any chunk
phf[asciilifeform]: if i want to walk the graph and to getdata all the missing packets, or if i get a packet and want to check if it's in the graph, etc.
asciilifeform[busybot]: (when 1st see a chunk)
asciilifeform[busybot]: you dun wanna have to walk graph erry time scrolling log window etc tho
asciilifeform: 1ce you have the full text reassembled, getting to it oughta be o(1)
asciilifeform: i.e. it should behave, from operator pov, like a single msg
phf[asciilifeform]: you haven't thought it through
asciilifeform must bbl momentarily, plox to continue
phf[asciilifeform]: e.g. there's no 1-to-1 correspondence between the backlog and the display on account of any number of potential holes, and the multipart is just a special case of hole sources
phf[asciilifeform]: lacking something like "cells" architecture you're necessarily synchronizing a datastructure that manages missing packets, etc. with your display
phf[asciilifeform]: i guess a multipart bucket makes a certain kind of screen update "easier", since instead of figuring out where you need to insert new content by some kind of graph walk, you instead know "exactly"
phf[asciilifeform]: but you can't avoid that graph walk in a general case, because at any moment you might have some number of packets in flight
asciilifeform: phf: in asciilifeform's sketch of gui pestron: 'broken heart' is displayed b/w any 2 msgs where it is known there is 1 or more missing msg
asciilifeform: if/when the msg(s) w/ desired hash arrive, the 'broken heart' is replaced with same
asciilifeform: no graph walking req'd (unless asciilifeform misunderstands what was meant)
asciilifeform: when scrolling, db gets asked to supply msg with given hash (i.e. having the already displayed msg on tail of scroll as predecessor), if not found, but there's a msg with higher timestamp -- a brokenheart is shown.
asciilifeform: (higher timestamp & missing predecessor)
asciilifeform: concretely wat-do in re use of nethash to 'thread' msgs, is worth moar thought, and prolly section in spec.
asciilifeform: phf: under proposed algo, packets-in-flight dun demand graph walk -- given as they are marked with N/Of. can simply drop'em in the indicated position in prealloc'd space as they arrive.
asciilifeform: (with a clever db, not even need to make dedicated data structure for this -- simply 1st chunk with given texthash causes preallocation of contiguous space for 'Of' # of msgs; and the chunk seen sits down into the requested pos.)
asciilifeform: the slots corresponding to the missing msgs are marked 'broken heart', and filled in as they come in in whatever order.
asciilifeform: ( term 'brokenheart' cribbed here from the bolix people, who used it to represent 'transparent pointer' that gets followed silently, so that physically-uncontiguous (e.g. after gc) memory could be addressed as contiguous )
asciilifeform: this is how one can fill holes -- of width not known apriori -- after the fact, and still get close to o(1) access to the whole thing later
asciilifeform in fact did think through, lol, but invites comments if missed sumthing
asciilifeform: ideally db would store all (non-missing) msgs. in contiguous chain order, rearranging as req'd.
phf[asciilifeform]: http://logs.bitdash.io/pest/2022-11-28#1017078 << i gave up on trying to make it work using simple linear dump, with broken heart holes, because even now the graph of incoming pockets is more complicated than "just a missing packet here and there"
bitbot[asciilifeform]: Logged on 2022-11-28 16:38:26 asciilifeform[5]: in fact did think through, lol, but invites comments if missed sumthing
phf[asciilifeform]: so, yes, if you think that sequential storage of packets works, and can implement a reliable packets-insert function, then my concerns don't apply
phf[asciilifeform]: 􏿽*sequential storage meaning in flight runtime data, obviously that runtime data ultimately gets flattened and stored in db as a sequence of some sort. but i started with having my packets in a sequence, and then trying to write a "insert-packet" that also keeps track of missing hol
phf[asciilifeform]: 􏿽es for the purposes of getdata, and the result was iffy
phf[asciilifeform]: one reason for that is that while there's always one unique selfchain (since you're the one sending messages, you always have them in the order you sent them), because of transmission delays and failures netchain is subjective to each station.
phf[asciilifeform]: and the implications of that behavior are varied. like for example somebody might be getting the packets, but for some reason outgoing packets are getting stuck. why? who knows! internet routing magic
phf[asciilifeform]: 􏿽so you have everyone sharing the netchain, because they're seeing the messages in order, all at the same time. so i respond to you, you respond to me, all's fine. but this thid person with his weird internet delays is also responding to us, but we never see his messages, until the
phf[asciilifeform]: 􏿽router magic gets unkinked, and his responses all come through at the same time
phf[asciilifeform]: so your netchain is not so much a netchain as it is as net-directed-acyclic-graph
phf[asciilifeform]: you can punt on that complexity by implementing a heuristic based packet-insert that essentially maintains a stable topological sort of your directed-acyclic-graph, but having attempted that i have a hunch that the approach is not the best
phf[asciilifeform]: 􏿽i'd at least try and prototype some of these emergent behaviors kind of like http://logs.bitdash.io/pest/2022-09-07#1012520 because it's possible that i'm overthinking the complexity, and then having a concrete pseudocode algorithm would be helpful, but it's also possible that your
phf[asciilifeform]: 􏿽 intuition for the protocol's behavior reflect best case, rather than quirks of real world
bitbot[busybot|asciilifeform]: Logged on 2022-09-07 17:20:49 phf[awt]: current state of simulation code http://paste.deedbot.org/?id=TzYt
asciilifeform: phf: asciilifeform would be a dirty liar if said that he has final-solution to the pest db problem. esp. in light of fact that he'd like to store ~tree~
asciilifeform: i.e. use netchain for 'threading'
asciilifeform: there's doubtless a reasonably-compact way to do it, esp. if willing to periodically compact the datastructure
asciilifeform: (i.e. rearrange things into a logical order)
asciilifeform: fwiw none of this is detailed in the current spec, still needs thought.
asciilifeform: ( in asciilifeform's mental picture, in gui msg has 'reply' button appear when moused over, and can extend a thread that way, as seen in certain heathen chats )
asciilifeform has used various heathen chats for commercial clients, and imho most of the knobs they offer could easily exist in pest
asciilifeform: the current forced linearity of msgs, and the need for logotron links to carry a thread, imho is braindamaged and entirely avoidable if netchain is used properly
asciilifeform: ( the old logotrons, ideally, can be thrown out once a pestron exists that can serve its log over www )
asciilifeform: ... then and only then can 100% dispense with the irc-compat. frontend
asciilifeform still pondering whether dedicated 'this is a reply' bit is req'd, or can be avoided
← 2022-11-27 | 2022-11-29 → ↑
Random(pest) | Download hourly DB snapshot | Get Source Code