asciilifeform: $ticker btc usd
busybot: Current BTC price in USD: $54147.32
asciilifeform: !w poll
watchglass: Polling 17 nodes...
watchglass: 84.16.46.130:8333 : Could not connect!
watchglass: 185.163.46.29:8333 : Could not connect!
watchglass: 205.134.172.27:8333 : Alive: (0.023s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951 (Operator: asciilifeform)
watchglass: 205.134.172.6:8333 : (172-6.core.ai.net) Alive: (0.084s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Return Addr=0.0.0.0:8333 Blocks=703951
watchglass: 54.39.156.171:8333 : (ns562940.ip-54-39-156.net) Alive: (0.111s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 205.134.172.4:8333 : (172-4.core.ai.net) Alive: (0.143s) V=70001 (/therealbitcoin.org:0.7.0.1/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 205.134.172.28:8333 : Alive: (0.083s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Return Addr=0.0.0.0:8333 Blocks=703951 (Operator: whaack)
watchglass: 208.94.240.42:8333 : Alive: (0.160s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 143.202.160.10:8333 : Alive: (0.235s) V=70001 (/therealbitcoin.org:0.7.0.1/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 205.134.172.26:8333 : Alive: (0.277s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Return Addr=0.0.0.0:8333 Blocks=703950
watchglass: 54.38.94.63:8333 : (ns3140226.ip-54-38-94.eu) Alive: (0.317s) V=88888 (/therealbitcoin.org:0.8.88.88/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 213.109.238.156:8333 : Alive: (0.385s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 71.191.220.241:8333 : (pool-71-191-220-241.washdc.fios.verizon.net) Alive: (0.057s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951 (Operator: asciilifeform)
watchglass: 185.85.38.54:8333 : Could not connect!
watchglass: 103.36.92.112:8333 : (terebe.ns01.net) Alive: (0.633s) V=99999 (/therealbitcoin.org:0.9.99.99/) Jumpers=0x1 (TRB-Compat.) Blocks=703951
watchglass: 176.9.59.199:8333 : Violated BTC Protocol: Bad header length! (Operator: jurov)
watchglass: 192.151.158.26:8333 : Busy? (No answer in 100 sec.)
PeterL: http://logs.nosuchlabs.com/log/asciilifeform/2021-10-03#1060615 << I was reading through this paper
dulapbot: Logged on 2021-10-03 22:32:02 signpost: punkman: https://pdos.csail.mit.edu/~petar/papers/maymounkov-bigdown.pdf << highly recommended reading
PeterL: On p. 3, they say to solve a check block you need to have solved all adjacent blocks but one
PeterL: Wouldn't you also be able to solve a block if you have a check block that contains a subset of all adjacent blocks but one?
PeterL: I am not sure if trying to do that would just make everything more complicated or if it could actually speed up the solution?
signpost: PeterL: not sure I understand the question.
signpost: the check blocks don't "contain" adjacent blocks. they are produced via in-place xor of all the adjacent blocks.
signpost: and if you reduce their degree sure, you're making decoding easier, but the paper describes exactly how the degree distribution was designed, and what loss-correcting properties are gained.
signpost: (in fact, this is a tunable parameter which can be set depending on the lossiness of the channel)
signpost: for example, take a look at how the "suboptimality" paramemeter affects the degree distribution.
signpost: aka 'E'
signpost: anyway, naturally to solve a particular message block m5 from a check block c1 comprised of m1 m2 m3 m4 m5, you need to be able to xor c1 m1 m2 m3 m4, which yields m5
PeterL: my thought is you could xor c1 with c2 comprised of m1 m2 m3 m4, that way you can get m5 even before you have the full set of other block solved?
signpost: ah, not sure. I'd think the edges would be randomly spread out enough to not encounter tons of overlap, but have not done the math.
signpost: worthwhile if you wanted to.
signpost: other question is whether keeping a lookup table to know when this occurs is worth it.
PeterL: that was what I was thinking, it is just easier to wait for there to be d=1 block to solve from than to check each block for possible overlap
signpost: I am loathe to add state because without it the thing parallelizes mightiy
signpost: *mightily
signpost: *add more state than the pile of solved message blocks
signpost: because once solved, not fiddled with, so you can have however many cores brrrrr-ing through check blocks and looking at the same message block array for solutions.
signpost: also worth considering that new check blocks only ever become solvable when somebody solved a new message block, and then only if that was an edge. so it's straightforward which check blocks to see if solvable when a new mesage block is born.
signpost: not sure it needs to be more complicated than "1) eat check block, determine edges 2) solve if solvable 3) see if made other check blocks solvable, if so #2, else #1"
signpost: 3 should've just been *other blocks solvable, as it could've yielded an aux block, to which same checks apply
signpost: aux block's basically a check block an encoding layer beneath that repairs loss, whereas the check block encoding supplies the "does not matter which packets are lost so long as loss is random"
signpost: *high degree check block
signpost: anyway mighty welcome to have others thinking through this.
PeterL: http://logs.nosuchlabs.com/log/asciilifeform/2021-05-17#1036450 << only a terrorist would be interested in P2P transfer of large files, I am surprised it is still there!
dulapbot: Logged on 2021-05-17 13:30:33 trinque: has backups also, figures this webpage will eventually rot, or maymounkov found to have once fucked someone, or etc
PeterL: I was glancing thorugh some of the other papers on the site, "Many users would like their communications over the Internet to be private, and for some, such as reporters, lawyers, or whistleblowers, privacy is of paramount concern." < He forgot to mention criminals and terrorists!
signpost: seems like a very cool guy.
signpost: normies don't give a fuck about p2p, so there's not what to oppress here.
signpost: don't recall if I mentioned, but also one of the two that did kademlia. https://www.cs.utexas.edu/users/browne/CS395Tf2002/Papers/Kademlia-Maymounkov.pdf
asciilifeform: signpost: is where asciilifeform 1st encountered him
PeterL: Quote above is from the Vuvuzela paper, seems like they are trying to solve a simmilar set of issues Pest is meant to address
punkman: signpost: paper also assumes reliable channel (TCP), do you think same algo would work over UDP?
dulapbot: Logged on 2021-10-07 13:06:04 dulapbot: Logged on 2021-10-03 22:32:02 signpost: punkman: https://pdos.csail.mit.edu/~petar/papers/maymounkov-bigdown.pdf << highly recommended reading
PeterL: I thought the whole point was that this corrects for unreliable channel, so it should work fine on udp instead of TCP?
punkman: I think point was fast p2p file transfer, not unreliable channel
asciilifeform: punkman: with 'getdata' we have a reliable (as can be) channel via udp
asciilifeform: (tho with proper rateless code, you don't actually need 100% reliable channel, just need j of k parts of message, in whatever order, to reconstitute)
asciilifeform: however this aint part of the spec (and imho doesn't initially need to be), is only needed for fast warez/trb blox/etc movements
signpost: punkman: does not at all assume reliable channel
signpost: just assumes enemy cannot snipe packets intelligently
signpost: this oughta be provided amply by encryption, or something's wrong
signpost: (consider, if one can zap any check block of degree 1, nothing's getting solved)
PeterL: that's why you need to be able to solve by mixing blocks without any degree 1!
signpost: punkman: https://pdos.csail.mit.edu/~petar/papers/maymounkov-online.pdf << this is the original paper, and one consequence of it was fast p2p, but not primary aim
signpost: (nodes don't even need to coordinate; your friends can just start generating check blocks of w/e binwad and pissing them at you)
punkman: "Finally, the consistency of our algorithm is based
punkman: on a common, unspoken assumption that nodes
punkman: communicate over TCP. This ensures that receiving nodes won’t have holes in their knowledge of
punkman: the streams,"
signpost: doesn't in principle matter, unless I miss something
punkman: "It is an interesting question whether there is an quivalent of this algorithm that works just as well over a lossy UDP channel."
PeterL: It seems like there is a common perception that UDP is very lossy?
signpost: say I want you to blast me with a particular item. I send you a message that specifies a given hash I want sprayed my way, and parameters with which to tune the stream
signpost: you either get this message or don't, and either start sending or don't
signpost: if I don't get a stream after a while, I just fart another request your way.
signpost: iirc he contemplates sending a whole table of edges to the other side, which doesn't seem at all necessary. other side just fires up the same prng with same params.
signpost: folks are welome to tell me where I'm dense though, for sure.
PeterL: if you use a prng to generate a stream of blocks, you could give each of your friends a different seed to the prng so that they do not overlap?
signpost: *welcome
signpost: PeterL: could possibly just start encoding in a random place in the original message.
signpost: so you can feed all incoming check blocks into the same decoder, but could also do as you suggest.
punkman: I'll have to reread to grok what the "holes in streams" means
signpost figures plenty of shit to experiment with here.
punkman: been painting house, very tired
signpost: There is, of course, a handful of hacks that can get
signpost: around this problem, however most of them will increase the number of rounds in the protocol which
signpost: is undesirable.
signpost: ^ this is probably the thing, he wants an efficient transfer. I want zero coordination. possibly other problems arise, will see.
PeterL: I read the "online codes" paper a while ago, my eyes glazed over from the math and I didn't understand what it would be used for. The "Rateless codes and big downloads" paper made much more sense to me.