billymg: asciilifeform: sadly this is true
billymg: asciilifeform: can use combination of width: 100%; max-width: 1024px; (for example)
billymg: when i read a book the pages aren't 3ft wide, and in newspapers the text is arranged into columns
billymg: but imo that's not necessary either
billymg: and in your www even without changing the sidebar you could still gain a significant amount of horizontal real estate simply by letting the layout expand to 100% of the viewport width
billymg: there's a reason it's arranged in a sidebar, because given the physical constraints it's the best layout
billymg: yeah, exactly
billymg: i can't imagine any layout where your www's sidebar is arranged horizontally (without the use of some kind of JS dropdown menus or a ton of scrolling -- or wrapping, where must scroll 3 pages down before actual content starts)
billymg: asciilifeform: each chan item isn't particularly wide, normally in cases where you have menus with N not-very-wide items they go to the left or right side in a vertical orientation
billymg: asciilifeform: cool, i'll take a look at that as well then. for now i just need to do some basics on the html/css side to make it palatable to designers and other frontend monkeys (such as myself)
billymg: things like that
billymg: asciilifeform: not complete yet, but for instance: http://paste.deedbot.org/?id=L_Cq
billymg: your current live theme will be preserved as 'classic.css' -- i promise to have it be a pixel-perfect recreation
billymg: i hope you don't mind me rewriting some of it and submitting a patch. i'm also adding a 'css_file' knob to the config for easy theme switching
billymg: asciilifeform: i've got your logger running now. i'm working on some custom styling for it. your html is killing me lol
billymg: asciilifeform: ok great, pip was my first route as well, ty
billymg: i'm not having any trouble getting these dependencies working, just wondering if you have a prescribed version or method of acquiring them that you recommend for dulap users
billymg: the only ebuild i see for flask is flask-0.12.2 and i don't see one in there for psycopg2
billymg: asciilifeform: which version of flask are you using on dulap? and for flask/psycopg2, do you install and manage through portage, pip, manually?
billymg: whaack: np
billymg: whaack: strangely enough i'm seeing the same results for your node on my end http://paste.deedbot.org/?id=v0YN
billymg: and per watchglass code, return addr is the responding node's 'addr_from' (the IP it claims as its own in its version message), which watchglass displays only if different than the ip used to connect to that node
billymg: whaack: jumpers advertise "services" of a node, see https://en.bitcoin.it/wiki/Protocol_documentation#version
billymg: gotcha
billymg: asciilifeform: as in attempting to send you bogus blocks or requesting bogus blocks from you?
billymg: but i didn't think of the 'actually valid' in true chain problem
billymg: asciilifeform: yeah, that was the intent, just see if the node will send over a block, any block
billymg: for network scanning purposes only
billymg: i mean in watchglass, just to get the hash of a block so to request it
billymg: asciilifeform: could something like this be used? https://github.com/Blockstream/esplora/blob/master/API.md#get-blockhash
billymg: so you need an actual node?
billymg: ahh
billymg: asciilifeform: from there if you added some watchglass methods for getting blocks i could then incorporate those into the crawler (if that is what you meant by trying to analyze block propagation)
billymg: probably the www too, why not
billymg: as soon as that's up will publish genesis for the crawler portion
billymg: asciilifeform: at the moment i've started writing the flask frontend for this so i can get the live updated stats up somewhere public
billymg: and for me, just seeing that table sorted by len(peers) desc made it plainly, undeniably obvious that trb nodes are the only real nodes on the network
billymg: my goal in making this crawler is to get more "bitcoiners" running trb nodes, and i suspect some al gore / nate silver stats and infograffix will widen the pool of those who see there is a problem
billymg: regarding the lack of surprise to the network being, in practice, centralized, this is true for folks following along, but i doubt it is widely known
billymg: asciilifeform: nothing that's really jumping out, user agents seem inline with overall network distribution (i.e. mostly satoshi 0.20, 0.21, a few 19s, 18s, 17s)
billymg: lemme check..
billymg: in my results 2691 nodes returning a single (self) peer
billymg: not sure if this something you've observed before, where node returns only one peer in response to getaddrs, and that peer is self
billymg: asciilifeform: btw, reason i filtered those results only on peers > 1 is because all but handful of nodes returning only a single peer returned a non-self ip http://paste.deedbot.org/?id=ljL4
billymg: asciilifeform: good to know
billymg: asciilifeform: agree
billymg: as if majority of nodes on network are in a sort of "client only" mode, receiving peers from this small group of mega spam nodes
billymg: and that of those, the ones that are not TRB, return a crapload of spam
billymg: what i found interesting is that only a handful of nodes on the network actually return peers
billymg: asciilifeform: sure, trivial to do that, but that wasn't my point
billymg: the ~8500 "actual" nodes number seems to be inline with what heathen trackers report as the total number of nodes on the network (~9k), so i suspect the crawler is nearly complete in mapping out reachable nodes
billymg: have a look at some of the results http://paste.deedbot.org/?id=hG11 -- there's an interesting pattern wrt number of peers returned
billymg: good morning, asciilifeform. my crawler seems to have hit a spam vein on the network, total unique IPs in the db exploded over the last two days to over 100k (these all come only from what a node returns in a 'getaddr' request). of those, when subsequently interrogated, only 8580 respond with a valid version message
billymg: trb* nodes
billymg: ^ two nodes i noticed with some mega peer lists
billymg: !w peers 208.94.240.42
billymg: !w peers 82.79.58.192
billymg: asciilifeform: ah, interesting
billymg: so even after "reopening" will be a large net reduction
billymg: i'm betting it'll be some kind of 50/50 thing, or up to the employee to some degree (and of course most would rather not work at home than pretend to work in the office)
billymg: lol i was just making a joke off the "yo dawg..." meme
billymg: asciilifeform: it's a browser in your browser, dawg
billymg: actually, shithub account not even needed, he's got it published at the top of his source files https://github.com/ayeowch/bitnodes/blob/master/protocol.py
billymg: i'm not terribly motivated to reach out to him myself, but i'll share his email with someone who wants to since i already have the shithub account required to view it
billymg: he's got his email listed
billymg: ahem, sorry, "digital ocean"
billymg: someone forgot to shutdown their amazon instance
billymg: lol yeah
billymg: i initially looked at it with the idea of repurposing for my crawler, barfed at 1001 dependencies pulled in, then remembered, "hey, watchglass does this"
billymg: asciilifeform: lotta hoops to jump through in setting it up: https://github.com/ayeowch/bitnodes/wiki/Provisioning-Bitcoin-Network-Crawler
billymg: ok, i've taken some notes from the patterns i saw in watchglass, otherwise i've just been trying to keep it consistent/readable
billymg: asciilifeform: btw do you perhaps use any kind of python linter for auto-formatting or have a style guide i should reference? i'm familiar with javascript conventions but not always sure what python programmers prefer
billymg: asciilifeform: definitely will
billymg: thinking about it now though i should probably sort on the last_probed column when selecting from the db
billymg: meaning each node gets hit approximately once every 13 minutes, with the ~5k in the db now. the maxsockets value is the main means of throttling currently so i'd guess one would want to set this lower when starting out so as not to wear out their welcome with any nodes on the network
billymg: looking at the last_probed column in the db, and with max sockets set at 50 (i made this a config value), it processes the entire list of nodes in the db in ~13 minutes
billymg: the thing flies now though
billymg: asciilifeform: it was quite the crash course in concurrent programming for me
billymg: not yet implemented is a "probe history" column, which will store the results of the last N poll attempts for each node (also for debug/investigative purposes)
billymg: this can be used for debug/investigative purposes, and the proggy uses it to determine a "version message strategy" (i.e. whether or not it includes the prb byte and which self version it advertises) to coax peers out of the interrogated node
billymg: as well as storing the complete version message payload that was broadcast when doing the probe
billymg: (vs. previously just using them for the next polling pass)
billymg: i'm also now storing the last list of peers returned by each node as a step towards being able to build a graph of the network
billymg: 16 of those have 'therealbitcoin.org' in the user agent string
billymg: the updated version of the crawler has been humming along nicely since last night, it's now up to ~4900 nodes discovered (heathen sites report over 9000)
billymg: which is funny considering how sensitive they seem to be to the extra 'relay' field at the end and a 99999 value for the version
billymg: asciilifeform: i'm just reporting back my observations. i tried with bogus IPs ("0.0.0.0" and "8.8.8.8" fwiw) for both the 'addr_recv' and 'addr_from' fields. the nodes i pinged simply did not care what IPs were in these fields and responded with correct/valid version messages of their own
billymg: tried on both a trb and a prb
billymg: asciilifeform: more likely it just doesn't make a difference in this particular case. for completeness i tried sending some random ips and the nodes i tried probing still returned valid/correct version messages
billymg: asciilifeform: i just tested with the values flipped to what i believe to be the correct order and it worked exactly the same as before. so i don't think this has any effect, just something i noticed
billymg: to me it looks like the order is at least inconsistent in those pack and unpack calls
billymg: it's only where you create the payload for the outgoing message that i think they are flipped http://paste.deedbot.org/?id=5BTe
billymg: asciilifeform: height field is in the correct position, addrs are in the correct order when you unpack the response payload too http://paste.deedbot.org/?id=R3w- (which is why the results in the output are correct)
billymg: going by how the spec describes the 'start_height' as the height of the emitting node and what the watchglass code does there i'm inclined to think the addr values should be flipped
billymg: i can't tell if they should be flipped or if i'm interpreting the spec wrong in terms of the "receiving/emitting" perspective
billymg: asciilifeform: based on the version msg spec are the addr_me, addr_you values in watchglass encoded in the right order?
billymg: http://logs.nosuchlabs.com/log/asciilifeform/2021-05-05#1035642 << will get to this eventually, i do want to write some guides
billymg: yeah, one less dependency to worry about
billymg: http://logs.nosuchlabs.com/log/asciilifeform/2021-05-05#1035654 << this wasn't even on my mp-wp todo list but since using postgres for the crawler it's now jumped near the top
billymg: thimbronion: fwiw that php emerge just finished successfully
billymg: jul 26
billymg: 2016 something or other
billymg: thimbronion: thing is, i installed that eselect-php-0.9.2 for the first time just now. so not sure how the tar.xz was already in distfiles unless it came with the dulap setup
billymg: well now that i have that dependency i'm just gonna try to `emerge -av =dev-lang/php-5.6.35-r1` and see what happens
billymg: so right, must've used the local copy since that url is dead
billymg: well i also have that at `/usr/portage/distfiles/eselect-php-0.9.2.tar.xz`
billymg: ahh
billymg: from `cat /usr/portage/app-eselect/eselect-php/eselect-php-0.9.2.ebuild` it looks like `SRC_URI="https://dev.gentoo.org/~mjo/distfiles/${P}.tar.xz"` (i'm assuming that's where it pulled it from then)
billymg: lemme look
billymg: i hadn't installed php on this one, so it grabbed it from somewhere
billymg: `emerge -av =app-eselect/eselect-php-0.9.2` worked for me, hrm..
billymg: thimbronion: trying now on my box...
billymg: do you see the ebuild at that same path on yours?
billymg: in my dulap (from that guide) eselect-php-0.9.2.ebuild is present at /usr/portage/app-eselect/eselect-php/eselect-php-0.9.2.ebuild so should be on yours too
billymg: thimbronion: i've found in the past with portage sometimes first emerging the stuck dependency before emerging the target package works (don't know why though)
billymg: what part is it stuck on?
billymg: thimbronion: i got the full mp-wp stack working on BingoBoingo's old rockchips, which i believe were running something very close to that (asciilifeform would likely know the differences)
billymg: from what i can see now i'll need a new watchglass method, basically the current getpeers_node but that also returns the version message response data (instead of only using it for the handshake)
billymg: asciilifeform: yeah, that's what i was thinking
billymg: asciilifeform: hrm, so regarding the once/10m rule, i'd like to collect the "probe" data from the version message response, as well as the peers from get_addrs. using watchglass's methods this means i'm hitting each node twice
billymg: ok
billymg: once
billymg: 1ce?
billymg: asciilifeform: ok, yeah, i'm relying on the watchglass methods for all protocol level interactions so that should be ok
billymg: asciilifeform: that sounds cool
billymg: asciilifeform: regarding spamming nodes with requests, anything you discovered in terms of a "minimum time between requests" value? to avoid my ip getting banned
billymg: but all the rest, []
billymg: actually not 100% true, these 3 gave a single peer result (itself): http://paste.deedbot.org/?id=LfmO
billymg: but i ran against a hundred or so prb nodes and none were returning peers when advertised version was set to '99999'
billymg: from here i'm just going to proceed with making the crawler send the correct version message depending on whether it's trying to reach a trb or prb node. but yes, perhaps could write it so it tries once with '99999' then tries with '70001' and records result
billymg: perhaps slipped in an extra if != '99999'
billymg: 99999 should be > 70001 though
billymg: works
billymg: lemme try
billymg: asciilifeform: ok, interesting. i've confirmed with this prb node that both the extra byte in the version message AND an advertised version of '70001' is required
billymg: hey, whaddya know, i left as is and only changed my advertised version from '99999' to '70001' and now it works
billymg: asciilifeform: i saw that too re: won't work for trb -- i must be doing something wrong because a test trb node is still returning peers with this extra byte
billymg: asciilifeform: yes, after getting this to work i was going to add something in that alters the message depending on whether the node identifies as trb or prb in the version message
billymg: will look at the docs again
billymg: i added '?' to the end of pformat to add the boolean at the end, and i added the last argument, True, to the struct.pack() call
billymg: any idea what i might have missed?
billymg: asciilifeform: just for testing this i modified mk_btc_ver_msg() to include the extra byte at the end. i'm running it against nodes in the db and not having any luck
billymg: 'power rangers bitcoin' another iirc -- personally i like the public relations bitcoin expansion
billymg: asciilifeform: ah, interesting. now i'm wondering what the crawler could do to coax a node into sending a 'heathen command' in a reasonable amount of time
billymg: asciilifeform: have you found a heuristic then that is not spoof-able for determining trb-compat?
billymg: oh, lol, right. it's right there in watchglass.conf
billymg: asciilifeform: so clients can spoof jumpers as easily as they can spoof user agent strings?
billymg: asciilifeform: query results: http://paste.deedbot.org/?id=sMBJ
billymg: asciilifeform: ack re: pinging jurov. will do
billymg: whether or not it actually accomplishes anything, tbd, but i personally want the stats published somewhere so it's worth me doing it
billymg: and the wot, etc.
billymg: idea is to have a www with these stats and some intro guides/overviews for V and TRB
billymg: seeing the total trb node count creates a "number go up" metric that might incentivize more people to run trb nodes
billymg: thanks, yeah and i plan to publish the patch for it once the code is where i want it. i'm also doing this as part of what will be a recruitment effort
billymg: shinohai: ty, was caught up with some fiat work for a bit
billymg: asciilifeform: i think whaack was working on a replacement block explorer. this thing i'm building is much simpler, just a network crawler
billymg: asciilifeform: yeah, for 'peers' cmd -- good to know
billymg: jumpers=0x1
billymg: so currently only works with "TRB Compat" species?
billymg: asciilifeform: awesome, thanks, i'll look into that
billymg: and probably why after ~16hrs i'm only up to ~1700 nodes
billymg: perhaps that's why so many are returning 0 peers
billymg: asciilifeform: hrm, i guess i didn't...
billymg: i need to rewrite some of it though, it's not doing it in the most efficient manner currently
billymg: i started with a seed list of the nodes that you poll in this channel, and all the rest have been grabbed from running peers recursively
billymg: asciilifeform: yep, that's what it's doing
billymg: next step is to create a simple www dashboard to display these stats
billymg: my reason for doing so was because both bitnodes and coin.dance stopped tracking trb with their crawlers. i used to be able to check from time to time to see how many trb nodes are out there
billymg: asciilifeform: i wrote a simple btc network crawler that uses watchglass for node probing and dumps the results into a postgres db. it's been running since yesterday afternoon, here are some stats so far: http://paste.deedbot.org/?id=yTLT
billymg: good to know my rc is running the latest ascii warez
billymg: feb 11 for the 10.1 version
billymg: march 17, 2018
billymg: this is on my rc, and i've never run any emerge --sync or update or anything
billymg: `/usr/portage/dev-db/postgresql/postgresql-10.3.ebuild`
billymg: i actually see a 10.3 in there as well
billymg: just found it, ty
billymg: asciilifeform: any tips for emerging postgres on a dulap box? looks like the only ebuild in the portage tree is the 'evergreen' 9999 which just pulls whatever random code is sitting on github at the time
billymg: verisimilitude: yeah, i meant your APL-like example
billymg: for instance have read one of asciilifeform's articles discussing the dismal state of CPU instruction sets and how these seem to be optimized for obfuscation and diddling rather than human operation, and perhaps this relates somewhat, but definitely over my head
billymg: at this point i'm really just feeling around in the dark though
billymg: verisimilitude: couldn't you create a proof of concept of the language you envision on top of existing text encoding schemes? and then if it compiles down to machine code anyway i don't see why a new text encoding scheme is needed
billymg: at the moment i don't grok much beyond that it could be a performance optimization, and perhaps give way to some new (better) programming languages
billymg: verisimilitude: i'm afraid a lot of this is over my head. i'm sure i can help with some js or ux related questions when you get to that stage. and perhaps visualizing the client experience will help me better understand the benefit of the underlying system
billymg: yeah
billymg: you demonstrate there would be some space savings, but i've never found text storage size to be an issue really
billymg: verisimilitude: you seem to allude to the value this adds in your first article with "The mathematician can truly create notation that suits the context. The programmer cannot." could you explain more behind the "why" of creating native word-based rather than character-based text storage?
billymg: verisimilitude: link to description of your Elision concept?
billymg: asciilifeform: ah, interesting. drips without even butt in seat??
billymg: verisimilitude: yeah, even just a vague idea would engender some trust i think. most everyone else who's stuck around here for a while hasn't kept it a mega seekrit. i do understand the desire not to self-dox though
billymg: verisimilitude: ah. some other industry, or just self-financing? i don't mean to be rude, just usually curious to know how one earns their bread when i meet them
billymg: but one in the business of building widgets for idjits
billymg: i guess could be in UK or anywhere else for that matter
billymg: verisimilitude: if i may ask, do you work in some type of US tech company (of the "silicon valley" variety even if not in california)
billymg: verisimilitude: sure, though keep in mind i'm just a webshit programmer
billymg: verisimilitude: in any case, it could also be argued it's a matter of "hate the game, don't hate the player". if any group is successful in their kvetching in order to get more gibs then hate the idiot populace that buys it
billymg: verisimilitude: 'kvetching' was what led me to believe that's who you were referencing
billymg: verisimilitude: i won't disagree with you there, and in fact i thought of them when reading alf's statement. though having lived in nyc for as long as i did and with some of my best friends being jewish i'd like to point out that not every jew == 'da jooz'
billymg: asciilifeform: perfect description of current situation in the west
billymg: verisimilitude: doesn't this put us exactly back at current 'strongest cabal gets the goods'. if group Y is stronger they get Z, if not, X holds onto it
billymg: asciilifeform: sounds about right
billymg: verisimilitude: don't follow, i'm equally against welfare in the form of bailouts for banks fwiw
billymg: and it's the Yang brand of socialist who's selling them the "automated luxury space communism" idea. paid for by tax on bezos!!
billymg: verisimilitude: in this case the poor who want to be
billymg: asciilifeform: they oughta have their welfare cut off if you ask me
billymg: i forgot the word 'automated' in there somewhere too
billymg: ai filling in for the god
billymg: asciilifeform: ai-luxury-space-communism i believe is the pipe dream of the socialists these days
billymg: verisimilitude: heh, yeah, i still don't see how we've moved past the 'god' ideal
billymg: if he's buying bread or a house it's for him to eat or live in
billymg: verisimilitude: or do you mean that these labor tokens are regulated in such a way that no man ever has enough surplus tokens to make those kinds of investments
billymg: verisimilitude: i mean rather, i hire baker to bake bread for me, or builder to build house for me. the baker and builder are happy to exchange their labor for my tokens. i'm able to turn around and sell the product of their labor for more than i originally paid
billymg: verisimilitude: what about exchange of tokens now for labor that results in value created (for the purchaser) greater than that of the tokens originally spent for the labor? -- would this be a bad function or a good function?
billymg: what are the bad functions? (assuming good money)
billymg: i would be curious to read actually
billymg: ah, in that sense of ideal
billymg: handling* distribution of resources
billymg: verisimilitude: how do you propose distribution of resources in that society?
billymg: also there's only one way to find out
billymg: asciilifeform: makes sense, i'm not yet convinced this one is well enough crafted
billymg: thimbronion: lulz
billymg: asciilifeform: once upon a time long ago i opened reddit for "news" and entertainment, not everyone who visits reddit makes it their final stop
billymg: asciilifeform: if there's 18 year olds out there now cutting their teeth with this GME shit the smart ones won't stick around to bother playing a second round, and being 18 they got the time and energy to (potentially) accomplish a lot outside the chumpatron
billymg: asciilifeform: for food, the stuff cooked by women at home much better than what you get at most "restaurants" anyway. not sure about the Gb fiber part, i hear it's making it's way to rural US too
billymg: i grew up with naive view that "america is a free country and if you work hard blah blah blah", when i meet younger kids none of them seem to share this delusion
billymg: asciilifeform: expensive in US cities, sure, but now with covid you can move to rural parts of the middle of the country -- live much cheaper, collect same salary, still avoid orcs
billymg: asciilifeform: re: bonsai kitten, this is something i've thought about with regards to the younger generation. millenials and boomers, fully bonsai kittened -- zoomers are so disenfranchised from the earliest moments of their adolescent lives that some might instinctively sidestep bonsai bottle before getting inside
billymg: and little by little more will start to do just that
billymg: asciilifeform: why abuse of the word? if one can actually stop playing in their chumpatrons, and live out the rest of their life that way, it seems like that is escape. and robs them of some amount of value
billymg: but i see your point, and it's definitely true, but there's always a tipping point, no? i mean i know we're in uncharted territory historically, with internet-of-shit tracking devices at 3 per citizen, but still
billymg not sure if typo or misread on alf's part but wrote 'walking up' not 'waking up'
billymg: escape can also be in the sense of escape the fraud currency, into btc and barter systems, even if staying geographically in the zone
billymg: but if it's half the country walking up to pelosi's front door, not sure what they do then
billymg: asciilifeform: yes, that much is obvious, and can easily shoot a handful of 'em
billymg: thimbronion: ty, will check it out
billymg: right, and even any deputized karen in a grocery store yelling and pointing when she sees you without a mask
billymg: asciilifeform: 'militant wing' of empire being antifa in this case?
billymg: asciilifeform: re: no more carrot, only stick -- this just increases likelihood of violence and escape (and in large numbers). and as thimbronion said, violence is the more expensive way of having to shear the sheep. at some point cost of shearing sheep becomes greater than value of the wool
billymg would like to hear more about this red guard, and how it relates to current us situation
billymg: http://logs.nosuchlabs.com/log/asciilifeform/2021-01-28#1030340 << solve the "we can freeze your account when you misbehave" problem, in addition to the one you just pointed out
billymg: thimbronion: yeah, exactly
billymg: thimbronion: should make a lot of these kids finally start to understand the need for btc (what with their "trading" apps disabling trading whenever the house starts to lose)
billymg: thimbronion: it's been a fun distraction these past few days
billymg: http://logs.nosuchlabs.com/log/asciilifeform/2020-12-29#1027714 << thanks for the notice, updated
billymg: ty asciilifeform, and thank you trinque for such a convenient and reliable service, i'll send a withdraw command to deedbot shortly
billymg: !!v 904A7F4F6C742B7CF145E659A4BB4E766D35F631B44880DCCAB26C3B0DA08F4C
billymg: asciilifeform: works for me
billymg: shinohai: lmao "a name chosen by space professionals, for space professionals"
billymg: i figured i'd try to use it to make one final payment then withdraw whatever's left