billymg: http://logs.bitdash.io/pest/2021-12-28#1001402 << yeah, the rk, while great for a blog and even the crawler, doesn't have enough ram to get the full (all chans) log history into memory. so if there's a long query going on while the bot tries to write a new line the connection sometimes times out
bitbot: Logged on 2021-12-28 20:36:13 signpost[billymg]: db connection issue?
billymg: i was going to attempt a fix but put it on the back burner thinking i'd have a faster box soon anyway, not anticipating the global collapse issues would affect the wait time
bitbot: (asciilifeform) 2021-12-02 asciilifeform: billymg: waiting on crate of ram, sadly (atrocious delays in shipments)
asciilifeform[billymg]: billymg: oddly enuff, asciilifeform ran his logger on identical rk for 8+mo, and db never fell
signpost[billymg]: billymg: needing all log lines in memory for a search seems weird
billymg: signpost: i was just speculating on a potential solution, since the disk on the rk is slow and could be causing some of the long queries that would cause another connection to timeout. see here when i first brought up the issue
bitbot: (asciilifeform) 2021-09-17 asciilifeform: billymg: i strongly suspect your proggy is closing it (on, e.g. timeouts or whatever externally-induced eggog) and not reopening
billymg: and yeah, as asciilifeform points out, might not be the logotron but actually the crawler that is generating some lengthy queries
billymg: but can try for yourself on my logger's www, the "search all chans" can take a while
signpost[billymg]: what db does it sit atop?
billymg: postgres 10.3
signpost[billymg]: hmm. happy to help eyeball queries if you want.
signpost[billymg]: (no rush, just offering)
signpost[billymg]: iirc in postgresql writes should only stack behind writes; reads should never block unless locks are being used.
signpost[billymg] has had plenty of cases where an `explain analyze` on a query yielded a multi order of magnitude speedup
billymg: signpost: sure, all the code for the crawler is here: http://billymg.com/bitdash-crawler-vtree/
billymg: and the logotron is linked there as well, basically asciilifeform's
billymg: the crawler will, like the bot, sometimes also need to be restarted when its postgres connection times out and closes
billymg: not 100% consistent, sometimes bot loses, but crawler keeps going, sometimes the opposite
billymg: i didn't spend more time debugging because i figured i could just throw more hardware at it. i've been working on some other things, including a new blog theme that's almost ready to publish, and a "v2" of the crawler www, which is probably a bit further out
signpost[billymg]: curious that postgresql is timing out.
signpost[billymg]: billymg: btw got that gpggram from me?
billymg: signpost: with the peering info?
signpost[billymg]: yar
billymg: yup, i tried peering yesterday but my blatta version is out of date (8885) and i'm not at a box where i can press 8883
signpost[billymg]: ah right, sorry, saw that in logs.
billymg: s/888/998
signpost[billymg] had a gas leak in house last night, groggy
signpost[billymg]: will be putting a proper detector in the attic now.
signpost[billymg]: and ground floor
billymg: signpost: in the new NH fort?
signpost[billymg]: haven't bought up there yet, real estate market sucks in winter. this is the tx house.
billymg: ah, gotcha
deedbot[billymg]: msg from deedbot.
signpost[billymg]: !!help
signpost[billymg]: anyone else able to see that?
signpost[billymg]: neato, it's in the logs.
signpost[billymg] agrees with asciilifeform that the wot ought to be reimplemented as part of pest-tronics, but til then, deedbot will be accessible here too.
asciilifeform[billymg]: hm not seeing it in weechat list
asciilifeform[billymg]: ( or hm, awt , does that currently show only peers ? )
asciilifeform[billymg]: !!help
asciilifeform[billymg]: noworky
signpost[billymg]: hm, weird.
asciilifeform[billymg]: defo saw broadcasts from it earlier
asciilifeform[billymg]: e.g. deedbot[signpost|awt] | msg from deedbot.
signpost[billymg]: !!help
signpost[billymg]: interesting, my first call to help had output at least for me. looks like I'll have to massage it moar.
signpost[billymg]: !!help
signpost[billymg]: seems like content of the NAMES list matters to it, and yeah, only direct peers appear in NAMES
signpost[billymg]: (in blatta)
signpost[billymg]: this seems correct, since hearsay oughtn't fill up a table of nicks.
asciilifeform[billymg]: it's annoying to not be able to tabcomplete a reply to a hearsay tho
asciilifeform[billymg]: imho oughta be at least a configable knob
asciilifeform[billymg]: (where, if enabled, erryone who spoke within N hours ago shows up in nicklist)
signpost[billymg]: yeah, in reconsidering it, peers either spam you or don't, no need to avoid another line or so to populate nicks. expiration definitely needed.
awt[billymg]: asciilifeform: only peers