Hide Idle (>14 d.) Chans
← 2022-06-05 | 2022-06-07 → ↓
busybot[asciilifeform]: Current BTC price in USD: $31492.27
asciilifeform: $ticker btc usd
asciilifeform: hmm no dice?
asciilifeform: shinohai: does one currently have to peer directly with the bot to make a query?
busybot[asciilifeform]: Current BTC price in USD: $31300.11
asciilifeform: ... a hm merely slow?
shinohai[asciilifeform]: Nah it *should* work, might be a bit slow. (Has worked fine testing from u0_a280 station which isn't peered to bot directly).
asciilifeform: $ticker btc usd
busybot[asciilifeform]: Current BTC price in USD: $31235.54
asciilifeform: shinohai: loox like the 1st time there was a buncha chain sync traffic
asciilifeform introduced 'prod' for this, among other, reasons -- no reason wai have to wait for a msg from $peer to stay synced -- but afaik not implemented in blatta just yet
busybot[asciilifeform]: The bot has been up for: 45 days 20 minutes and 54 seconds
shinohai[asciilifeform]: awt so far only bug I have encountered with 9977 is when trying to use "resolve" http://paste.deedbot.org/?id=UA2O
signpost[asciilifeform]: went ahead and farted up 9977 too.
asciilifeform: hmm signpost is deedbot on pestnet already or notyet ?
signpost[asciilifeform]: shinohai: what'd you do btw, separate pest station for bot?
shinohai[asciilifeform]: Yeah I run bot on separate station.
signpost[asciilifeform]: hm, working in direct messages atm.
signpost[asciilifeform]: hm, quite slow it seems
signpost[asciilifeform]: going to let it sit for a while and see whether this persists, and what happens in logs
signpost[asciilifeform]: I've got two pest instances on the same box here.
signpost[asciilifeform]: appears the lag is between those two, not within deedbot
signpost[asciilifeform]: awt: does having multiple local instances of pest break any assumptions of blatta?
billymg: signpost: i've seen significant bot delays with mine sometimes too
billymg: one is on another pest station on the same box as my primary, the other is on ec2
billymg: (with its own pest station also, of course)
billymg: !. uptime
bitbot: billymg: time since my last reconnect : 40d 1h 32m
billymg: not much delay that time, that's the one on the same box
billymg: !c net-summary
crawlerbot[billymg|billymg]: Bitcoin Network (IPv4 Nodes Active Within the Last 48 hours) Global: 7880; TRB-Compatible: 30; TRB: 13
crawlerbot[billymg]: TRB-Compatible by Country: United States: 8; Germany: 4; Romania: 3; Algeria: 3; Russia: 3; Singapore: 2; Canada: 1; Italy: 1; Mexico: 1; France: 1; Sweden: 1; Belgium: 1; Bulgaria: 1;
crawlerbot[billymg]: TRB by Country: United States: 8; Canada: 1; Romania: 1; Lithuania: 1; France: 1; New Zealand: 1;
billymg[asciilifeform]: fast also. if there's a pattern as to when it's slow i haven't figured it out yet
billymg: hmm... that last line not yet showing in the logger (after 2 min)
billymg: and there they both are, so looks like the second line either knocked it loose, or just a coincidence
billymg: but yeah, stuff like that i've noticed before, without any apparent pattern
asciilifeform: billymg: see this hypothesis
bitbot: Logged on 2022-06-06 16:34:51 asciilifeform: shinohai: loox like the 1st time there was a buncha chain sync traffic
asciilifeform: billymg: right nao seems that blatta doesn't 'prod', so you dun know yer outta sync w/ $peer until he speaks
asciilifeform: at which point can trigger a lengthy getmsg sequence
asciilifeform: if you have the debug info log scrolling in a visible shell, becomes apparent
asciilifeform seeing buncha getmsg back'n'forth even nao
asciilifeform: incl. repeatedly for same msg's from shinohai -- suspect that 'smalpest' has a bug where doesn't properly store old msgs
asciilifeform: in principle one ought never to see a req for same old msg from one given $peer
asciilifeform: (unless his hdd burned down etc)
jonsykkel[asciilifeform|billymg]: cant remember how my code works but iirc it retrys getdata evry 10sec on broken chains
shinohai[asciilifeform]: I switched back to blatta on main station yesterday asciilifeform (just fyi)
asciilifeform: then bug obv not in 'smalpest' lol
asciilifeform: wb jonsykkel btw
jonsykkel[asciilifeform|billymg]: wokring on a markdown replacment soon redy but i dont have a name for it so canot release - plz someone invent
billymg: asciilifeform: btw i've got something going with postgres full text search locally, still needs tuning before i can make it live
billymg: it's very fast
billymg: figuring out the results highlighting is what i'm currently working on
asciilifeform: billymg: what's missing from this pov in the orig. highlighter ?
billymg: the original highlighter gives false positive highlights, since it mirrors the behavior of the old search mechanism
asciilifeform: iirc if you were to pass the string b/w the " to it, oughta highlight correctly
billymg: i'll give an example, if i search for "ai machine" (no quotes) it will match 'ai' surrounded by word boundaries and words containing 'machin' (determined to be the stem)
bitbot: (trilema) 2019-05-20 asciilifeform: i suspect that mp_en_viaje is right, and even the 'ai beautifier' thing is redundant, all the machine needs to do is to work like the mirror in brothers grimm, whisper 'you are the fairest of them all' to the sow, will suffice.
billymg: the current highlighter also highlights the 'ai' in 'fairest', even though that was not matched by the search
asciilifeform not looked at reader.py for rather long time
billymg: in the highlighter the modification is easy, i can surround the token in '\b' (python regex word boundary), but the tricky part is getting the list of matches that postgres comes up with from a given stem
billymg: e.g. if you search for "banished" it will match the stem "banish", so all words with that stem match
asciilifeform: billymg: ah hm interesting
asciilifeform unrelatedly, looking fwd to 1y anniversary of 'p2p day'
billymg: i've found the function for going in the other direction, ts_lexize, but not yet found the one that produces the set of variations from a given stem
asciilifeform: billymg: imho ideally oughta have no mutation of search string if in doublequotes
billymg: yes, that's how it behaves
billymg: quoted strings are treated as literals
asciilifeform never liked 'stemming'
billymg: it requires some wrangling, but having the tokenized stem index speeds it up a ton
billymg: ftr my example here had quotes to show the boundaries of the query, not that the term was quoted in the query
bitbot: Logged on 2022-06-06 20:14:21 billymg: e.g. if you search for "banished" it will match the stem "banish", so all words with that stem match
billymg: if you feed it ""real me"" (this time quoted), it finds the actually relevant lines, and very quickly. it's two stage, first does a full text search for 'real' and 'me' then a regex for '\yreal me\y' (surrounded by word boundaries) of those results
asciilifeform: iirc was what asciilifeform suggested last wk
asciilifeform: i.e. pg search (w/ indices etc) on quoted phrase, then filter in proggy for grep-like conformance
billymg: this all happens in postgres, and happens even faster than the ILIKE over the indexed 'payload' field
asciilifeform: given as pg's results are a superset of the desired results
asciilifeform: a hm neato
signpost[asciilifeform]: responds quite quickly in private.
signpost[asciilifeform]: now lagging in private after sending messages here.
signpost[asciilifeform]: smells like b00g
billymg: asciilifeform: with some numbers: http://paste.deedbot.org/?id=W7Rv
billymg: the first two are both "all in pg" queries, with diff strategies. the third is the "wide net in pg, filter in python" method
billymg: the results are from my laptop, not the rk
billymg: on the rk the results are more pronounced, ~193ms vs ~15s for the first two queries
bitbot: Logged on 2022-06-06 19:36:50 signpost[asciilifeform]: awt: does having multiple local instances of pest break any assumptions of blatta?
signpost[asciilifeform]: yep, they're on different ports
signpost[asciilifeform]: messages appear to be trickling into deedbot's instance, but sporadically and slowly
signpost[asciilifeform] can do some deeper diligence soon
signpost[asciilifeform]: ^ saw that very quickly.
awt[asciilifeform|billymg]: signpost: pestbot is fully synced
awt[asciilifeform|billymg]: blatta currently indeed does not support PROD - so there are no retries if your chain is broken and for whatever reason you're missing a GETDATA response. Gotta wait until next BROADCAST or DIRECT command to trigger another GETDATA.
awt[asciilifeform|billymg]: That said I have not yet encountered a case with the past couple of versions in which a sync failed.
awt[asciilifeform|billymg]: You could actually speed up the process by starting off with an up to date message table.
awt[asciilifeform|billymg]: shinohai: resolve is not supported. Appears I neglected to fully remove support for that command.
shinohai[asciilifeform]: ah kk I was going thru code and pulling out all the stops xD
awt[asciilifeform|billymg]: asciilifeform: dupe GETDATA responses area little tricky to debug. You WILL get them since you're requesting missing messages from all peers.
awt[asciilifeform|billymg]: You won't see dupes in the db, but they may be logged multiple times.
awt[asciilifeform|billymg]: Quite possible that there is a timing bug in which a dupe GETDATA response triggers a dupe GETDATA request, but I have yet to see an infinite loop in the released versions.
shinohai[asciilifeform]: https://archive.ph/0CMGZ << lol Costa Rica ,gov rekt AGAIN ?
signpost[asciilifeform]: which hero will use CR's inability to run tech infrastructure as a tax-avoidance defense?
awt[asciilifeform|billymg]: shinohai: this has been ongoing
signpost[asciilifeform]: to be clear, CR's *govt* ineptitude only makes CR the *place* more appealing.
awt[billymg]: signpost: I'm currently unable to pay social security tax so I guess me
shinohai[asciilifeform]: I'm thinking more we should possibly join force and try to hold CR for ransom.
signpost[asciilifeform]: get in line I guess.
signpost[asciilifeform]: shinohai: I'm still waiting for russia's "give us 1mil BTC or we'll launch your own nukes at your head"
asciilifeform: http://logs.bitdash.io/pest/2022-06-06#1006477 << hm; prolly oughta amend spec, but thinking , possibly oughta ask orig. sender of the 'orphan' for the missing msg, and only if times out ask erryone else
bitbot: Logged on 2022-06-06 21:11:09 awt[asciilifeform|billymg]: asciilifeform: dupe GETDATA responses area little tricky to debug. You WILL get them since you're requesting missing messages from all peers.
asciilifeform: there's no particularly good reason to immediately ask entire peerage
bitbot: Logged on 2022-06-06 20:37:31 billymg: asciilifeform: with some numbers: http://paste.deedbot.org/?id=W7Rv
asciilifeform: http://logs.bitdash.io/pest/2022-06-06#1006462 << 15s is squarely in 'is it even up?' category for most folx...
bitbot: Logged on 2022-06-06 20:42:26 billymg: on the rk the results are more pronounced, ~193ms vs ~15s for the first two queries
← 2022-06-05 | 2022-06-07 → ↑
Random(pest) | Download hourly DB snapshot | Get Source Code