Hide Idle (>14 d.) Chans
← 2017-03-10 | 2017-03-12 → ↓
ben_vulpes: http://btcbase.org/log/2017-03-11#1624970 << where once you were a crackpot laboring in public obscurity, now you are a luminary of the republic. defections will continue apace until the old empires yield their last secret.
a111: Logged on 2017-03-11 00:16 asciilifeform: if some unknown d00d had not written to me last night, even now i might be doing it
mircea_popescu: wtf is wrong with ext!
mircea_popescu: ben_vulpes there is an ancient observation (toqueville) that slavery is not unbearable to peoples in proportion to its intensity, but in proportion to the velocity of its reduction. he supports it by showing that the germans, more abject slaves in 1700 than any central asian people, found their situation tolerable ; whereas the french, significantly freer ~and becoming freer~ found the uninstantaneous speed of the change INTO
mircea_popescu: LERABLE.
mircea_popescu: i suspect alf exhibits the same issue.
shinohai: I awake to the screeches of `PREEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEET` this morning with coffee. Hail to the Trumpreich.
mircea_popescu: o btw, where was that lulz
mircea_popescu: https://archive.is/vuBRg (rehash of http://btcbase.org/log/2017-02-27#1619007 ; proudly reported in the beobachter )
a111: Logged on 2017-02-27 12:10 mircea_popescu: dude, they fucking gutted them. olympus agreed to pay the usg ~70 billion yen in fines, and install obama's children as an "independent outside monitor". whole corp market cap being you know, 1.3trn or some shit. who the fuck pays 5% of the market cap as a fine already, what is this, Совет Экономической Взаимопомощи ?
mircea_popescu: o hey, one less shithead made in india around ?
mircea_popescu: http://btcbase.org/log/2017-03-11#1625313 << on the basis ~of actual measurements~ your position is indomitable. wtf, HALF A SECOND for ~8~ deep directory structure ?!
a111: Logged on 2017-03-11 04:05 asciilifeform: Framedragger: imho the 'use existing fs' thing is a dead end.
mircea_popescu: contrary to what ANYONE may pretend, ext4 IS NOT A FS!!!! it's a ridiculous toy at best.
mircea_popescu: ie, the reason usgtard is all "oh, random is not really broken" when phuctor came out exactly reduces to "well, it's fucking broken, but you should see the filesystem!"
Framedragger: asciilifeform: mircea_popescu: for completeness, i should state that it may be "workable" (in the sense of slightly less horrible) to just keep a flat dir tree structure, one or two levels deep - if you don't ask fs to list files in dir and just want to access filenames you already know, it's ~okay-ish. but i think i agree that the whole fs idea needs to be dumped, in general
mircea_popescu: bullshit.
mircea_popescu: im not using this thing. how is it better than windows ?
Framedragger: hey, you wanted some fs test, i'm just reporting on levels of shittiness found
Framedragger: it's not, it's not
mircea_popescu: yeah i don't have a problem with you. still monumentally pissed off.
mircea_popescu: to revisit http://btcbase.org/log/2017-03-11#1625097 which is apparently a daily fixture by now :
a111: Logged on 2017-03-11 01:11 mircea_popescu: i can't bring myself to move my piss away from dks, or anyone in his generation's face.
Framedragger: yeah i especially liked the amazing speed of directory deletion
mircea_popescu: NONE OF THEM DID ANYTHING USEFUL. AT ALL. BUNCH OF PRETENTIOUS POINTLESS USELESS IMBECILES NOT WORTH PISSING AWAY FROM!
mircea_popescu: "oh i had a job i went to work through traffic every morning"
mircea_popescu: fuck you and that roadside wench of your mother.
mircea_popescu: Framedragger suppose you store the blocks whole. what is the seek time of one dozen 1mb files dispersed randomly in a 8 deep directory structure, defined as (time when all are in ram) - (time when call was made) ?
mircea_popescu: make say 1mn total an' see
Framedragger: mircea_popescu: to be clear, the way this would work is, there'd still be symlinks at the bottom ends of the dir structure, pointing to blocks (which are stored in a single dir, say)?
mircea_popescu: nah. here's the scheme again :
Framedragger: also, as asciilifeform said, cache can really confuse the hell out of any metrics. e.g., disk cache. so i'd need to probably restart whole box to be sure (yes lol, but i think i should)
mircea_popescu: 1. actual blocks (1mb files) are stored in a directory structure, based on their hash say. this is 8 deep because hey, max filecount in a dir, we want to make a proper system.
mircea_popescu: 2. index to those blocks (say, eg, to find txn, or anything else) is stored in a SEPARATE dir structure, and at the bottom there's simlinks to the block files.
mircea_popescu: so your storage looks like /(blocks, txn, addies, whatever)/abcd/etc/(abcd.dat or abcd.symlink etc)
Framedragger: (you'd need to have a *lot* of blocks to have average num of files per second-to-deepest dir be >= 1; i dont think one needs 8 levels, but i see the point in trying this)
mircea_popescu: Framedragger the point is that we don't want to make any more provedly breaking systems.
mircea_popescu: if the total number of blocks your machine can produce is 2**4096, then your design will also store 2**4096 blocks.
Framedragger: mircea_popescu: sure, but do you expect to reach **10^24** nodes before trb-i?? (http://fd.mkj.lt/stuff/fsgraph2.png)
mircea_popescu: (and if this is untenable, THEN THE DESIGN GETS MODIFIED!!! no fucking "solutions" of shoving shit under carpet and letting mp discover it in 2017 whiole spending however many years eating food we didn't pay for and pretensions to "engineering" and "intellectual lifge" we don't deserve.)
Framedragger: anyway, fair enough, for now
mircea_popescu: Framedragger in any case i don't expect to optimize BEFORE DESIGNING holy shit. talk about early optimizations. this is the measuring stage. you optimize nothing.
Framedragger: suresure.
phf: http://btcbase.org/log/2017-03-11#1625335 << i'm surprised that ext4 is even considered, as opposed to ext2. i believe we even had a thread about it some long time ago
a111: Logged on 2017-03-11 13:51 mircea_popescu: contrary to what ANYONE may pretend, ext4 IS NOT A FS!!!! it's a ridiculous toy at best.
mircea_popescu: phf well he's considering what he's considering, seeing how he's doing the measuring. i was kinda biasing towards ext2 in the previous discussions (which i guess nobody reads or something ?) , but hey, can't impede man's independent manhood!
mircea_popescu: gotta start somewhere anyway.
Framedragger: http://btcbase.org/log/2017-03-09#1623751 i suppose i couldve started with ext2
a111: Logged on 2017-03-09 17:41 mircea_popescu: Framedragger the most pressing matter to my eyes right now is getting ext2/ext4 benchmarked for our specified purpose.
phf: i believe alf even pointed out the obvious "journaling file systems are going to journal"
mircea_popescu: Framedragger no seriously, nothing wrong with it. now we have numbers. they're good to have.
mircea_popescu: phf i don't follow. so what if they are ?
mircea_popescu: there's an item in the specification of journaling that it must not work which i missed or something ?
Framedragger: mircea_popescu: so in your proposed-to-be-tested scheme, there are two separate eight-deep trees? may i ask, why do blocks need their own tree - after all, it's just an int. do you expect block number to overflow an unsigned 32 bit int? because you *really* don't need 8-deep structure for dispersing 2**32 nodes (again: http://fd.mkj.lt/stuff/fsgraph1.png / http://fd.mkj.lt/stuff/fsgraph2.png )
Framedragger: but if it's for 'defensive' benchmarking, sure. just pointing out.
mircea_popescu: well if you're storing them by thyeir hash...
mircea_popescu: it's a hash. you're thinking store by blockheight ?
Framedragger: ah, ah
Framedragger: yeah but i forgot how to bitcoin. i guess blockheight bad idea?
mircea_popescu: can definitely also store by shifted blockheight, 0000/0000 etc. it will still be a thing as large as the other one
Framedragger: yes that's what i meant. which, i dunno, maybe bad assumption of 'only one chain', or sth.
mircea_popescu: (pretending for a second the design is sane, which it isn't -- who the fuck counts by int a set of hashed items omfg)
mircea_popescu: Framedragger only one chain is not bad assumption.
Framedragger: kk. so, ok. only thing is i'm swamped in march, so will have to wait. (if anyone wants c code i wrote so far, ping me)
mircea_popescu: (for shits and giggles, anyone looked at what happens once blockcount overflows ?)
mircea_popescu: Framedragger dun worry about it. lordship coming up for discussion soon anyway, and ima propose you (an' some other fellows), so.
phf: i think the deep value in an exercise like "replace db with a filesystem" is the reduction of moving parts. ext2 is a straightforward inode based tree, with a separate relocation phase, etc. journaling adds the whole overhead (for it's primarily cognitive) of secondary redundancy that you now have to factor into all your considerations
phf: *for me it's
mircea_popescu: no argument there.
mircea_popescu: what i want to hear is, (preferably proof) as to why journaling filesystem can't store files in directories!
mircea_popescu: Framedragger for the sake of argument : "here's link to my work so far published on my blog" is worth a guinea. http://btcbase.org/log/2017-03-11#1625386 is worth twopence.
a111: Logged on 2017-03-11 14:33 Framedragger: kk. so, ok. only thing is i'm swamped in march, so will have to wait. (if anyone wants c code i wrote so far, ping me)
Framedragger: ah yeah, blog. fucking backlog, man
Framedragger slowly chopping log
mircea_popescu: you ever go to school ? what usually happens there's a chick there that's really good pre-puberty. then she starts bleeding, and she skips some classes / homeworks / attentionpaying. and then... she can never catch back up again. because interlocking.
mircea_popescu: dun be the girl left behind!!1
Framedragger: problem is multiple homework/class/job-domains, and the context-switching :) but yeah
mircea_popescu: (and if the foregoing didn't happen where you went to school... you didn't.)
BingoBoingo: !!up grubles
deedbot: grubles voiced for 30 minutes.
asciilifeform: !#s bitcoinfs
a111: 23 results for "bitcoinfs", http://btcbase.org/log-search?q=bitcoinfs
a111: Logged on 2014-10-31 00:21 asciilifeform: as in, 'bitcoinfs.'
asciilifeform: http://btcbase.org/log/2017-03-11#1625333 << as i also described earlier, there are fundamental problems with all 'general purpose' fs, from bitcoin pov. mainly, they 'pessimize' for 'least common case', which happens to be our ~most~ common case : creation of file
a111: Logged on 2017-03-11 13:51 mircea_popescu: http://btcbase.org/log/2017-03-11#1625313 << on the basis ~of actual measurements~ your position is indomitable. wtf, HALF A SECOND for ~8~ deep directory structure ?!
asciilifeform: the other fundamental problem, and the reason why asciilifeform's interest in recycling old fs for trb is ~0, is that imho trb needs LESS dependency on open sores crud, rather than moar
mircea_popescu: we create 6 files /hour
asciilifeform: mircea_popescu: symlink is a file for this purpose
asciilifeform: and you'll be creating 2-3000 per block
mircea_popescu: ah, is it ?
mircea_popescu: myeah. that's no good.
asciilifeform: but for the l0gz, lemme finish: replacing bdb, massive turd, with ext (or reiser, or any) equally or greater mass turd, (and now with linux dependence !!) is not a win.
mircea_popescu: this is a misrepresentation : the turd is in there already.
mircea_popescu: currently, on all nodes.
asciilifeform: bdb - is
mircea_popescu: so there's no "replacing". excising part of the tumour.
mircea_popescu: bdb sits on a fs.
mircea_popescu: currently the dependency is bdb + ext? ; if it becomes ext? it is less not same.
asciilifeform: mno. the current dependency is 'bdb + AN fs'
mircea_popescu: it's more like bdb + ALL fs in this list.
asciilifeform: anyway i described an algo that wasn't retarded and doesn't pull in 10,000 lines of open sores ???. and it's as if this neverhappened, for some reason.
mircea_popescu: becomes no bdb ; this fs. huge improvement.
a111: Logged on 2017-03-10 16:57 asciilifeform: btw i will also put down in the log, one very simple possible algorithm for a 'txidx-fs' :
mircea_popescu: it's not "as if it never happened". it ACTUALLY DID NOT HAPPEN.
mircea_popescu: how am i going to profile your dreams ?
asciilifeform: profiling it right now, actually
asciilifeform: since ben_vulpes never came back with a #
mircea_popescu: well so then. it never happened because you never published. dun dun dun.
asciilifeform: meanwhile, opened the binder of horrors, with trb src, and found some lulz, which is actually what i sat down at the terminal to share:
asciilifeform: http://btc.yt/lxr/satoshi/source/src/main.h?v=makefiles#0709 << these are what shitoshi actually stored with the txid
asciilifeform: notice, they aren't simply block indices. these'd be fixed length, and might even get reasonable performance in ye olde bdb;
asciilifeform: BUT the tard HAD TO, for some reason, make these variably-sized!
asciilifeform: he stores TX-WHERE-SPEND lists!!!
asciilifeform: so neither my 'bitcoinfs', nor mircea_popescu's let's-abuse-ext, will work, until this garbage is excised
mircea_popescu: i'm not even proposing we abuse ext, i just wish to know if it could work. so far, unencouraging.
asciilifeform: (there is no actual reason to cache where-spents! aside from how the tard probably thought 'my db is slow. let's make it store redundant liquishit, that'll help')
asciilifeform: now, there is one nuance ( that i'm not convinced shitoshi adequately dealt with, either , see http://btc.yt/lxr/satoshi/source/src/main.cpp?v=makefiles#0755 for instance )
asciilifeform: namely, that a particular tx can reside in more than one block, if the leading end frays (forks)
asciilifeform: however it is possible to handle this sanely:
deedbot: http://phuctor.nosuchlabs.com/gpgkey/952995C2D088259A83873F4F1C5ABEA57B4E9EBB4D2B41D03C4916989F98B4CF << Recent Phuctorings. - Phuctored: 1405...3289 divides RSA Moduli belonging to '199.182.78.50 (ssh-rsa key from 199.182.78.50 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US MI)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/952995C2D088259A83873F4F1C5ABEA57B4E9EBB4D2B41D03C4916989F98B4CF << Recent Phuctorings. - Phuctored: 1677...4143 divides RSA Moduli belonging to '199.182.78.50 (ssh-rsa key from 199.182.78.50 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US MI)
asciilifeform: there are two possible places for a duplicated tx: an orphaned (nonlongestchain) block, and a snake-tongue (if you will), one of two fork prongs of equal length, on the leading end.
mircea_popescu: it's probably the cause for the whole spend thing, as a sort of unexamined insurance.
asciilifeform: the Right Thing, as i see it, is : if a tx is in orphaned block, it does not belong in the index table
mircea_popescu: anyway, the correct solution has to go along the lines of "nope. only one block."
asciilifeform: if trb is in a state of snake tongue, ALL of the affected tx do not belong in the index table
asciilifeform: (when new block comes in, to lengthen one end or the other of the snake tongue, the antecedent blocks get checked O(N), 'manually')
asciilifeform: eeeevery single fucking time i sit down and assume that shitoshi did the obvious, simple, correct thing, and build a hypothetical on ~that~, i later open the binder and find that -- guess what
asciilifeform: he did something entirely opposite.
asciilifeform: and which created 1,001 new problems that had no business existing.
asciilifeform: 'let's make blocks variable length' 'let's make tx variable length' 'let's make tx INDICES variable and UNBOUNDED length' 'because mother dropped me as a baby'
deedbot: http://qntra.net/2017/03/choo-choo-motherfuckers-trump-train-begins-rolling-over-opposition/ << Qntra - Choo Choo Motherfuckers: Trump Train Begins Rolling Over Opposition
mircea_popescu: choo choo indeed.
asciilifeform: sooo mircea_popescu , to revisit upstack , the entire doublespendpreventer mechanism in trb relies on this nonsense , http://btc.yt/lxr/satoshi/source/src/main.cpp?v=makefiles#0855 << is where it marks spent, and http://btc.yt/lxr/satoshi/source/src/main.cpp?v=makefiles#0847 is the doublespendtrap
asciilifeform: and in turn this relies on storing the spend-tx-array (yes) INSIDE the indices db!! http://btc.yt/lxr/satoshi/source/src/main.h?v=makefiles#0713 <<
mircea_popescu: the insanity of putting such a thing there, as opposed to you know, THE FUCKING WALLET
asciilifeform: which prevents the use of ANY sane (fixed-length keys to fixed-lenght values) db !!
mircea_popescu: pretty much has to go.
asciilifeform: without gutting and replacing the entire logic of trb.
mircea_popescu: (note that at no point the preservation of satoshi's indexing scheme was contemplated)
mircea_popescu: diana_coman ^ i think you might lulz.
asciilifeform: it isn't clear why to preserve anything else, if this is amputated. it is the very example of 'grandfather's pistol' -- any mistake is a) not necessarily detectable with ANY kind of testing b) potential reactor melt
Framedragger: http://btcbase.org/log/2017-03-11#1625413 << i will also remind that http://btcbase.org/log/2017-03-11#1625312 which (i forgot this last night) means that in my view, there's no friggin' way eight-level-deep tree structure can hold transactions. every symlink is a file, and on top of that, with 8 levels, most about every transaction will create multiple additional folders
a111: Logged on 2017-03-11 16:05 asciilifeform: and you'll be creating 2-3000 per block
a111: Logged on 2017-03-11 03:53 Framedragger will check tomorrow if the insane size was from his shitty c. but actually, probably not - in ext3/ext4, a folder is an inode and an inode points to unique data block - minimum size of which is 4k. given an expansive recursive tree, you get what you get.
asciilifeform: (permits double spend)
mircea_popescu: asciilifeform pfff.
mircea_popescu: proper doublespent check, not this bs.
mircea_popescu: it wouldn't surprise me if a corner case were discovered where THIS permits doublepsend
Framedragger: *59M* for 1000 'transactions', *before* symlinks are even saved. just the folders (7-level-deep folders.)
mircea_popescu: as it is, now.
asciilifeform: mircea_popescu: you're talking 'substantial rewrite'.
mircea_popescu: but understand it's not the case "this - works, replacement might not"
asciilifeform: mircea_popescu: could be interesting to test for double spends 'manually' (using specially written proggy that walks the blox)
mircea_popescu: the case is -- this probably does not work.
Framedragger: mircea_popescu: which means that i don't see how it could work even given mystery-amazing fs performance (...), *space-wise*.
mircea_popescu: i've not seen any. would be more interesting to test for the hole manually by fuzzing on testnet
mircea_popescu: afaik very little of that was done early on.
mircea_popescu: Framedragger so basically each index is an extra terrabytwe
mircea_popescu: well i've seen some multi-tb hdds :D
mircea_popescu: (obviously -- not ssds ahahahah)
Framedragger: well. if we do linear extrapolation, bout 14400 terabytes for transaction index given 250mil transactions....
mircea_popescu: think, it's a tree. 1/3 to 1/2 is cut by that
Framedragger: 59M for 1000tx. *before* symlinks are stored. (granted, they'll be small)
Framedragger: you're right, the growth *won't* be linear. but...
mircea_popescu: anyway. try ext2 too sometime ?
Framedragger: guess so.
Framedragger: mircea_popescu: you sure you want benchmarks? /me thinks it's a lost cause
mircea_popescu: the logs are hungry for numeric values.
Framedragger: but good to have concrete reproducible data to back it up, sure.
diana_coman: ahahaha mircea_popescu; it sounds... familiar already
mircea_popescu: Framedragger the mystery of the black box that manages to require 59kb to store 8 x 4byte directories, ie a whooping 99.99% wastage....
mircea_popescu: i guess it's technically just 99.93% huh.
Framedragger: i'm sure it's justified by the other amazing parts of the fs, and when we look at the other parts we'll see glorious code that works
mircea_popescu: diana_coman ^ i thought you might lulz
mircea_popescu: (denouement : eulora work is pretty much EXACTLY the above two things.)
diana_coman: 99.93% lulz basically, quite
mircea_popescu: no i meant the epic "justified by the other amazing parts of the fs, and when we look at the other parts we'll see glorious code that works"
diana_coman: I took that to be very-much-needed-sarcasm ; Framedragger knows best whether it was still innocent hope instead
mircea_popescu: i think it's innocent sarcasm.
Framedragger: yeah the sarcasm is probably not black enough tbh
mircea_popescu: in other lulz, trilema serves the 350kb zuleika story in ~500ms. FILESYSTEM IS ABOUT AS SLOW!
mircea_popescu: no wonder "cloud computing" and all the rest of the ineptorade.
asciilifeform: konsoomer fs is liquishit. esp. when you entirely do NOT fit the profile of 'konsoomer', who sits and creates file and is happy is it takes less time than for the enter key to bounce back up
asciilifeform: and you don't need variably-sized turds, or deletion, or defrag
asciilifeform: or rebalances, etc.
mircea_popescu: so "consumer fs is great if you don't use it". hey, check it out, exactly like the us army, or for that matter cia, apparently.
mircea_popescu: these great papier mache roofs that work fine as roofs until it finally rains, heh.
asciilifeform: the 1 fs i know of that MIGHT come close, is reiser
asciilifeform: but it has own problems
mircea_popescu: no wonder indian diaspora is so fucking happy. gypsy fucking culture through and tyhrough.
asciilifeform: (iirc reiser is what i originally suggested for this experiment)
mircea_popescu: anyway bbl
phf: indian philosophy seems to be "if everything's fucked, just eat, fuck and do your daily puja"
asciilifeform: i thought philosophy was supposed to distinguish man from rat
phf: it does, hence the daily puja part
phf: "you want meaning in your life? lol. here's 50 books of meaningless rituals. that you have to perform daily. the smarter you are the more rituals you get. knock yourself out."
asciilifeform: 'Anti-transaction replay in a hardfork'. just for mircea_popescu apparently.
ben_vulpes: asciilifeform: yes, i move extremely slowly.
ben_vulpes: did you get your number yet?
asciilifeform: nope, and not likely to any time soon, realized that shitoshi left us blocks where you have to PARSE ALL TX TO SPLIT INTO TX motherfucker
asciilifeform: ( i , like complete idiot, wanted to 'let's write a quick c proggy like blkcut' )
asciilifeform: don't wait for me ben_vulpes
mircea_popescu: phf aha. quite spurious failed civilisation(s), that continent
Framedragger: asciilifeform: i wonder how these folks did it https://eprint.iacr.org/2012/584.pdf
Framedragger: maybe not applicable to trb
asciilifeform: Framedragger: if i had to guess -- probably by munging bdb
asciilifeform: or hm, looks like they didn't
asciilifeform: 'On May 13th 2012 we downloaded the full public record of this system in one of its two major forms, which consisted of about 180,000 HTML files. After parsing and processing these files, we built a graph...'
asciilifeform: must've used bci or one of whatever other heathentrons existed in those days.
mircea_popescu: eh bash ftw.
asciilifeform: mircea_popescu: you have a bash script that parses tx ?!
mircea_popescu: no, the 180k html files above
mircea_popescu: betcha it was curl and pipe
Framedragger: https://github.com/behas/bitcoingraph refers to that paper. it has this tool, https://github.com/behas/bitcoingraph/blob/master/scripts/bcgraph-export , for allegedly dumping transactions for given range of blocks. that tool refers to https://github.com/behas/bitcoingraph/blob/master/bitcoingraph/bitcoingraph.py#L128 which has has a shitty python callstack but boils to
asciilifeform: Framedragger: ben_vulpes already wrote a block & tx parser, his is the closest we have right now to a usable one
Framedragger: ah ah right, not bad then
asciilifeform: forget about python, nobody wants to wait for two weeks for the result.
Framedragger: apparently offloads export/computation to neo4j, a hipsta graph db
Framedragger strategically retreats
mircea_popescu: lol does it even work still ?
mircea_popescu: problem with these hipster solutions is they're only good for about half a year.
mircea_popescu: "oh, nobody uses X anymore". dude... if i wanted to accessorize i'd just read cosmo.
trinque: that's why they love docker; automated jars of shit
mircea_popescu: but 6 months later no longer worky.
Framedragger: good for cryopreservation yes
mircea_popescu: "oh it musrt be a configuration problem"
Framedragger: ah trinque up'd himself just to remind me that docker is shit lol
Framedragger: fair point sir
trinque: !~step1 on the road to recovery!
jhvh1: 1. We admitted we were powerless over alcohol—that our lives had become unmanageable.
BingoBoingo: !~step4
jhvh1: 4. Made a searching and fearless moral inventory of ourselves.
Framedragger: currently stealing booze from that inventory. some good old shit in there i tell you
ben_vulpes happy to chime in on docker shit parade
Framedragger: i have some good memories from using bsd jails some years ago. the core concept of isolation (fs, process, network, etc) is not bad. providing integrated interface not a bad thing, either. problem with docker is it doesn't do it in a consistent way, is too bloated, is ~proprietary +/-, and does the abstraction in a way that invites lazy people to be even more lazy and reckless.
ben_vulpes: "the hft guy" wrote what i consider the canonical beating, mirrors my experience quite exactly
ben_vulpes: dude *the* problem?
Framedragger: ah i may have read it, did it mention DBs, too, as in "why the fuck would you put db in there"
ben_vulpes: let us not mention the ever-shifting api, the utter failure of to do MOTHERFUCKING GARBAGE COLLECTION, the miserable and useless state in which it leaves disks
Framedragger: shitloads of shit.
asciilifeform: Framedragger: you will notice that tools which 'invites lazy people to be even more lazy and reckless' proliferate, whereas ones that penalize shaved chimps for being the worthless biowaste that they are, tend to disappear and then have to be excavated from 1km of dirt, or evaporate entirely
Framedragger: (i'll just mention that for persistent data, you are supposed to use bind mounts, not internal docker storage.)
ben_vulpes: the endless pile of "i would like to perform what i consider a basic operation of type X" "here have a shell script of awk and sed"
asciilifeform: naggum was afaik the first to formally write down this observation, in his perl essays
ben_vulpes: Framedragger: "if you care about your data integrity, our tool is not for you"
mircea_popescu: asciilifeform the fundamental problem with "human rights" / "our democracy" ie humanism -- if mothers empower sons to escape fatherly punishment, the incentive on the 2nd sons is to be mothers rather than fathers.
asciilifeform: basic neh
mircea_popescu: yeah, basic element of the pile of "lalalacanthear" aka "left"
mircea_popescu: in unrelated lulz, turkish-dutch relations at historic low over dutch deciding they have the power to forbid turkish foreign minister visit the turkish consulate in holland. because hey, if it works on random pick-up artist dork, why not on turkish foreign minister.
mircea_popescu: after all they're both male, right ?
mircea_popescu: meanwhile the two are economically about the same size, but the dutch army would count as a half regiment of the turkish.
asciilifeform: happy nato friendz
asciilifeform: hey if there can be a 'latvian army' why not a dutch one.
asciilifeform: let'splaypretend.
mircea_popescu: is holland even in nato ?
mircea_popescu: a yeah there they are, 50k contingent. lol.
asciilifeform: since day0 neh
mircea_popescu: halfromania.
mircea_popescu: asciilifeform i never can remember which of the ex-relevant colonial powers are not in which instruments. eu except britain, nato aparently w/o sweden, but with norway, there's something else without norway and so on
mircea_popescu: buncha speshuls.
asciilifeform: brits still technically in eu
asciilifeform: for whoknowshowlong
mircea_popescu: yet another layer of indirection.
mircea_popescu: you know it's going to shit when the cheese starts widening in this manner.
Framedragger just had some cold borscht. << amazing and recommend for those who haven't tried
Framedragger: http://btcbase.org/log/2017-03-11#1625589 << yeah that's true, and i suppose one could do better than encouraging this trend
a111: Logged on 2017-03-11 19:33 asciilifeform: Framedragger: you will notice that tools which 'invites lazy people to be even more lazy and reckless' proliferate, whereas ones that penalize shaved chimps for being the worthless biowaste that they are, tend to disappear and then have to be excavated from 1km of dirt, or evaporate entirely
asciilifeform: in other noose, hey, hey, lbj, my CsI:Tl scintillator diode is here
Framedragger: http://btcbase.org/log/2017-03-11#1625594 << docker containers are *supposed* to be ephemeral. 'volatile containers' in itself is not a worthless concept. don't take it as me advocating for them, just, ftr.
a111: Logged on 2017-03-11 19:34 ben_vulpes: Framedragger: "if you care about your data integrity, our tool is not for you"
ben_vulpes: you say this as though you think i'm unaware
ben_vulpes: i have wrestled this pig
ben_vulpes: i--dirty. it--enjoyed.
ben_vulpes: but that's not all! say one wants to get into a running container where something went wrong to inspect it's state
ben_vulpes: "oh, no, why would you ever want to do that? aren't you a professional opsbro? with centralized logging? containers are ephemeral, you low-dough derp! they crash and go away!"
ben_vulpes: no bitch, you will give me that process that is hosed. i want to diddle it further.
ben_vulpes: silicon-valley developer finger trap. can't go away soon enough.
ben_vulpes: worse than mongo in that it contaminates /everything/.
Framedragger: ben_vulpes: fair enough.
ben_vulpes: "great for development environments" no, no it is not.
ben_vulpes: it's not great for anything.
mircea_popescu: it's great for otherwise idle dorks to dick about with.
mircea_popescu: spinsterism. it's a thing.
Framedragger: i expect folks regard the "crash and go away" as a feature - just restart it bro, etc.
Framedragger: any suggestions for isolation from folks here? just cgroups/chroot?
mircea_popescu: http://btcbase.org/log/2017-03-11#1625537 entirely. ther'es hordes of chicks who are taught to dick about with "google analytics". there's hordes of dicklets who docker. there's a whole population of people who in 1817 would have been making textiles by hand.
a111: Logged on 2017-03-11 17:20 phf: "you want meaning in your life? lol. here's 50 books of meaningless rituals. that you have to perform daily. the smarter you are the more rituals you get. knock yourself out."
mircea_popescu: gyess what ? they still are.
mircea_popescu: Framedragger honestly i know of no better than "get a box"
ben_vulpes: > isolation
ben_vulpes: grow a budget, get some servers, problem?
mircea_popescu: the whole point of the "we'll solve problems with plows dragged by 1024 chickens" is that you then DO NOT tyurn aroundand pretend each chicken is an individual bipedal miniox.
trinque: Framedragger: the fuck is "isolation" ?
ben_vulpes: trinque: shibboleth of ruby where nothing gets installed cleanly
ben_vulpes: this is seen as an inevitability and not a cancer somehow
Framedragger: *sigh* i'll grant it that concepts are leaky and can't apply hammer to all problems etc etc; but you can't pretend that the core concept of isolation is not useful. just isolating fs, process, and network is already useful.
trinque: you may not use the term "isolation" without defining it
trinque: in your terms, go ahead.
ben_vulpes: and then i waste a year and untold piles of shitbucks on docker
mircea_popescu: i am saying the core concept of isolation is not useful. what now.
trinque: yeah but he was supposed to say it!
mircea_popescu: oh i see.
Framedragger: a process by which one restricts the system-provided resources that a particular entity (just "process", for now, say) is exposed to; the effect of which is that the entity functions as if in a stand-alone system dedicated to its purpose
asciilifeform: http://btcbase.org/log/2017-03-11#1625637 << isolation is , at MINIMUM, is when you have a rack of... isolated machines
a111: Logged on 2017-03-11 19:57 Framedragger: any suggestions for isolation from folks here? just cgroups/chroot?
mircea_popescu: nonsense.
mircea_popescu: your fucking x windows read the input of all other windows dude. wtf.
mircea_popescu: tell me now you run without a window manager.
asciilifeform: i did say separate machines neh
trinque: Framedragger: and what of the myriad privilege escalations that have happened right through the cocker daemon
trinque: "oh but in theory"
mircea_popescu: asciilifeform we all did but he's not hearing it!
mircea_popescu: trinque "socialism is not a bad concept just wasn't correctly implemented."
trinque: "no this thing somebody else built defines me and I have the tshirt"
trinque: aha
mircea_popescu: "i did not say this before any implementation failed though -- it was fine while it wasn't obviously failing"
Framedragger: "used docker" therefor "defined by tshirt" q.e.d.
mircea_popescu: Framedragger more like "used docker, liked it, wants to dockerize other things" therefore defined by tshirt.
Framedragger: that said, docker's 'main aim' isn't even isolation for the sake of security, so my weak point is even weaker.
trinque: and it's "main aim" -> NOBODY CANCELED BASH
Framedragger: but rather more to do with deployment, reproducability, ops, etc
mircea_popescu: listen, your position is horrible because you said "cant say x" and then x was immediately said and ... what now ?
trinque: twenty year old kid says in 201x "oh I will solve the problem of system state management on unix"
trinque: oh really
mircea_popescu: well ? you said that without ANYTHING to back it up ? that MAKES YOU exactly defined by tshirt
mircea_popescu: "can't say our democracy is a waste of everyone's time". ooops...
Framedragger: i only see folks denying the concept of isolation, i maintain that it's not altogether a useless concept. disagreement in premises
mircea_popescu: Framedragger actually your statement was "if you affirm X i will immediately prove you wrong".
asciilifeform: mircea_popescu: rerun !
Framedragger: (re. x windows, obviously one can have different ways and capabilities per particular system/process, etc)
asciilifeform: i saw that pisser befoar!!
mircea_popescu: Framedragger the xwindow thing was a simple example of the lowest fucking level of this. what isolation. it never yet happened.
trinque: Framedragger: no, I said it's not a fucking thing on the same machine!
Framedragger: trinque: re. unix, myeah, naivete, i agree
trinque: I told you to define it and you sulked
mircea_popescu: let's try that definition.
Framedragger: well i did, but you disagreed with the definition
Framedragger: don't say i 'sulked' tho.
mircea_popescu: isolation is the metaphysical concept of running software independently on a machine which shares bus access among undefined peripherals and has no way to isolate cpu instruction runs.
Framedragger: that said, fuck *again* i expose myself to arguing for a point i do not heartily believe in. need to reassess my shitty approach.
mircea_popescu: until you have net interface that somehow (how the fuck ?!) doesn't get to read/write to memory, your isolation is soup served in a calender.
mircea_popescu: it may feed you, but it will sure as fuck wet your pants.
mircea_popescu: ~may~ feed ~must~ wet pants.
asciilifeform: Framedragger: 'isolation' in the sense you picture it, would be a useful thing if it actually worked. items like, e.g., vmware, come close, but still no cigar (you can leak bits via cache skulduggery, as discussed every six months or so in the log for past 3+ yrs, and various other ways)
ben_vulpes: omfg colander
trinque: yeah but calendar is hilarious too
mircea_popescu: ah calender is the paper forming thing yeah.
asciilifeform: ben_vulpes: i think he had that one burned in rom
ben_vulpes: sure but it's been going on like this for what three years how
mircea_popescu: colander.
mircea_popescu: asciilifeform aha, apparently i got fixated on it
Framedragger: trinque: asciilifeform: i concede your point lol. (last good one re cache was just a few days ago, too http://btcbase.org/log/2017-03-06#1622509 )
a111: Logged on 2017-03-06 16:26 shinohai: https://arxiv.org/abs/1702.08719 " we extract 96% of an RSA private key from a single trace."
ben_vulpes: has the republic dispelled with the illusion of the multi-user unix machine yet?
asciilifeform: ben_vulpes: no moar so than the 'illusion' of the multi-directory hdd
mircea_popescu: historical unix multiuser machine was purely political construct. like i could let ben_vulpes live in my vacation home, and it's "multi user" now.
trinque: really it's a reaction to poverty, "lala no I really can't hear my neighbor taking a shit"
ben_vulpes: kinda surprised to not see a gf lending metaphor from mircea_popescu
asciilifeform: trinque: reminds me of the multiple legends that abound re the old sov habit of hanging carpets on walls. 'we're eastern orcs, we've been doing this since the mongols' -- nope. false. 'it's because of cold cement in hruschev-era flats' -- possible. 'it was so as not to hear the neighbours fuck' -- most likely.
ben_vulpes: "yes i let you use her but she's still mine no matter what show you two got up to in the bedroom"
trinque: hah there's something there too
asciilifeform: i keep waiting for someone to introduce this fine tradition in usa
trinque: "nothing can have permanent consequence, mother loves me"
asciilifeform: wall carpeting.
mircea_popescu: ben_vulpes that's actually how it works neh ?
mircea_popescu: asciilifeform complicated, consider most of angloworld used wallpaper + wainscotting up until the us took over the civilisation.
mircea_popescu: wallcarpet may well be just advanced wallpaper.
ben_vulpes: mircea_popescu: i thought?
asciilifeform: mircea_popescu: we had wallpaper AND carpets
asciilifeform: was sop
mircea_popescu: that's a little rich.
mircea_popescu: ben_vulpes otherwise it'd be a little lulzy, what, dude stuffs girl in van, rapes her, it's now his through the magic of fuckjuice ?
mircea_popescu: (though i suspect some segments of interwebs actually think it works thus)
ben_vulpes: i don't see any disagreement here
mircea_popescu: then let's create one!!1
ben_vulpes: my popesculator appears to function just fine
Framedragger: (spoiler, wow it's a pos)
Framedragger: (object of study, not article)
ben_vulpes: hah oh you hadn't?
ben_vulpes: yeah the thing approximately works as an uptime monitor for impoverished ustards running 20 side projects on a single aws t1.micro
ben_vulpes: which BY THE WAY
ben_vulpes: bezosnet is a far more capable docker than docker ever will be.
asciilifeform: ben_vulpes: none of these liquishits 'work' per sane people definitions of working
ben_vulpes: asciilifeform: well computers don't work per sane definitions of working
ben_vulpes: what upon them possibly could
asciilifeform: ben_vulpes: dunno about you, i don't buy or keep around boxes that randomly lose bits.
asciilifeform: this is the elementary, 0th thing required of a comp.
ben_vulpes: eh you have a ups in play as well
asciilifeform: if i find that a piece of iron behaves nondeterministically in whatever way (purpose-built rngs aside) -- i throw it out.
ben_vulpes: yank cord, demonstrate deterministic failure plx
asciilifeform: ben_vulpes: you don't have a mains failure alarm ?!
mircea_popescu: an ups is not that expensive wut
ben_vulpes: nor is it a computer!
ben_vulpes: > in whatever way
asciilifeform: ben_vulpes: belt and suspenders, my comp worx 100% deterministically if i were to pull out the ups. disk controller battery.
asciilifeform: (it gets test-drained weekly, also.)
ben_vulpes: fine, fine
asciilifeform: and iirc ben_vulpes works mainly on laptops. what ups does he need.
asciilifeform: his thing has 6-10hr 'ups'.
ben_vulpes: oh don't even get me started on the ways in which the laptops don't behave deterministically
ben_vulpes: i have seen the most amazing kaleidescopic fireworks shows upon opening the machine
ben_vulpes: you know, SOMETIMES
trinque: ben_vulpes: recall my lenovo's RF interference lulz?
ben_vulpes: trinque: oya
asciilifeform: trinque: was this in the logs ?
trinque: proximity to cell phone causes some video card problem that turns off the display, have to switch VTs to get it back
Framedragger: hahahaha
ben_vulpes was too kind to call trinque during business hours, lols not worth angrytrinque
trinque: lol
asciilifeform: trinque: if you can reproduce this on demand, you can make 'nsa airgap receiver', give lulztalk at some shitcon.
ben_vulpes: was guaranteed way to get boy out of house though "hey i'm here" *slamming around*
trinque: can reproduce it every time
asciilifeform: trinque: how close does the pnoje have to sit to the lappy ?
asciilifeform: (presumably it is the called one, ack'ing the call, that sets it off)
mircea_popescu: prolly worth a vlog
trinque: within a few inches of the left side of the bottom case
asciilifeform: those hoses, they want to connect.
Framedragger: !#s https://thehftguy.com/2016/11/14/what-is-docker-and-when-to-use-it/ << on packaging and deployment practices, decent, probably seen by some folks here
trinque: you're not speaking to people who have not done this ~for long periods of their adult life~
trinque: why teh fuck are you still trying to redeem it
trinque: understand the only thing it exists is so some faggot at a web startup can make his makework job even more complicated so he can hide the fact (mostly from himself) ~that he produces nothing, and is nothing~
Framedragger: trinque: not trying to redeem it any longer, at all, simmer down (but you were right.)
Framedragger: (anyway, i never would have recommended to use it in critical production. but looks like it's shit for personal local machine projects, too)
ben_vulpes: critical production as distinct from toy production?
jurov: glad I've never used docker. apparently they succeeded to reimplement oracle-corp-bezzle without backward compat, wd!
ben_vulpes: jurov: you have no idea what you're missing
ben_vulpes: TEH ISOLATION
jurov: yes hear, they insulated the app from pebkac well
ben_vulpes: that's what they sell on, but it turns out that the insulation is measured in briefcases of hundies
Framedragger: @all thanks to this chat i'll now make some urgent recommendations to startup i'm involved with. maybe it's not even gonna be fucked in the ass if moves decisively away. a bit ashamed i had $opinion on $thing-not-researched in the first place.
trinque: oh hey teh forum works
ben_vulpes: heh now i'm imagining a research report which is just a link to log line
ben_vulpes: "URGENT: SR MGMT MUST READ LOGS"
ben_vulpes: "can you give us a tldr, kiddo?" "NO TLDRS ARE FOR FAGGOTS SIX MONTHS STARTS YESTERDAY"
Framedragger: "if you like report, pay bitcoin to this foundation address here -> " hey maybe one day it'll even happen
Framedragger: the buttsex jokes are for reading teabreaks
jurov: Framedragger: experimenting really helps. i tried saltstack lightly, and the communication between master and minions was incredibly unreliable
jurov: made my opinion about devops
Framedragger: :). one decent part of devops is CI, i thought. it's even mentioned as possibility for trb dev http://btcbase.org/log/2016-05-15#1466922
a111: Logged on 2016-05-15 15:56 mod6: what i also want to build is a CI thingy for trb
trinque: I dunno who canceled bash and ssh
ben_vulpes: bashops uber alles
trinque: aha
Framedragger: if shitloads of machines, maybe ansible?
trinque: no! damn it
trinque: lol
ben_vulpes: Framedragger: dude start by writing some tests mebbe
trinque: b-but muh hackernews
Framedragger: (even hft guy doesn't have anything bad to say about ansible)
Framedragger: i agree, start from tests, of course.
trinque: oh good then.
Framedragger: snarky trinque uses ssh and bash for 100 machine setups, too
trinque: the fuck I don't
Framedragger: i guess retort it "call me when you have 100 machines to adminster, kid"
trinque: that's exactly what I do for 350
ben_vulpes: Framedragger: 'tisn't as though hftbro is exhaustive banger-on of every dumb thing, just the ones he's faced with
Framedragger: what can i say, too much HN for me
ben_vulpes: rots the mind
jurov: don't forget the part about proper management of secrets on the master and general resilience of master against attacks from clients/minions
ben_vulpes: time would be better spent curating a tumblr full of naked girls and american muscle
jurov: it is always incredible hairball of code, impossible to audit anyhow
ben_vulpes: time-cost of audit impossible to estimate at least, and if so and security is a concern, why bother.
ben_vulpes: "hey boss, this looks like something written to suck devs into paying a consulting company to fix bugs in their own software, shall we introduce it as a dependency in our stack?"
ben_vulpes: "the /fuck/ are you smoking?"
jurov: "let's buy oracle, they are solid"
ben_vulpes: once the category of sv finger-trap is introduced, every option must demonstrate that it is /not that/.
ben_vulpes: jurov: i suppose that i'm pretty lucky to have never worked under anyone who thought oracle worth including in the matrix
phf: in my experience places that have oracle tend to pay best rates, and paraphrasing http://btcbase.org/log/2016-12-28#1592046 when choosing a zaibatsu to pledge allegiance, choose the one that pays the most
a111: Logged on 2016-12-28 18:31 asciilifeform: 'when choosing astrologer, hire the cheapest'
phf: when you're an astrologer, choose the client that pays the most? hmm, pissed opportunity there
a111: Logged on 2017-01-24 00:58 trinque: gonna DNS at all, might as well do it at the most-fed ministry
trinque: gonna SQL ...
trinque: as I understand it PostgreSQL is more or less "closest thing to oracle without having sold first-born per core"
phf: that's another reason, why i don't for example, "hack my python with vim like a real hacker". give me the filthiest, most feature rich IDE, where i can just push spacebar to get half of my scaffolding, or whatever kids these days
ben_vulpes: last few times i cracked a pyject, elpy and python-mode and friends proved...resistant to providing utility
mircea_popescu: ben_vulpes hey, at least she has the gap!
scriba: Logged on 2017-03-11: [21:20:45] <Framedragger> snarky trinque uses ssh and bash for 100 machine setups, too
mircea_popescu: in other news : if your horse is undisciplined it's not the harness that needs improving. it's the horse.
mircea_popescu: phf not a bad heuristic, except for the part where 30odd yo is kinda too old to work for a salary just like 20something yo is kinda too old to "work in pr" / http://trilema.com/2010/bani-pentru-piariste/
Framedragger: #trilema is mircea_popescu's constant disappointment, then
mircea_popescu: whyssat ?
Framedragger: just a feeling.
phf: seen it
phf: you've been reading logs too long, if
asciilifeform: phf: lol 404 ?
phf: yeah, i know, i broke a deploy
phf: few minutes
scriba: Logged on 2017-03-11: [21:18:59] <Framedragger> :). one decent part of devops is CI, i thought. it's even mentioned as possibility for trb dev http://btcbase.org/log/2016-05-15#1466922
Framedragger: phf: not enough docker
Framedragger: asciilifeform: where thing builds software and runs it tests every time a change is made. i assume you know his and are therefore asking rhetorically tho
asciilifeform: http://log.mkj.lt/trilema/20170311/#873 << lemme guess, a 'bashop' is a bishop who is handy with bash ?
scriba: Logged on 2017-03-11: [21:19:12] <ben_vulpes> bashops uber alles
asciilifeform: Framedragger: in point of fact i did not
asciilifeform: believe it.
Framedragger: guess that's a *good* sign
jurov: asciilifeform: "continuous integrastion" - running tests on every commit
Framedragger: (continuous integration is the keyword)
Framedragger: its tests*
asciilifeform: seems like a shit idea tho. 'oops there isn't a trb running on dulap, because ooops i broke the build'
Framedragger: build environment != production environment
phf: btcbase is back, with some new experimental features..
mircea_popescu: asciilifeform bash ops.
asciilifeform: phf: oh hey mouseovers
mircea_popescu: asciilifeform ci as such is not even a bad idea. compile and run pile of tests frequently, an expansion of "save whenever you stop typing"
Framedragger: i thought mouseovers were already there? if you mean timestamps over nicks
Framedragger: (obviously you need to have written those tests first...)
mircea_popescu: obviously.
Framedragger: not obvious to some devops bros.
asciilifeform: Framedragger, mircea_popescu : i can picture it
asciilifeform: would seem to rely on traditional (push/pull) versionatrons tho
mircea_popescu: you can in principle even have emacs-style "pile of all purpose tests" with refinements etc
mircea_popescu: but from a management perspective, it makes you write < 1k lines code 3x as slow and > 10k lines code 3x as fast.
phf: ok, so experimental features on btcbase is that in memory presser now works. each patch has a "tree" link, that shows you a list of files (ones that are explicitly touched by the patch are marked as "modified"), and can look at each individual file
asciilifeform: oh hey!!
mircea_popescu: asciilifeform rat droppings are never found in singlets or how'd that go.
mircea_popescu: people who magicnumber magicnumber, it's just how they go.
asciilifeform: mircea_popescu: the hilarious bit is that prb munged up the script system beyond recognition, BUT they kept this turd! merely moved it, http://btc.yt/lxr/satoshi/source/src/script/interpreter.cpp?v=0.10.0rc4#0256
mircea_popescu: it's good!
asciilifeform: mircea_popescu: unrelatedly, didja ever calculate what would happen to trb (or for that matter prb) if one were to produce a colliding txid ?
mircea_popescu: did anyone ever calculate anything wrt bitcoin ?
asciilifeform: nobody but mircea_popescu , hence the q
phf: there's also a search function that lets you search through patchset, and it shows ~first appearance~ of particular string with corresponding file/vpatch
mircea_popescu: in principle it should not be possible to collide a tx.
phf: (searching through a particular press doesn't work yet)
phf: err
asciilifeform: mircea_popescu: all you need is an x, y, x!=y, where sha256(sha256(x))==sha256(sha256(y)), and x is a valid tx
mircea_popescu: well yes.
asciilifeform: given that a tx is >256bits long, these pair necessarily exist.
phf: and finally grapher now properly shows broken vpatches (i.e. ones that don't have all of the antecedents), for example http://btcbase.org/patches?patchset=experimental all polarbeard ones, and my phf-shiva-swank are broken
mircea_popescu: not so easy to find though.
asciilifeform: phf: how did those end up existing ?
asciilifeform: mircea_popescu: i did not say that it is easy to find. but would like to know the consequences of one being found.
phf: regrinds
asciilifeform: and the pair -- broadcast
asciilifeform: phf: i suppose the correct term would be 'orphaned' patches
phf: right
asciilifeform: mircea_popescu: same q can be asked re block hashes
mircea_popescu: asciilifeform the consequences would likely be dire, ie, irrecoverable/invisible variant blockchain
asciilifeform: mircea_popescu: that's what i ended up concluding
asciilifeform: which is one of the reasons why shitoshi's nonpositional tx indexing is retarded
asciilifeform: (i.e. by hash of antecedent, rather than by where-it-is-in-blockchain)
mircea_popescu: i don't think at the time he did this he had a very clear idea of the mempool / blockchain tx disjunction.
mircea_popescu: which explains a lot of things includingthe miner/node bug
asciilifeform: ditto blocks, it is retarded for their hash to uniquely identify them for other calculations
mircea_popescu: block/candidateblock. cb should be identified by hash ; block should be identified by height.
asciilifeform: just as i described for 'tx vs larval-tx' disjunction.
mircea_popescu: tis been in the logs a few tiems.
asciilifeform: mircea_popescu: the obvious interesting sequel to the q, is what would be the first symptom of a colliding pair of tx having been fired.
mircea_popescu: variant sized blocks.
asciilifeform: as i currently understand, both x and y would have to be valid tx, for anything to happen (if y is invalid, it will not be mined).
mircea_popescu: but it can still end up in node's idea of blockchain
mircea_popescu: just have to put it in your story of the block
asciilifeform: bifurcation of ~block~ is even more interesting picture
asciilifeform: you then can have forklets ~merge~, topologically, into what appears to be 1 chain
asciilifeform: but with different nodes having entirely different pictures of history, depending on which one they saw first
mircea_popescu: the largest problem being that as difficulty increases, it becomes more and more feasible to simply seek a hash, in comparative terms. considering the disproportion of effect (one gives you a 1/2/2/2/2/2 of 50 btc, the other fucks up the toy entirely) it's a virtual certainty that eventually it will be economically reasonable to divert resources from mining to this hash colliding.
mircea_popescu: this is a fundamental and inescapable problem of usg-style hash functions. stuff like mpfhf mitigate THAT problem, but at a humongous cost.
asciilifeform: there is also the possibility of not doing the idiot growing-string-of-zeroes method for difficulty growth
mircea_popescu: yeah, but that fixes nothing. as long as you use hashing, you have collisions.
asciilifeform: consider, for instance, instead of difficulty zeroes, you simply must match N bytes of ~previous~ block's hash.
asciilifeform: and yes.
mircea_popescu: that'd certainly be the right way to do it. it's not "must start with 5 0s, it's "must start with 5 bytes equal to prev block's"
mircea_popescu: this alone puts a large kaibosh on shitty mining.
asciilifeform: ~exactly~ N, not 'at least'
asciilifeform: but yes
mircea_popescu: at least.
mircea_popescu: 0000000000000000000000000000000000000000000004 block is valid now, too.
asciilifeform: if 'at least', the degenerate case wins, i can qualify a block as the successor to itself
mircea_popescu: for difficulty, yes
mircea_popescu: but not for dblspends.
mircea_popescu: there's more than one qualification.
asciilifeform: how would you disqualify it for dblspends ?
asciilifeform: if predecessor indexing is done by height, rather than hash
mircea_popescu: because block 500 with hash x can't be followed by block 501 with hash x equal to itself because that block 501 would be trying to spend outputs that are already spent.
asciilifeform: lol what is that device
asciilifeform: on the pvc pipe legs
mircea_popescu: a common cage
asciilifeform: looks like american mailbox , or nazi crematorium, hybdir
mircea_popescu: oh. a riding fucker.
asciilifeform: ah in usa for some reason these called 'sybian', after one common brand
mircea_popescu: something like that
asciilifeform: unrelatedly, i have combed trb src for a cap on tx input and output maxima, and found none
asciilifeform: anybody know where ? or are they capped strictly by the block cap
asciilifeform: ( for a certain purpose i need to calculate the max bytefootprint of a tx )
phf: i believe it's not explicitly bound (i combed for limits while writing lisp btc and found none for any of the struct combinations)
phf: everything's limited by how much you can stuff into a block
asciilifeform: the equation i've come up with, is --assuming all scripts occupy their maximal size, 10000 bytes-- : 14+i*(10043)+(10011)*o
asciilifeform: where i is the number of inputs, and o -- of outputs.
asciilifeform: (and given that the scripts can be -- each -- individually anything up to 10000 -- you can have, theoretically, 1MB tx)
asciilifeform: and any other size you want, below that.
asciilifeform: this is pretty monstrous, it means that if you want no-heap (proper adatron) handling of tx, you gotta preallocate 1MB for EVERY SINGLE ONE when reading'em
mircea_popescu: asciilifeform in theory they shouldn't be capped outside of the block size.
asciilifeform: now on the other hand, it may be possible to calculate max tx size IN GIVEN block
asciilifeform: because block header tells us tx count.
asciilifeform: and it is possible to compute a mintx.
mircea_popescu: anyway, proper adatron -> trb-i -> fixed 2/2 txn model.
phf: possibly proper adatron solution would be to allocate a block and use it as operational space too
mircea_popescu: ie alloocate ada-heap ?
asciilifeform: mircea_popescu: if you haven't guessed, i have an incomplete one here (currently calling in my head 'nqb', 'not quite bitcoin') and was trying to adapt it to the very simple task of eating the existing blocks and parsing out the tx.
mircea_popescu: no dice with that.
mircea_popescu: ~profoundly retarded~ has a meaning.
asciilifeform: phf: no, see, you gotta know max size to make array.
asciilifeform: otherwise, as mircea_popescu said, nodice.
asciilifeform: one possible solution is binning
asciilifeform: ('child', 'woman', 'man', 'elephant' say)
asciilifeform: fixed bottles.
asciilifeform: what's the statistical distribution of tx sizes, gotta wonder
asciilifeform: how many 'elephants' ?
asciilifeform: (2+ standard dev, say)
mircea_popescu: 800ish > 50% ; 1600 20% sort of thing
asciilifeform: when didja last measure this mircea_popescu ?
mircea_popescu: > 10kb tx are rare, but not inexistent. and then there's periodic massive ones.
asciilifeform: massive as in, most of a block ?
mircea_popescu: asciilifeform i measure it continuously, this is just distilled impression over years
mircea_popescu: asciilifeform take for instance famous davout txn paying out bitbet.
mircea_popescu: (which i didn't originally even think will get mined)
asciilifeform: how much did that weigh
mircea_popescu: lemme fish it out, pretty sure he published it
asciilifeform: large, but nowhere near block
mircea_popescu: 1 bitcet fee.
mircea_popescu: nah. though a 999kb one existed.
asciilifeform: do you happen to recall in which block?
mircea_popescu: bb41a757f405890fb0f5856228e23b715702d714d59bf2b1feb70d8b2b4e3e08
mircea_popescu: block 364292
asciilifeform: ooh nice
asciilifeform: ty mircea_popescu
mircea_popescu: nofee either. miner skulldulgery.
asciilifeform: pretty obvious spamola
mircea_popescu: it mostly wipes a large script bs.
asciilifeform: from the original spamwave
asciilifeform: (when gavin et al made 'brainwallets' seeded with common dictionary words, by the megatonne)
mircea_popescu: anyway, pruned 550 or so bs uxto
asciilifeform: pruning ain't a thing
mircea_popescu: well, they were unspent before. fucking up the prb dried shit echafaudage
ben_vulpes: phf: cool thanks, will investigate
asciilifeform: (there is no way to demonstrate the destruction of coin in any permanent way, so this tx will remain relevant for so long as bitcoin marches along)
mircea_popescu: anyway, point being, most txn are small but long tail exists.
asciilifeform: however ! looks like we ~do~ have a fixed-size element: the script.
asciilifeform: these can be the fs 'block'.
asciilifeform: (the 10,000 + overhead, rounded to nearest cylinder.)
asciilifeform bbl, meat
mircea_popescu: script needs to be killed / specified.
mircea_popescu: dumbest fucking idea. "oh, instruction field in your money".
mircea_popescu: ahahahaha. so in further "everyone in sweden is just asking to be raped with a toilet plunger", nobel prize for literature -> bob dylan.
asciilifeform: http://btcbase.org/log/2017-03-11#1626033 << noshit! but it can only happen in trbi
a111: Logged on 2017-03-11 23:56 mircea_popescu: script needs to be killed / specified.
asciilifeform: mircea_popescu: i was attempting a nonretarded trb-fs.
Framedragger: asciilifeform: are you doing this to prototype your 32b tx header index idea?
ben_vulpes: > "the Commission believes that the significant markets for bitcoin are unregulated."
mod6: heheh
ben_vulpes: "exchanges that account for the bulk of trading are subject to little regulatory oversight and that existing know-your-customer or identity-verification measures are lax and can be easily bypassed"
ben_vulpes: "because we've made it impractical to trade the underlying, nobody trades the underlying on our wholly-owned venues"
mod6: <+ben_vulpes> bashops uber alles << yeah, CI to mean doesn't mean "must use current devop-isms". we could build our own automated toolset. np.
ben_vulpes: "daily trade volume is only a small volume of total bitcoin mined"
mod6: *to me doesn't mean
ben_vulpes: "in stark contrast to gold, where daily trade volume exceeds total mined by several orders of magnitude"
mircea_popescu: ben_vulpes isn't all teh butthurt delicious though ? "we are very upset that apparently we don't matter ; also we won't mention mp's warning about all this because then it won't burn so much."
mircea_popescu: mod6 right you are. in fact it's pretty much what you['ve been doing.
ben_vulpes: to jump over to mod6's thread for half a second, both he and i have our own suite of automated tests
ben_vulpes: suites*
ben_vulpes: for v i suppose, nevermind.
mircea_popescu: also wtf are they on about, 3600 bitcoin mined a day, 10-15k traded a day (on fiatola outlets), 30-50k total.
mircea_popescu: that's a degree of magnitude.
ben_vulpes: total outstanding, not traded over daily mining vol
mod6: in fantasy land, you can make up reality as you go. np.
ben_vulpes: anyways, having thought about "testing" trb, i am interested to hear what kinds of tests framedragger would write
mircea_popescu: !~ticker
jhvh1: mircea_popescu: Bitfinex BTCUSD ticker | Best bid: 1180.9, Best ask: 1181.0, Bid-ask spread: 0.10000, Last trade: 1181.0, 24 hour volume: 27111.34861041, 24 hour low: 1129.0, 24 hour high: 1196.6, 24 hour vwap: None
mod6: ben_vulpes' is making some super cool v automated tests as well. there is a bit of overlap, but perhaps one day, his will become the defacto-standard tests. mine are a bit brittle to say the least.
ben_vulpes: i went so far as to set up a solipsistic test net before bumping into the dumb-as-rocks "needs 2 nodes in order to mine" shit
ben_vulpes: so before even thinking about CI the testing strategy needs some actual thought, planning, and likely at least one patch. maybe more.
mircea_popescu: overlap is not badf either.
mircea_popescu: but yeah, ben_vulpes has point.
ben_vulpes: one quickly starts thinking thoughts like "verify whole chain for every vpatch"
ben_vulpes: "sync from net for every vpatch"
ben_vulpes: or possibly recently "sync from on-host, known-fast node for every vpatch"
ben_vulpes: probably could be done on a machine costing ~2btc/yr
ben_vulpes: entirely baseless estimate
ben_vulpes: (well, not entirely baseless, i've put a modicum of thought into the topic and have been pricing a new box to boot)
mircea_popescu: prolly should churn the chain as a test yeh. also a tx fuzzer would be great in general.
ben_vulpes: what means "tx fuzzer" to mp?
ben_vulpes: if afl is not a lurking piece of garbage, plugging trb into that might yield some interesting strange.
mircea_popescu: it means you feed a trb to be tested randomly generated "txn"
mircea_popescu: at the very least try see what happens in the obvious cases, doublespend, bad block etc.
ben_vulpes: should even be possible to test reorg behavior!
mircea_popescu: signatures don't match, scripts are bad, inputs are bad, outputs are too long, on it goes
mircea_popescu: prolly but that's finnickyer. unless of course we bother to create 3rd item here, which would be a test chain.
mircea_popescu: to be fed to node to be tested from block 1 onwards, can set own difficulty then.
ben_vulpes: hey, if genesis is programmable, can generate chains for testing on the fly
ben_vulpes: can even diddle trb-observed clock to get difficulty curve to do whatever
ben_vulpes: i hope the point is coming through, though, that 'CI' is utterly irrelevant in the face of the scope of systems choreography demanded and actual tests to be written
mircea_popescu: well, i said it's not even a bad idea back then.
mircea_popescu: "anachronistic message board" ?
mircea_popescu lost interest.
asciilifeform: liquishit that tests to spec is still liquishit.
asciilifeform: per dijkstra's 'tests reveal presence of bugs, but never their absence'
mircea_popescu: this is no kind of argument against having test suite.
asciilifeform: not argument against; but for realistic expectations.
mircea_popescu: there aren't any expectations.
asciilifeform: engineering without expectations is not detectably different from taking a shit and other excretory exertions
asciilifeform: 'oh i'll grunt and -- come what may !'
a111: Logged on 2017-03-08 18:29 mircea_popescu: which is why all the impudent cuntlets want to be "in a creative career" aka hallucinated non-science. if irresponsible activity bereft of verification, ie, the antithesis of science, then their imposture has some space
mircea_popescu: this is nosnense.
mircea_popescu: engineering is specifically the practice of eliminating expectations from human behaviour. it is the hopeless adventure.
asciilifeform: failure is not a defined concept absent expectations.
asciilifeform: bridge holds 100 tanks indefinitely, or 500 for 30 minutes. is an expectation.
mircea_popescu: not so. these are the materials, this is thereby the bridge.
asciilifeform: that's called an african pile of shit, or, charitably, 'junkyard wars' contest. not engineering.
mircea_popescu: (materials, here, means : the river ; the metal ; the tanks)
mircea_popescu: there's no expectation involved. for these tanks on this river out of this metal this is the bridge.
mircea_popescu: there aren't two. this one.
mircea_popescu: there's no expectation involved whatever.
asciilifeform: shifting from a conceptual to a present 'this is the bridge you have, motherfuckers, drive' is dirty thinking.
asciilifeform: at the design stage, there is an expectation, engineer knows in advance that if the thing buckles under any number of tanks that its surface might contain: stalin will shoot him
mircea_popescu: no serious engineer thinks or cares about this
asciilifeform: what does 'serious engineer' even mean, if not 'cares about this'
mircea_popescu: "not cares about this"
mircea_popescu: cares about this -> straight to alchemist corner
asciilifeform: then mircea_popescu's 'serious engineer' includes docker rat
mircea_popescu: no, because docker rat fails to correctly pile up his materials.
asciilifeform: why does he fail, if not from refusing to agree to a verifiable expectation of correct function ?
asciilifeform: fails because did not pray to vishnu correctly ?
mircea_popescu: because he did not pile up all his materials.
asciilifeform screws in brand-new gas mask canister
asciilifeform: mircea_popescu: say what means 'pile up all his materials' plox
mircea_popescu: for the docker guy ? make pile of tanks (what he's engineering for) ; river (what he's engineering against) ; metal (what he's engineering with).
mircea_popescu: people are utterly resistent to doing this, principally because hopemechanism/bad engineers.
mircea_popescu: plenty of examples scattered in the logs, but anyway.
asciilifeform: in what sense does 'pile' in this construction not equal to what other folx mean when they say 'plan' ?
asciilifeform: let's take concrete example,
asciilifeform: asciilifeform, drawing up FUCKGOATS, had to pick what voltage regulator to use, they come in wide variety of max & peak voltage and current ratings
asciilifeform: the choice is constrained by the peak & avg. drains of the other components
asciilifeform: at no point did asciilifeform make a physical 'pile' to determine the correct answer, it was arrived at on paper, with pen
asciilifeform: where is then the 'pile' ?
mircea_popescu: that's a pile alright.
mircea_popescu: the meaning of pile is that they are defined, specified, circumscribed.
asciilifeform: but not 'an expectation.' ? why the word substitution .
mircea_popescu: docker dude dun do that.
mircea_popescu: there's no expectation involved.
asciilifeform: the expectation is that the capacity of the reg is > the current drain. of the floor in a house > the pianos. etc
mircea_popescu: this is no expectation.
mircea_popescu: what interval do you wait ?
asciilifeform: well in the particular example given, failure would have become apparent within a millisecond or so
asciilifeform: and you could smell it.
mircea_popescu: expectation is the equivalent of the continuous tense, you may have expectations out of your household. but no expectations are involved in fuckgoats. if the current is what it was specified it'll work and if not it will burn. what do you ~expect~ ?
asciilifeform: you expect the floor to stay put when the piano is brought into the house.
mircea_popescu: if it will or if it won't, either way.
asciilifeform: and of the gadget, to shit forth rngolade , rather than malodorously combusting
mircea_popescu: expect less live longer.
mircea_popescu: i expect girl to shave. ie, tomorrow. i don't expect anything out of the floor.
asciilifeform: mircea_popescu seems to think that 'expect' means 'unfoundedly expect'
mircea_popescu: no, it means actual expect. something you want and will take time.
mircea_popescu: "we're expecting her to deliver my child in june". that's expect.
asciilifeform: mircea_popescu would not structure phrase, e.g., 'when i aim the rifle and pull trigger i expect to hit the target' ?
mircea_popescu: but i wouldn't say "when i pull trigger i expect round to discharge". because i don't.
asciilifeform: but not 'i draw up a house and expect that the floor will hold my 48 pianos' ?
asciilifeform: how does aiming the rifle differ from drawing up a house, other than time parameter ?
mircea_popescu: cuz im involved in the aiming neh
asciilifeform: i suspect that we've unearthed the point of disagreement here
asciilifeform: mircea_popescu does not picture building the house with own hands
mircea_popescu: why not ?
asciilifeform: or so i read from 'im involved in the aiming'
asciilifeform: presumably you could just as readily be involved in the felling of trees, mixing of cement, whatever else goes into building house.
mircea_popescu: i didn't imagine that's what you mean.
mircea_popescu: yes, if you meant "maybe alf typos the code and doesn't notice" then yes, expect.
mircea_popescu: how this would work -- i've no idea, but technically i suppose it's there.
asciilifeform: y'know, i could write an eliza that 'wins' arguments this way
mircea_popescu: myeah. i expect you could.
asciilifeform: can i use 'i didn't imagine that's what you mean' in the phrase table, or has mircea_popescu trademarked !
mircea_popescu: seriously though, trivial failure mode. compiler wil lcomplain if nothing else.
asciilifeform: butno srsly i have nfi what mircea_popescu meant here.
mircea_popescu: let's insist.
asciilifeform: aite, i'm all ears
asciilifeform: how does aiming the rifle differ, other than timewise, from designing a house that you then build with own hands
mircea_popescu: mp goes hunting. if mp goes hunting without a rifle and bullets, mp has fucked up the tanks part of engineering. if mp goes hunting in unknown martian planet, ends up eaten by hunter-eater, mp has fucked up river part of engineering. if mp goes hunting in zoo, shoots english bobby through overpenetration, mp has fucked up tanks part of engineering. if mp goes hunting in hunting place and hunts quarry but misses his shot, tech
mircea_popescu: nical alf-raised problem, mp still hunter.
mircea_popescu: same exact thing with the docker people -- if theior problem was that they wrote spr1ntf it'd be one thing
mircea_popescu: their problem, however, is that they have nfi what they're doing, like that dog driving van on trilema.
asciilifeform: aaaah we had this thread!!
asciilifeform: !#s матчасть
asciilifeform: ^ what mircea_popescu referred to as 'tanks part'
asciilifeform: we have a w0rd for it
mircea_popescu: well i fucked it up, said tanks twice. first one is metals.
mircea_popescu: but yes, it's not a novel thought by any means.
asciilifeform: literally, 'material part', that is, of the immutable physical pieces of a problem.
asciilifeform: ^ is this funny only to asciilifeform
mircea_popescu: lol no, it's kinda funny
mod6: <+mircea_popescu> signatures don't match, scripts are bad, inputs are bad, outputs are too long, on it goes << yeah, exactly. eventually coverage like this is a must. eventually.
mod6: heheh.
mod6 goes to read about Zuleika
mircea_popescu is curious what peeps think.
danielpbarron likes it so far
asciilifeform: sed -n -e 's/^.*Ping timeout: //p' trilema.log | sort | uniq | cut -c1-3 | tr '\n' ' '
asciilifeform: 240 244 245 246 248 249 250 251 252 255 256 257 258 260 264 265 268 272 273 276
asciilifeform: whywhywhy.
asciilifeform: why never 239
asciilifeform: or 278 !!!
asciilifeform: tell me o wise men.
← 2017-03-10 | 2017-03-12 → ↑
Random(trilema) | Download hourly DB snapshot | Get Source Code