Results 1 ... 111 found in all logged channels for 'f:phf cmucl'
(pest) phf: a non-portable solution (in sbcl/cmucl) is to crank up speed 3 safety 0, type annotate everything and then lisp will generate equivalent code that truncates values to specified types
(pest) phf: http://logs.nosuchlabs.com/log/pest/2023-01-01#1020083 << yeah, i ran into this issue with the luby prototype. cmucl uses XOROSHIRO, sbcl MT19937, python also uses mersenne twister, but whether it corresponds to sbcl's mt remains to be seen
(pest) phf[busybot]: i can't pull from the top of my head how fast cmucl's evaluator is compared to its compiler, but that should be the baseline
(pest) phf[asciilifeform]: asciilifeform, i suspect it's a combination. a mainloop on an ivory is conceptually close to an evaluator on cmucl: where ivory instructions are "VOPs"
(asciilifeform) phf: http://logs.nosuchlabs.com/log/asciilifeform/2022-07-02#1110310 << you can see this contrast between older systems that are closer to their lisp machine roots, like cmucl or acl, that provide internally coherent superset of common lisp, beyond just a handful of helper functions, that's driven by the needs of the hackers rather than "nothing beyond the standard" ideology.
(asciilifeform) phf: (i have this cue file for a cd rip, and i have a cue parser that i wrote for cmucl, which works great, except it wants an external-format '(:latin1 :crlf), which for some reason was cut from sbcl. why? because fuck you)
(asciilifeform) phf: and with cmucl's case was tricky, because you had to make hevea spit out decent html from legacy latex, without touching latex (for philosophical, cut in stone reasons)
(asciilifeform) phf: asciilifeform: i maintained cmucl's documentation with it for a bit
(asciilifeform) phf: e.g. cmucl friendly web server, araneida, is a collection of hacks more so than a complete software package. i would often come acros snippets posted to forums where some problem was worked around, but for whatever reason never upstreamed.
(asciilifeform) phf: this is before i got court marshaled for unstable log due to cmucl issues :>
(asciilifeform) phf: cmucl is also significantly scarred, but stills retains much of "lisp machine nature"
(asciilifeform) phf: until cmucl stopped building on all my systems without some major porting effort i used hemlock as a kind of poor man's lisp machine substrate
(asciilifeform) phf: if could somehow reduce the surface to emacs 19, or cmucl's hemlock, but otherwise you have all this hidden state, that kicks in on triggers, and fucks your life up.
(asciilifeform) phf: asciilifeform: original cmucl version was mmaped, but when porting to sbcl due to frequent crashes had to make the whole thing in memory, which is also major reason why ia haven't expanded it. needs to be made mmaped again, to support multiple channels. otherwise unreasonably memory heavy
(trilema) phf: spyked: http://thetarpit.org/posts/y03/05c-development-log-i.html#selection-147.0-153.252 << i've used araneida for early btcbase, i've dropped it in transition from cmucl, but i'd say that it's the least encumbered of the bunch. i don't remember it having any "administration interfaces". it's written in the late 90s hacky style, and it's missing functionality (for example i wrote entire form parsing logic when i was trying to add patch
(trilema) phf: well, after reading the CADR documentation, source code and generally spending more time on a lisp machine, i realized that modern common lisp is a cargo cult. i'm not sure how naggum didn't see it, but possibly because his lisp was emacs/cmucl/franz
(trilema) phf: http://btcbase.org/log/2018-08-20#1843300 << little known fact: slime's architecture was originally implemented in a similar project for erlang called distel, by the same author luke gorrie. lukego also wrote an emacs clone in erlang and tcp/ip stack in cmucl.
(trilema) phf: i don't know what they did in sbcl that it gives 8.75863727 locally, but with double float cmucl gives 8.758457270000001 and with double-double-float it's 8.75845727w0
(trilema) phf: ben_vulpes: note that older versions of asdf actually work. i've been running 1.369 in my cmucl, and it does require manually updating half of your asdf to remove various later extensions, but it works
(trilema) phf: one solution could be to, e.g. post cl-irc, but hard coded to a specific sbcl implementation. whoever wants to port it elsewhere (like cmucl or whatever) can just post a #-sbcl #+cmucl vpatch, etc.
(trilema) phf: asciilifeform: serve event is a cmucl way of tracking streams of every kind (since no threading, repl mainloop is a reactor pattern). sbcl has just lifted the whole thing, but then never really redid the architecture
(trilema) phf: cmucl fwiw was designed like a lisp machine (though it had damage done to it by modernizers already), where the evaluator + vops was you primary interaction mode, and compilation was a way to evaluate a piece of code to a vop like status
(trilema) phf: asciilifeform: i recall their being tight hashing code, but every time i do the same thing and discover that it was sb-md5 (and there's a corresponding cmucl prior art), so no proper shasum
(trilema) phf: fwiw all our production lisp runs on sbcl, including btcbase. as much as i'm pimping cmucl, it's not "modern" enough to host a website on unix. i still think it's a better target for a hypothetical on the iron common lisp
(trilema) phf: (3s on python, 9s on cmucl, 1.2s on sbcl)
(trilema) phf: i think a good exercise is to rewrite one of the existing c machine subtrates from cmucl or sbcl in ada. but left for another lifetime
(trilema) phf: i also doubt that's what we'll ultimately go to. wire/remote is tied to cmucl, and nobody's going to untie it
(trilema) phf: asn.1 is nothing of the kind, nor is cmucl's wire. it's your typically byte packed binary formats
(trilema) phf: most lisp rpc is done using serialization/deserialization protocols. personally, that is also my preferred method, something like asn.1 or the format cmucl's wire uses
(trilema) phf: cmucl has REMOTE and WIRE packages which let you do rpc between two instances (naturally it's bit rotted in the trunk, and only working version at this point is on my branch). it's a combination of simple "spin up server/connect to server", serializer/deserializer and an rpc convention.
(trilema) phf: http://btcbase.org/log/2017-07-08#1680544 << nope, in fact my "optimized" one i.e. the version where cmucl doesn't complain about unknown types with (speed 3) (safety 0), is about twice as slow as Go version. i'm curious why that would be the case, but haven't had a chance to investigate.
(trilema) phf: sbcl is the original "modern"11! lisp. it gives "style warnings" on my code, but completely ignores type errors in ben_vulpes's code (compiling same code in cmucl gives a lot of warnings about the types being all kinds of confused)
(trilema) phf: nope, recompiled the binary from different file, new name, etc. same result. it's something about how cmucl packs an executable
(trilema) phf: right, i figure as much from reading cmucl's source code (which uses k.)
(trilema) phf: asciilifeform: i wonder how much of scar was removed in sbcl. i know while newman was still hacking, he removed a lot of gunk. i vaguely suspect that cmucl can't be improved without tackling the bootstrapping problem, which is probably going to look very similar to sbcl's solution
(trilema) phf: asciilifeform: there are text files (he makes adequate technical logs of most of his efforts. i'm using his notes to recreate arm port for cmucl), but i won't be able to find them at the moment (i'm not on my main machine)
(trilema) phf: well, that's my favorite one https://www.cons.org/cmucl/doc/crosscompile.html
(trilema) phf: you could maybe put cmucl's interpreter vm into l0, have vops as operations that you need but that you don't have in a vm (simd, floating point)
(trilema) phf: actually it's all a lot more clearer when you read the early cmucl papers. the mess grew in unix user space..
(trilema) phf: so cmucl's compiler interleaves vops, while cmucl's interpreter uses vops as bytecode instructions
(trilema) phf: so cmucl introduced this whole idea of vops (virtual operation)
(trilema) phf: there's some strategy in building multithreading in a lisp that all the commercial lisps share with cmucl and that's different from sbcl, but i'm not quite sure what it is yet
(trilema) phf: re cmucl threads i think that it doesn't always preempt correctly. like it has explicit yield, which you don't always have to call, but it being there implies. also hunchentoot wasn't working right without putting a yield somewhere in the scheduler. it's all very vague, because i've not spend any time looking at it
(trilema) phf: well, you were right, it's the lack of threads. you can't build for cmucl green threads like it's native.
(trilema) phf: cmucl is basically a very straightforward lisp machine port, so there's less unixisms built in. sbcl is modernized for unix, but with corresponding tradeoffs
(trilema) phf: *porting cmucl to raw iron
(trilema) phf: one of the major reasons why porting cmucl might be easier, a lot more naive image construction (doesn't have that whole sb! sb- machinery), and lack of threads
(trilema) phf: so i don't know how sbcl does it, but cmucl has unix.lisp, which kernel relies on early on in the operation, and yes, contains all the "talk to the world" crud. cmucl actually does some hack where only subset of unix is used for operations and then when you (require unix) it adds some of the userspace
(trilema) phf: did anybody do a musl sbcl? i have a musl cmucl in the backlog, but way down the list (perhaps once i get this x60 going)
(trilema) phf: --- /Users/pf/cmucl20d-build/src/hemlock/XKeysymDB fd8e6454cb410b82d1aeabc2b91c1
(trilema) phf: http://btcbase.org/patches/hashes_and_errors#L118 you don't really want to do this. you're subseq'ing there to strip the a/ b/ but that's not at all a guarantee! i have a vpatch with `diff -ib -ruN /Users/pf/cmucl20d-build/src/hemlock/abbrev.lisp src/abbrev.lisp` in it for example. at the very least you want to abstract it away into its own function. that would correctly operate on a hashed-path datastructure.
(trilema) phf: also as a rule you don't really want to let string output streams escape their scope. they don't have standard type (one cmucl it's lisp::string-output-stream for example), so you can't test for it, and for all intents and purposes they act as incomplete builders: you can't do anything with them except get their value, so why not get value there and then?
(trilema) phf: that's the thing, once it starts pushing 3+ hours for setup i'd rather be hacking on cmucl or similar
(trilema) phf: i've been low-key reviving cmucl's hemlock, can't use portable hemlock nor climacs for that matter, because neither have terminal versions. needless to say the process is slow and painful. like right now i'm trying to figure out why scrolling the buffer is slow slow you can see each individual line redrawing. also arrow keys don't work
(trilema) phf: http://btcbase.org/log/2016-10-03#1551544 << sbcl already has an arm port, courtesy of nyef. i am doing the same work for cmucl, but i'm very far from actually getting there. i'm going by armv7 with vfpv3-d16, mostly because that's what ccl does and their code is imho most readable
(trilema) phf: ben_vulpes: actually right now i'm heathen-ing it, by using quicklisp too. problem is btcbase depends on 35 different projects (like hunchentoot pulls 8 deps, cl-irc 2, etc.) so cmucl version i had all 35 hand patched to various degrees. the right way seems to be giving all those project the v treatment: pull them in, strip them of fluff, genesis, etc.
(trilema) phf: (an example is https://gitlab.common-lisp.net/cmucl/cmucl/blob/master/src/compiler/x86/float-sse2.lisp#L1426 x87-set-floating-point-modes prelude with keywords says some handy things about what's being emitted to put it in context, but then (:generator 6 ...) is your regular asm, mov xor ...)
(trilema) phf: but yes, cmucl in this channel started because i was thinking of porting it to hardware. it's obviously what every lisper wants/tries to at some point. in order to achieve it though, i kind of have to read and grok and read again the code the cmucl code, which is what i've been doing..
(trilema) phf: the problem is that then you're stuck redoing many man years of rewriting cmucl/sbcl in them
(trilema) phf: the thing that ascii mentions he'd pay for "cmucl on hardware" explicitly involves ripping out C code and then doing necessary work to support some minimum set of x86 drivers like keyboards etc.
(trilema) phf: sbcl/cmucl already compiles whatever code you ask it to to native, architecture specific assembly, that's not the C layer's role
(trilema) phf: mircea_popescu: that's not what the layer between os and lisp does in the case of sbcl/cmucl. it explicitly doesn't do primitives, because those are, like you said, written in lisp, that's compiled into native bytecode (using VOB's, i.e. chunks for assembly)
(trilema) phf: well, for what it's worth rewriting cmucl vm in asm would be a gnarly but straightforward exercise
(trilema) phf: well cmucl's approach is to hire some locals to carry a palanquin
(trilema) phf: mircea_popescu: so in order to bridge the gap between the two you need knowledge of both patterns. cmucl has knowledge of its own, it doesn't also want to have knowledge of all the shit that goes into gcc
(trilema) phf: mircea_popescu: cmucl's assembly has a different pattern in order to support all the tricky-in-c-trivial-in-lisp code behaviors
(trilema) phf: mircea_popescu: there's machine code in cmucl, the problem is bootstrapping.
(trilema) phf: http://btcbase.org/log/2016-09-17#1543401 << sbcl codebase ~is~ cmucl codebase, so all the same people who wrote above wrote majority of sbcl. newman's work in adding sane bootstrapping is reproducable by doing early diffs and perhaps should come as vpatch on top of cmucl since it's particularly clean. but here's the thing, the way cmucl does bootstrapping is borrowed directly from a lisp machine, and breaks down precisely because the
(trilema) phf: "whole machine" approach was jammed into a c vm layer. but if we were to actually attempt a lisp on hardware hack, cmucl's approach would involve considerably less complexity
(trilema) phf: one of the reasons i went with cmucl originally is because it has known tight memory behavior and sbcl dev equally famous stance that "memory is cheap", which, for the case of keeping log in memory, was discouraging
(trilema) phf: adlai: same code and data that was taking up ~~500mb with cmucl blew up lispworks to 4gb, and now blowing up sbcl to 1gb, which results in heap exhaustion errors, despite the fact that there's still extra heap available
(trilema) phf: apparently whatever i was doing with cmucl no other lisp can handle
(trilema) phf: well, cmucl version was very cmucl specific, when i ported it to lispworks i made it portable, but i'm sure there are some small things left
(trilema) phf: i've used cmucl in the past, like 10 years ago, and still have most of my lisp code for it/in it. than i switched to ccl and deployed few things commercially with it. the only lisp i've never explicitly worked with except toying around is sbcl
(trilema) phf: but the rewrite happened for two reasons, one is that i wanted to do interesting analysis on entire corpus (which is something python will strugle with) and that tmsr canon called for technology ownership. cmucl seems like a reasonable stack to own, considering that it's abandonware. fwiw it benchmarks faster than either sbcl (by fraction) or lispworks (by a huge margin, wtf now) on doing bit twiddling like the search that we do
(trilema) phf: i'm pretty sure cmucl's networking/threading interaction is busted. it shouldn't be, for all practical purposes it's an erlang model (green threads, and async io on vm level), but in practice the code is dodgy..
(trilema) phf: that is fair, but i'm not yet sure if it was cmucl networking stack not keeping up, or digitalocean being saturated. in either case it could've been brought down by increasing the number of attacking bots.
(trilema) phf: i figured, but i suspect it's a much more mundate explanation. nothing's change in gpg/gpgme setup from the previous core, so this must be something novel in the way cmucl is packing itself to a core file, or perhaps it's not reloading those foreign libs properly
(trilema) phf: asciilifeform: fwiw, cmucl has all kinds of nooks and crannies that i don't want to bring up on account of our recurring "why cmucl" conversation
(trilema) phf: asciilifeform: nobody's asking you, this is for people who want a fixer-upper. you don't want to use cmucl either, because it's half baked. it does feel just a tad "i just want to", but i can't necessarily fault you for it
(trilema) phf: log server runs without swap, but it's because cmucl stays within it's heap and nginx memory variance is negligible
(trilema) phf: asciilifeform: yeah, it's the worker exhaustion, but since it's my favorite threadless cmucl, situations under which "no available worker" are numerous and fantastical
(trilema) phf: well, tradeoffs that sbcl made are more in line with modern practices, so there was a transition from old and stodgy cmucl to new and shiny sbcl
(trilema) phf: cmucl doesn't have threads not because it's a baby system. it doesn't have threads because sbcl both made adding functionality of that nature easier and canibalized developers from cmucl
(trilema) phf: you only say that because you think sbcl is somehow "clean". chrodes managed a very neat hack, a build system improvement, that's where neat hacks have ended, and modernization began, but it's not like the legacy cmucl codebase somehow disappeared from sbcl
(trilema) phf: maybe i should see if i can get it working on cmucl and adopt it as my default server, hmmm. it does look hell of overengineered
(trilema) phf: well, that's an equivalent of running cmucl inside x86 browser virtualization
(trilema) phf: i.e. my cmucl runs on a clearly compromised intel cpu and can even do interesting things. imagine future(tm) where only way to run cmucl is by crosscompiling into x86 emulator inside your web browser ;]
(trilema) phf: there's no solution to this kind of shit outside of "i use cmucl with emacs 19 and ilisp" or whatever, which is programmer's equivalent of mp's 2005 thinkpads
(trilema) phf: so either cmucl or i have finally gone mad. (setq *connection* (irc-connect)) few lines later (error "~a" *connection*). error comes back as "NIL"
(trilema) phf: but none is the answer to why cmucl, and i don't think there's one. mostly why not? it's a question of discipline for me, can i move slow and not freak out against deficiencies. i think that "looking for silver bullet" is just a waste of time, and there's no way that i can possibly explore entire cmucl solution space, until a whole new set of base capabilities is required
(trilema) phf: sbcl has arm support over cmucl
(trilema) phf: which cmucl has
(trilema) phf: by day to day i mean non-lisp stuff, but yeah, even taking that thing live revealed a lot of annoyances in cmucl (asciilifeform will say told you so)
(trilema) phf: the web part is fine, but i want a bit more reliability out of a bot, maybe move it into a separate process even. this whole ascii's "does cmucl even has proper threads" is biting me in the ass
(trilema) phf: an alternative to tmux can be "detachtty", which is basically a tmux for non curses processes. it's a bit handier than tmux, since it gives you a pid/ctrl/log for one process only. you can issue batch commands, by doing something like echo '(foo)' | ssh server attachtty --socket=/tmp/cmucl-123.sock
(trilema) phf: jurov: i can often get slightly better performance out of cmucl than out of c on critical paths, so that was purely algorithmic question under the assumption that i can easily get within same ballpark with the right algo
(trilema) phf: i've been mulling over that question with logs. fwiw, entire log can be kept in memory for analysis, annotation, whatever, 180mb as utf-8 byte arrays. with unicode strings takes up twice the memory on 16-bit cmucl, and ~~4 times on 32-bit sbcl. i'm not yet convinced that transcoding everything you get into string and then transcoding it back to a bytearray onto the wire is the best strategy
(trilema) phf: it's a bit down the pipeline for me. there were some attempts to make cmucl go entirely syscall, but as it stands it's probably 40% there. i want to link it to musl first, and then play around with control interfaces uses curses/cl-charm, which is when i'll test out this thing
(trilema) phf: i have an untested uploader, takes a vpatch and a seal, lets through only a valid vpatch/seal combination. i don't think this exhaustively covers what we need though, and is also written for cmucl/araneida which is not giving me best of reliability.. http://btcbase.org/upload
(trilema) phf: gabriel_laddel: i was drunk when answering you last time. i like the idea of a self-contained lisp system with ux built on top of xproto/xlib/zen, because can still attach c program outputs, and stream own output to unix systems proper. i've been building my own tools to be essentially a combination of cmucl repl, that spawn xlib windows on demand. not sure if you know but x is essentially a binary network protocol (described in
(trilema) phf: gabriel_laddel: anything that https://common-lisp.net/project/cmucl/doc/clx/ doesn't answer?
(trilema) phf: it seems like people use cffi as a trivial-ffi compatability layer, which mostly just gives you least common denominator ffi (it was a pain to work with after cmucl's native alien facilities)
(trilema) phf: well, in that respect cmucl is a decent litmus test. every single package that fails to load from quicklisp (named-readtables is the recent offender) turns out to be a turd on a closer inspection
(trilema) phf: also there's less eyes on cmucl, i'm not at all convinced that the politics around sbcl are to my liking, where's cmucl is essentially abandonware. it generates code, it has a nice interpreter, the runtime is smaller, it already has treeshaker, it's got History, etc.
(trilema) phf: the internals are more similar then not, and to fully grok sbcl you have to know cmucl anyway.
(trilema) phf: yeah, i'm leaning that way myself, but i'm not fully convinced yet. i'm for example trying to build sufficient knowledge of cmucl right now, to be able to take full responsibility for the behavior of my deployed system, and it's a ~slow process~. it's handy to have paid professionals assist you
(trilema) phf: asciilifeform: phf: they all have fuckin' ~huge~ binaries so far as i'm concerned << i don't think i was too picky at the time, i.e. cmucl dumps 20mb images, and my main concern was audio latency anyway. oh i also used mlkit for that reason, it uses a combination of frame based memory management and gc, so you can have realtime guarantees.
(trilema) phf: trinque: re lisp rps, you might want to look at cmucl's WIRE and REMOTE packages, https://common-lisp.net/project/cmucl/doc/cmu-user/ipc.html. while you can send sexps over the wire and slime/swank do it by sending readable forms in netstring format, you start running into issues when you need to ipc opaque blobs, like lambdas, hashtables or clos instances. cmucl's ipc solves all those issues, unfortunately married to cmucl. i think it woul
(trilema) phf: i've crashed cmucl a few times, but only when i would reach into heap to access vectors directly. acl and lispworks never crash on me