Glancing at your comment history, I see you're already familiar with the concept of groupthink bubbles being extremely popular with people, and a big business. And you mention a wish to mitigate them. I'd hoped to write something more useful here, but it's very late, and drafts are just not gelling. So merely... I suggest the above comment provides opportunities for such mitigation.
One sidebar caveat re "René Girard observed that human communities in crisis often resolve internal conflict through scapegoating", and "humans will scapegoat at all times", and treating this as "human liturgy", rather than as culture-specific pathology.
Consider a South American fishing village culture, which didn't do concepts of "shit happens", nor "unintended consequences", nor "systems failure". So everything that goes wrong, is the result of some one individual's intent, exercised by direct action or by supernatural whammy. Your neighbor. Your family. Your enemy. Some fish escape the group net? Gather around to figure out who did it. Your plant not growing well? Stub your toe? Pay the witch doctor to tell you who whammied it/you, and to whammy them back. Described as a singularly toxic interpersonal environment.
Now moving subcultures to a safety/learning/challenge culture is so very hard (aviation CRM, etc), perhaps it's a valid approximation to view its absence as fixed. But having just read a comment suggesting unaddressed climate change should be blamed on the science community because reasons, I'm sensitized just now to "our local cultural dysfunction is simply human nature - it's them that's the problem". Even acknowledging bad actors fueling the dysfunction in which they thrive.
If someone wants to dig deeper into the scapegoat principle/process I recommend picking up René Girard's I See Satan Fall Like Lightning, I stopped halfway though it a few years ago and intend to resume it sometime. Also I expected to see the book itself as reference in the blog post and was surprised when I checked the references section.
> "I think one of the most aggravating, restricting facets of lunar surface exploration is the dust and its adherence to everything no matter what kind of material, whether it be skin, suit material, metal, no matter what it be and its restrictive, friction-like action to everything it gets on [...] the simple large-tolerance mechanical devices on the Rover began to show the effect of dust as the EVAs went on. By the middle or the end of the third EVA, simple things like bag locks and the lock which held the pallet on the Rover began not only to malfunction but to not function at all. They effectively froze. We tried to dust them and bang the dust off and clean them, and there was just no way. The effect of dust on mirrors, cameras, and checklists is phenomenal. You have to live with it but you're continually fighting the dust problem both outside and inside the spacecraft. Once you get inside the spacecraft, as much as you dust yourself, you start taking off the suits and you have dust on your hands and your face and you're walking in it. You can be as careful in cleaning up as you want to, but it just sort of inhabits every nook and cranny in the spacecraft and every pore in your skin [...]" Eugene Cernan, Apollo 17 debrief[1]
An interactive microscope of regolith.[2] Like tiny broken glass, hard as rock, and sticking to everything like static-charged packing peanuts.
Fwiw, with its predecessor's Qwen3.5-35B-A3B-Q6_K.gguf, on a laptop's 6 GB VRAM and 32 GB RAM, with default llama.cpp settings, I get 20 t/s generation.
Have you tried running llama.cpp with Unified Memory Access[1] so your iGPU can seamlessly grab some of the RAM? The environment variable is prefixed with CUDA but this is not CUDA specific. It made a pretty significant difference (> 40% tg/s) on my Ryzen 7840U laptop.
Your link seems to be describing a runtime environment variable, it doesn't need a separate build from source. I'm not sure though (1) why this info is in build.md which should be specific to the building process, rather than some separate documentation; and (2) if this really isn't CUDA-specific, why the canonical GGML variable name isn't GGML_ENABLE_UNIFIED_MEMORY , with the _CUDA_ variant treated as a legacy alias. AIUI, both of these should be addressed with pull requests for llama.cpp and/or the ggml library itself.
Hmm. Perhaps there's a niche for a "The Missing Guide to llama.cpp"? Getting started, I did things like wrapping llama-cli in a pty... and only later noticing a --simple-io argument. I wonder if "living documents" are a thing yet, where LLMs keep an eye on repo and fora, and update a doc autonomously.
I hadn't tried that, thanks! I found simply defining GGML_CUDA_ENABLE_UNIFIED_MEMORY, whether 1, 0, or "", was a 10x hit to 2 t/s. Perhaps because the laptop's RAM is already so over-committed there. But with the much smaller 4B Qwen3.5-4B-Q8_0.gguf, it doubled performance from 20 to 40+ t/s! Tnx! (an old Quadro RTX 3000 rather than an iGPU)
That is pretty solid, I have a 2070 with 8GB VRAM and 64GB RAM, but I haven't run too much. I regret not getting a 3090 back when I built this machine.
Nod. Mine was VR dev leftovers. Fwiw, running 6ish prompts in parallel, roughly doubles my aggregate t/s (but requires cooling kludgery). If one's goal is not local, but rather real-time or consistent or transparent or scalable, there's AWS.
Oh, awesome. On my doables list was to try combining text tokens with "scent" embeddings, to give LLMs a higher-dimensional reading experience. In a file listing, larger files might smell "heavy" or "large". Recently modified files "untried" or "freshly disturbed". Files with a history of bugs, "worrisome". Complex files might smell of "be cautious here - fragile". Smelly `ls`.
Or, you might save token sampling telemetry (perplexity, etc) alongside a CoT and result. So when read, it's like a captured performance - this sentence smells "hesitant", that one "confused". Poetry vs prose. Or, a consistentcy checker might add smells of "something's not right here". Or... emojis that emote.
For a dog, that's not merely a lamppost, it's richly-evocotive local history. To a dev long experienced with some codebase, that's not merely a filename, it's that nasty file that bites.
One open question is whether you can find and calibrate embeddings to provide an informative whiff, without badly degrading reasoning. And be cautious of, and suspicious of changes to, a scary file, without becoming too avoidant. Also, salience bias. Also, imagine debugging scent hallucinations.
Activation-rich text - auxiliary non-linguistic embeddings as meta-signals... the random silliness local LLMs encourage.
So many formats, with different tradeoffs around readable/parsable/comments/etc. I wish there was a "universal" converter. With LLM's sometimes used to edit chat traces, I'd like ingestion from md/yaml, not merely a "render from message json".
So .json `[{"role": "user", "content": "Hi"}` <-> .md ` ```json\n[{"role": "user", "content": "Hi"}` <-> above ` ```user\nHi` <-> `# User\nHi` <-> ` ```chatML\n<|user|>\nHi` <-> .html rendered .md, but with elements like <think> and <file> escaped... etc.
Hmm. HTML has always had goals and tradeoffs which are in tension with many uses. XML too. Witness the very many versions of "write this instead, and it becomes HTML" - long and widely used and valued. Perhaps we collectively might have done better, but we didn't. Turns out LLMs also find different formats significantly easier to use for different things.
As a tradeoff example, yesterday I again tripped on the KISS "CDATA doesn't support HEREDOC-like prefix whitespace removal". So does one indent, compromising payloads where leading ws is significant, or not, confusing humans and llms.
Re reinvention and first principles, aside from engineering tradeoffs, it can be hard to understand design spaces and to be aware of related work. I suspect there's a missing literature to support these, but professional organizations have been AWOL, and research funding dysfunctional. And commercial conflicts of interest. And it's hard. But now coding LLMs are messing with "don't reinvent wheels" payoff tables. Perhaps we'll someday be able to be explicit about design space structure and design choice consequences too. And perhaps we're already getting transformatively more flexible around format extension and interoperation. TFA isn't just a new format - it's a github repo which will help teach LLMs how to do progressive execution of fenced code blocks, making the next format which does this potentially easier to create. "Merge in what X does, but <change request>". Yay?
IIUC, non-meme carcinization is something vaguely like "similar tradeoffs pressure towards similar forms in diverse contexts". LLMs might help us more easily understand tradeoffs, implement forms, and manage diversity?
> If you're still looking for a name let me suggest "hyper text".
Perhaps "WWW SPA document"? Using markdown with highly-progressive fenced blocks?
Hypertext (one word, coined 1960s) is quite a broad category. Subcategory "WWW" could fit, as TFA seems WWW-ish. A markdown document format, and progressive rendering of tags and code, seems HTML-like. Though with greater progressiveness - code blocks with streamed execution rather than merely compilation. The progressive JSON callbacks, React, integrated client and server code execution, and server-side rendering, seem closer to WWW SPA than to HTML. Though SPA files often seem more "source" than "document". And the multiple-page "App"-ness of SPA doesn't fit well. SPA seems a better fit than "full-stack". Perhaps some name analogous to "isomorphic javascript"...?
> how powerful and unique this situation is [...] cannot be easily replicated
I've long wished for an HN-like community which combined software folks, sci ed researchers and creators, and teachers - to get tighter loops around "What do you think? / Could you try this?". But it still wouldn't be the power of playtesting.
I wonder how one might offer playtesting as a service?
> I want to target people of all ages and backgrounds
This opens up opportunities for guerrilla usability testing with adults. Though I've little idea what forms of it might fit culturally in Japan. In Cambridge MA, I could easily do street conversation testing.
Which still leaves ethical issues. One participant was distressed, "Disgusting!", and I only figured it out later... a (Powers of Ten zoomed) E. coli micrograph can look like poop.
> The importance of playtesting cannot be understated.
s//overstated/. Very yes. I've been gobsmacked by things which turned up in user testing. Example: a scene opens with a cartoon character reading a sign aloud... and the cognitive load spike (new setting + audio + text) disappears the entire event from memory. Literally: pause for feedback, "You should say what this is up front", hit replay, "Oh yes, this version is much better than the first one".
I think one of the biggest problems is scale. You need someone to analyze how people are experiencing your game/software. I think there are all sorts of ways you can get feedback. Having people screen record might be a good solution.
> s//overstated/
Oops! I will fix that. Thank you!
reply