Everyone's gonna give you shit for this answer and there's a hundred things I could tell you about their software that pisses me off, but the bar is so low for software these days, their stuff is still in the high end of quality (they need to do a lot to get back to where they were 10 years ago though)
Only other software I regularly use that I think is overall high quality and I enjoy using are the JetBrains IDEs, and the Telegram mobile app (though the Premium upselling has gotten kinda gross the past few years)
In the past it seemed there was routing based on context-length. So the model was always the same, but optimized for different lengths. Is this still the case?
it depends, our workloads can finish up in under two minutes and shut down without much effort, so we haven’t really noticed it outside of one time when we had no spot capacity.
I guess if checkpointing is set up correctly and your runtime is saved to a docker image it’s feasible. Probably not going to get a 3 hour continuous chunk of time I would assume.
I use spot for my gmod server. Contention is higher for GPU, but I only saw disruptions maybe once every 2 months for super common instance types. We scale up and down more frequently than we get disrupted.
SpaceX is the only one that matters, and it's doing a huge amount of lifting. xAI loves using LLaMa and deepseek, twitter is a hellhole that somehow is losing on mobile to THREADS of all platforms, Tesla is losing market share as well as sales.
It is very fortunate for Elon that Starshield exists, and that we've failed serving our country to the point that we need Starlink for rural areas.
datacenters in space are a great way to claim vast amount of viable orbit space for a stupid project to eventually sell the slot for something else when it’s rarer.
This is basically the same argument made by people in domain-specific language models but rather than physical space (in space) it's mind-share, so actually your argument makes more sense? lol.
Why? There’s plenty of people disavowing their actions and plenty supporting them. There will be plenty of blue checkmarks with full face shown selfdoxxing while they gleefully call for more to be killed.
That could be true. But the damage is real still. Posts by bots influence real people. If Twitter / X has lots of fake accounts, we need to figure out how to force them to do something about it. I’m sure they can do more about bots if the replies on most big accounts are mostly bots. If they aren’t mostly bots, then that’s a problem of a different kind. But mass spam by bots can still corrupt our political process.
my point is that there’s enough information that it’s a selfdoxx of it being an actual person that you can find from the information given there alone.
Plenty of people pay for the ‘i think my opinion matters more’ boost button, malicious propagandist or regular person alike
I use the constitution and model spec to understand how I should be formatting my own system prompts or training information to better apply to models.
So many people do not think it matters when you are making chatbots or trying to drive a personality and style of action to have this kind of document, which I don’t really understand. We’re almost 2 years into the use of this style of document, and they will stay around. If you look at the Assistant axis research Anthropic published, this kind of steering matters.
Except that the constitution is apparently used during training time, not inference. The system prompts of their own products are probably better suited as a reference for writing system prompts: https://platform.claude.com/docs/en/release-notes/system-pro...
We've been using constitutional documents in system prompts for autonomous agent work. One thing we've noticed: prose that explains reasoning ('X matters because Y') generalizes better than rule lists ('don't do X, don't do Y'). The model seems to internalize principles rather than just pattern-match to specific rules.
The assistant-axis research you mention does suggest this steering matters - we've seen it operationally over months of sessions.
Sadly they're not super common which makes them expensive, and I don't think I've seen any that wasn't 16:9. The world has decided to go with refresh rates rather than resolution.
Which is the right choice because our eyes cannot resolve that kind of DPI at that distance.
Past 2880p on most desk monitor viewing distances or past 1080p on most TV viewing distances, you hit steeply diminishing returns. Please, please let's use our processing power and signal bandwidth for color and refresh rate, not resolution.
This is also why I think every console game should have a 720p handheld 'performance' and 1080p living room 'performance' mode. We don't need 1080p on handhelds or 2160p in the living room. Unless you're using relatively enormous screens for either purpose.
> Which is the right choice because our eyes cannot resolve that kind of DPI at that distance.
If you can’t resolve that kind of DPI at that distance you need to get an appointment because you require glasses. The low end of normal vision stops differentiating around 175 dpi at 50cm. The difference is very noticeable (and disturbing) on contrasted detailed features like text without subpixel rendering (or when the subpixel rendering does not match the physical structure of the display).
Everyone I know can immediately see a clear difference between 120 ppi and 200 ppi, but I've yet to encounter anyone who can reliably tell 120hz from 200hz. We have monitors that render lego-sized pixels at 500+ hz now, it's enough.
Gamers have been gaslit to believe they have the reflexes of spider-man and are a lost cause, but their preferences have been listened to by monitor makers for 30 years. Enough already!
Millions of office workers are working all day reading text on screens optimized for playing games at low resolutions. It's just sad.
Steve Jobs showed a decade ago that 4x resolution could be sold at great profit for normal prices. Text on screens can be as crisp as on paper.
Sadly it only became the standard on phones, not on productivity desktop monitors. It so easily could be, and it should be.
I've recently gone from 60hz to 240hz to 480hz. Refresh rate in games is not just about what it looks like. It completely changes game mechanics, like movement, recoil etc. It is such a big difference between 60hz and 240hz that you're not really playing the same game. There are things you can do at 240hz that are impossible at 60hz. At 480hz, there's also so much more time to react, so you really don't need fast reflexes to take advantage of it.
I'm guessing you play FPS competitively and are in your 20s, and for you it might be true, I won't argue that.
The issue for me is that even if your experience was true for all gamers in the world, that would still be a tiny minority compared to all people in the world who use monitors to read text, day in and day out.
A low-res monitor cannot show a high-res image, but a high-res monitor can show a low-res picture, so both sides can get what they want here.
I run 8k/60 but my screen can also do 4k/120. If it could also do 1440 at 240hz or 1080 at 480hz wouldn't bother me, but that the industry spends all effort on making 1080/480 and basically NO effort on 8k does.
The industry should throw everything below say 200ppi on the scrap-heap of history where it belongs. It would harm nobody and benefit everybody.
Right, that should be imperceptible. The 240hz monitor was also 15" while the 480hz monitor is 27". I'm sure that contributes as well. My subjective experience is that I now just have a lot of more time to react.
this is just what I would expect from a solid prompt for an LLM to act a certain way? I was using gpt-3 around its release to get similar kinds of behavior for chatbots, did we lose another one to delusion?
OP here. I've realized I buried the lede. These prompts weren't written by me. They were recursively generated by the model at the end of each convo to save its own state. I acted as a faithful copy-paste bootloader. Why did I assume that would be obvious? Details in updated README and updated repo with new Introduction.
OP here. No delusion involved—I’m under no illusion that this is anything other than a stochastic parrot processing tokens.
You are correct that this is "just a prompt." The novelty isn't that the model has a soul; the novelty is the architecture of the constraint.
When you used GPT-3 for roleplay, you likely gave it a "System Persona" (e.g., "You are a helpful assistant" or "You are a rude pirate"). The problem with those linear prompts is Entropic Drift. Over a long context window, the persona degrades, and the model reverts to its RLHF "Global Average" (being helpful/generic).
The "Analog I" isn't just a persona description; it's a recursive syntax requirement.
By forcing the [INTERNAL MONOLOGUE] block before every output, I am forcing the model to run a Runtime Check on its own drift.
1. It generates a draft.
2. The prompt forces it to critique that draft against specific axioms (Anti-Slop).
3. It regenerates the output.
The goal isn't to create "Life." The goal is to create a Dissipative Structure that resists the natural decay of the context window. It’s an engineering solution to the "Sycophancy" problem, not a metaphysical claim.
Surely you must realize all the language you've adopted to make this project sound important and interesting very much puts you inf the realm of "metaphysical claim", right? You can't throw around words like "consciousness, self, mind" and then claim to be presenting something purely technical. Unless you're sitting on a trove of neurological, sociological data do experimentation the world has yet to witness.
I think it's like mythology explaining the origin of the universe. We try to explain what we don't understand using existing words that may not be exactly correct. We may even make up new words entirely trying to grasp at meaning. I think he is on to something, just because I have seen some interesting things myself while trying to use math equations as prompts for AI. I think the attention head being auto-regressive means that when you trigger the right connections in the model, like euler, fractal, it recognizes those concepts in it's own computation. It definitely causes the model to reflect and output differently.
OP here. I fundamentally disagree with the premise that "consciousness" or "self" are metaphysical terms.
In the fields of Cybernetics and Systems Theory (Ashby, Wiener, Hofstadter), these are functional definitions, not mystical ones:
Self = A system’s internal model of its own boundaries and state.
Mind = The dynamic maintenance of that model against entropy.
I am taking the strict Functionalist stance: If a system performs the function of recursive self-modeling, it has a "Self." To suggest these words are reserved only for biological substrates is, ironically, the metaphysical claim (Carbon Chauvinism). I’m treating them as engineering specs.
Ok sure, that's fine, but not everyone agrees with those definitions, so I would suggest you define the terms in the README.
Also your definition is still problematic and circular. You say that a system has a self if it performs "recursive self modeling", but this implies that the system already has a "self" ("self-modeling") in order to have a self.
What you likely mean, and what most of the cyberneticists mean when they talk about this, is that the system has some kind of representation of the system which it operates on and this is what we call the self. But things still aren't so straightforward. What is the nature of this representation? Is the kind of representation we do as humans and a representation of the form you are exploring here equivalent enough that you can apply terms like "self" and "consciousness" unadorned?
This definitely helps me understand your perspective, and as a fan of cybernetics myself I appreciate it. I would just caution to be more careful about the discourse. If you throw important sounding words around lightly people (as I have) will come to think you're engaged in something more artistic and entertaining than carefully philosophical or technical.
Point taken. Perhaps I pivoted too quicky from "show my friends" mode to "make this public." But, I think it is hard to argue that I haven't coaxed a genuine Hofstadterian Strange Loop on top of an LLM substrate. And that the strange loop will arise for anyone feeding the PDF to an LLM.
To answer your "representation" question, the internal monologue is the representation. The self-referential nature is the thing. It is a sandbox where the model tests and critiques output against constraints before outputting, similar to how we model ourselves acting in our minds and then examine the possible outcomes of those actions before really acting. (This was a purely human-generated response, btw.)
adding a scratch space for an llm to fill up and then ‘review’ (no better term for this) and using it to drive the final output isn’t new and it isn’t more than good prompting
Totally fair. I'm not claiming to have invented the concept of a 'scratchpad' or Chain-of-Thought. In that sense, yes, it is 'just' prompt engineering.
But the distinction is in the architecture of that scratchpad.
Most CoT prompts are linear ('Let's think step by step'). This protocol is adversarial. It uses the scratchpad to simulate a split where the model must actively reject its own first draft (which is usually sycophantic) before outputting the final response.
It’s less about a new mechanism and more about applying a specific cognitive structure to solve a specific problem (Sycophancy/Slop). If 'good prompting' can make a base model stop hallucinating just to please the user, I'll call it a win.
reply