SQLite on ZFS needs the Fsync behaviour to be off, otherwise SQLite will randomly hang the application as the fsync will wait for the txg to commit. This can take a minute or two, in my experience.
Btw this concern also applies to other databases, although probably it manifests in the worst way in SQLite. Essentially, you’re doing a WAL over the file systems’ own WAL-like recovery mechanism.
This is a bug in zfs or in sqlite, sync=disabled should never cause actual corruption (it should at worst make existing corruption bugs in sqlite more likely & cause loss of committed sqlite transactions)
I highly doubt it's an SQLite bug, considering how thoroughly they test their code to behave correctly as long as their assumptions are filled. And those assumptions are clearly violated when SQLite runs on ZFS with sync=disabled (since writes may not be written to disk despite fsync).
ZFS isn’t viable for SQLite unless you turn off fsync’s in ZFS, because otherwise you will have the same experience I had for years; SQLite may randomly hang for up to a few minutes with no visible cause, if there isn’t sufficient write txg’s to fill up in the background. If your app depends on SQLite, it’ll randomly die.
Btrfs is a better choice for sqlite, haven’t seen that issue there.
The latest comment seems to be a nice summary of the root cause, with earlier in the thread pointing to ftruncate instead of fsync being a trigger:
>amotin
>I see. So ZFS tries to drop some data from pagecache, but there seems to be some dirty pages, which are held by ZFS till them either written into ZIL, or to disk at the end of TXG. And if those dirty page writes were asynchronous, it seems there is nothing that would nudge ZFS to actually do something about it earlier than zfs_txg_timeout. Somewhat similar problem was recently spotted on FreeBSD after #17445, which is why newer version of the code in #17533 does not keep references on asynchronously written pages.
You know what's even easier than doing that? Neglecting to do it or meaning to do it then getting pulled in to some meeting (or other important distraction) and then imagining you did it.
> Neglecting to do it or meaning to do it then getting pulled in to some meeting (or other important distraction) and then imagining you did it.
If your job is to make sure your file system and your database—SQLite, Pg, My/MariDB, etc—are tuned together, and you don't tune it, then you should be called into a meeting. Or at least the no-fault RCA should bring up remediation methods to make sure it's part of the SOP so that it won't happen again.
The alternative the GP suggests is using Btrfs, which I find even more irresponsible than your non-tuning situation. (Heck, if someone on my sysadmin team suggested we start using Btrfs for anything I would think they were going senile.)
In any interesting tech position, "your job" is a myriad of things. And if everything is a priority, nothing is a priority. They ask us to automate our work so that "someone else on the team could do it during an outage at 3 am" for a reason. So my point is that moving away from the exotic and towards the commodity is an IT imperative.
Facebook is apparently using it at scale, which surprised me. Though that’s not necessarily an endorsement, and who knows what their kernel patcheset looks like.
This isn't an inherent property of ZFS at all. I have made heavy use of SQLite for years (on illumos systems) without ever hitting this, and I would never counsel anybody to disable sync writes: it absolutely can lead to data loss under some conditions and is not safe to do unless you understand what it means.
What you're describing sounds like a bug specific to whichever OS you're using that has a port of ZFS.
The output power of a rocket is different than a blow torch. The sound pressure alone is near or above atmospheric pressure. That concrete is being exposed to vacuum and more than 1atm pressure fairly rapidly. The heat on top of that and if the concrete has any moisture, the water inside is going to rapidly sublimate and condense several times a second. That is on top of the concrete being slammed into by an entire atmosphere of pressure in the meanwhile.
There is a reason that NASA dumps a few million liters of water under their rockets.
The spin axis isn't a valuable reference since it depends on your frame of reference, which is a concept that gets rather ambiguous as you get close to a black hole anyway.
Plus you can just "rotate" a black hole to get it to have the same spin axis as another black hole. You can't "rotate" or "translate" a black hole in space to make the other three numbers change. Those require ingesting matter or emitting hawking radiation and that is the only thing that changes those properties.
I'm figuring that the low-rent planets, stars, etc. in the vague vicinity of an actual black hole would provide a fine frame of reference.
> Plus you can just "rotate" a black hole to get it to have the same spin axis as...
Quip: If you have the tech & budget to meaningfully rotate a spinning black hole, then you've got the tech & budget to change the other parameters, too.
There is a cheap and even for current humanity achievable way to rotate a black hole; change your frame of reference. That is not only true for angular momentum but linear momentum and position. Those are entirely dependent on the observer and their frame of reference. Spin, magnetic charge and mass are not.
Two black holes who differ only by their position, linear and/or angular momentum but are equal in all other parameters are not distinguishable from simply seeing the same black hole twice from a different perspective.
Two black holes who differ in any of the three properties of mass, spin or magnetic charge are distinguishable by those properties (but even that is arguable to some extend).
edit: The rent prices of a planet don't matter since frame of reference is an actual term here, there is no frame of reference more valid than any other for determining the linear or angular velocity or the position of a black hole.
The quantity a = cJ/GM^2, the spin parameter of the Kerr metric, is not observer-dependent. No observer (or system of coordinates) can turn an axisymmetric system into a spherically symmetric one.
Also,
> Two black holes who differ only by their ... angular momentum ... are not distinguishable
is contradicted by
> Two black holes who differ in ... spin ... are distinguishable
(Spin) Angular momentum is J in my first line above.
Spin and Angular momentum are two very different things. Angular momentum measures the velocity of a black hole in, eg, an orbit. Spin measures the rotation of a black hole against it's rest frame.
In a Kerr black hole spacetime there is no orbital angular momentum (OAM, L), only the intrinsic spin angular momentum (SAM, S). One can get OAM in a relativistic n-body black hole problem. Each of those black holes will have its own SAM.
See §5.11. ANGULAR MOMENTUM in Misner, Thorne & Wheeler (MTW) and in particular Box 5.6 D (Intrinsic Angular Momentum) and E (Decomposition of Angular Momentum into Intrinsic and Orbital Parts). The latter gives J = L + S. Admittedly, in MTW §33 the authors prefer to use S (e.g. eqn. 33.4) but that raises the important caveat for binary black hole (BBH) mergers at the end of Box 33.4 (I)(A)(4), which refers back to Box 5.6). Newer textbooks and other sources (including Wikipedia [1]) prefer J, although commonly it gets called spin angular momentum (as in [1] and my earlier comment). Carroll's textbook calls it the Komar angular momentum (near eqn. 6.73, referring to eqn. 6.48) and "spin (angular momentum)" (above eqn. 6.47). This is the sort of thing that annoys mathematicians and non-relativist physicists about relativists; confusion is completely understandable.
A binary black hole (BBH) is not a Kerr solution. No exact analytical solutions to the Einstein Field Equations for a BBH have been found, only approximations and numerical solutions (see the comprehensive review by Baker et al. at <https://arxiv.org/abs/1010.5260> and \vec{L} therein, notably at section D(2) in the second column on PDF p. 26, "In a related phenomenon, the direction of the total angular momentum (\vec{L} + \vec{S}_1 + \vec{S}_2) may change.").
No change of coordinates can turn a BBH into a Kerr solution; the former radiates gravitational waves (if there is no incoming gravitational radiation), the latter doesn't.
(Another way of distinguishing is in the algebraic symmetries of the Weyl curvature tensor. Kerr is a Petrov type D spacetime, BBH spacetimes are generically type I up to some degeneracy measure.)
Finally, I can tie this in to neutron stars: the (exterior) Hartle-Thorne metric is an approximation of the Kerr metric useful for relativistic stars without horizons (neutron stars, white dwarfs) and without regard to interior differentiation. Its usual write-down uses J, but sometimes S, and sometimes both (e.g. <https://arxiv.org/abs/1507.04264>, where at the top of p. 2 the authors give J = GS/c^3).
You're still referring to two distinct properties with Angular Moment being one and Spin being the other. Because a black hole can in fact orbit things and that would give it angular momentum. The spin value is merely how fast it rotates around an axis (which you can define but that's just an observational data point) and is unrelated to the external movement in spacetime.
Maybe uutils could have a build feature that specifically turns off the prefix matching and will break stuff but allows using newer and more useful flags in exchange. I've VERY rarely seen prefix-matched flags being used so I'd wager a distro could be fine deploying it that way.
Ie, setup the features "gnu-compatible-opt-matching" and ship it by default, then gate the extra features behind not turning on that feature.
It's a good idea to make the prefix matching optional. I think it might be confusing to gate other features behind it though. I guess we'll get to this once we find flags that are important enough. So far, we haven't really had significant issues with this; compatibility remains our primary focus for now.
The pilots take off when they take off. If their take-off clearance isn't obviously time limited, they are certainly allowed to take their time. If they rush the take-off they might skip crucial items from their checklists in the haste, we learned that lesson the hard way already. Or what would have happened had the southwest jet had an issue on take off and had to reject? There is a number of reasons to reject a take off before decision speed that will take the plane down the rest of the runway on full brakes.
AI is entirely deterministic. ChatGPT and StableDiffusion and friends are fed endless amounts of random seeds along with every input to keep them from just saying always the same thing.
"Deterministic" only insofar as it's repeatable, but their behaviour is not predictable. If the behaviour is not predictable, how can it be validated as being correct?
If for a given set of inputs there is a deterministic output, then the overall behaviour to a series of inputs is just as deterministic and predictable.
I'm not sure what you mean with "is not predictable" when you also admit that it's repeatable.
Predictable as in, can say in advance what it will do.
Repeatable means you get the same output a second time, given the same input. Predictable means you can say in advance what it will do, and can then check the output against your prediction.
If you can't predict the outcome, you can't validate the process, and can't guarantee its performance.
I think the point is that reality is essentially a chaotic system. That is, yes, you can repeat a failure from an input once you've seen it, but the search space is too big to enumerate beforehand.
That entirely depends on what exactly you implement here, it's entirely possible to implement an AI with continuous & linear properties, meaning that you can extrapolate it's behaviour between a set of inputs with decent accuracy and it won't suddenly change it's behaviour between continuous & linear inputs.
But AI isn't different than existing software systems either. Both will take an input from reality and take actions upon it.
That's useless in this case. You need to be able to prove that it will work with all inputs, and there are too many combinations of inputs to exhaustively enumerate.
There is no way to determine that a non-trivial neural network won't drastically diverge in output due to small changes in input (eg one pixel attacks on image classifiers). This is true for all current models I know of.
Almost all neural network implementations have continuous outputs (ie the nodes in the output layer produce a value between 0 and 1). That doesn't change the above issue at all.
This is much less of an issue with traditional methods
A generative AI may well be deterministic and generate repeatable output for a given inpiut, but that doesn't mean the output is correct for every input. It may merely generate the wrong answer consistently.
Roughly, they are asking whether you can take an AI system and effectively reduce it to an analytic function and tell, without actually running the AI, what the output is going to be with a particular input.
The tape under the turing machine is infinite, so it's certainly not limited to 47M states. The conjecture states that all machines that halt will do so within 47M states.
I can still get SQLite to trivially corrupt indexes and table data by running it on top of NFS and dropping the network. I wouldn't put much money on this statement on their website.
Btrfs is a better choice for SQLite.