Exactly. I question why the parent says you have to re-encrypt the drive.
Microsoft has the KEK or passphrase that can be used to derive the KEK. The KEK protects the DEK which is used to encrypt the data. Rotating the KEK (or KEKs if multiple slots are used) will overwrite the encrypted DEK, rendering the old KEK useless.
Or does BitLocker work differently than typical data at rest encryption?
BitLocker recovery keys are essentially the key to an at-rest, local copy of the real key. (I.e., they need access to the encrypted drive to get the real encryption key)
When you use a recovery key at preboot, it decrypts that on-disk backup copy of the encryption key with your numerical recovery key, and uses the decrypted form as the actual disk encryption key. Thus, you can delete & regenerate a recovery key, or even create several different recovery keys.
And if you don’t use a lot of data, at least US Mobile has a by the gig plan. My family has three phones on it for a total $30 per month. Those months that we go over, it automatically charges $2 for each extra GB, with data pooled between the lines.
It is easy to switch between Verizon, AT&T, and T-Mobile as well. This was helpful for me as all three of the networks normally have one bar or less at my house. T-mobile WiFi calling works more reliably than Verizon.
Mirroring between s3 providers would seemingly give protection against your account being locked at one of them.
I expect this becomes most interesting with l2arc and cache (zil) devices to hold the working set and hide write latency. Maybe would require tuning or changes to allow 1m writes to use the cache device.
I’m guessing v4 C didn’t have structs yet (v6 C does, but struct members are actually in the global namespace and are basically just sugar for offset and a type cast; member access even worked on literals. That’s why structs from early unix APIs have prefixed member names, like st_mode.
There may have been a early C without structs (B had none,) but according to Ken Thompson, the addition of structs to C was an important change, and a reason why his third attempt rewrite UNIX from assembly to a portable language finally succeeded. Certainly by the time the recently recovered v4 tape was made, C had structs:
~/unix_v4$ cat usr/sys/proc.h
struct proc {
char p_stat;
char p_flag;
char p_pri;
char p_sig;
char p_null;
char p_time;
int p_ttyp;
int p_pid;
int p_ppid;
int p_addr;
int p_size;
int p_wchan;
int *p_textp;
} proc[NPROC];
/* stat codes */
#define SSLEEP 1
#define SWAIT 2
#define SRUN 3
#define SIDL 4
#define SZOMB 5
/* flag codes */
#define SLOAD 01
#define SSYS 02
#define SLOCK 04
#define SSWAP 010
It seems pointless to issue flush commands when writing to an NVMe drive with a direct IO implementation that functions properly. The NVMe spec says:
> 6.8 Flush command
> …
> If a volatile write cache is not present or not enabled, then Flush commands shall complete successfully and have no effect.
And:
> 5.21.1.6 Volatile Write Cache
> …
> Note: If the controller is able to guarantee that data present in a write cache is written to non-volatile media on loss of power, then that write cache is considered non-volatile and this feature does not apply to that write cache.
If you know your application will only ever run against enterprise SSDs with power loss protection, then sending flush commands to the drive itself would indeed be pointless no-ops. But it if it's a flush command that has effects somewhere between the application layer and the NVMe drive (eg. if you're not using direct IO) or if there's any possibility of the code being run on a consumer SSD (eg. a developer's laptop) then the flush commands are probably worth including; the performance hit on enterprise drives will be very small.
IOCTLs can tell you if write caching is enabled or not. Can they reliably tell you whether the write cache is volatile, though? Many drives with PLPs still report volatile write caches, or at least did when I was testing this a few years back.
Per the definition of volatile write cache in the standard I quoted, pretty much any drive TLC drive in the hyperscalar, datacenter, or enterprise product lineup will have great write performance. They have a DRAM cache that is battery-backed, and as such is not a volatile write cache.
A specific somewhat dated example: Samsung 980 Pro (consumer client), PM9A1 (OEM client), and PM9A3 (datacenter) are very similar drives that have the same PCI ID and are all available as M.2. PM9A3 drives have power loss protection and the others don’t. It has very consistent write latency (on the order of 20 - 50 μs when not exceptionally busy) and very consistent throughput (up to 1.5 GB/s) regardless of how full it is. The same cannot be said of the client drives without PLP but with tricks like TurboWrite (aka pseudo-SLC). When more than 30% of the NAND is erased, the client drives can take writes at 5 GB/s but that rate falls off a cliff and gets wobbly when the pseudo-SLC cache fills.
Thanks! Yes, as the sibling noted, if you limit this to PLP drives it makes sense, but that is also a special case. Outside of the latency hit (which is significant in some cases), FLUSH is also nearly free on those though.
The original idea of boot environments in Solaris came from Live Upgrade, which worked at least as far back as Solaris 8. Live upgrade was not part of Solaris, rather it was an addon that came from the services or enterprise support parts of Sun.
Solaris 11 made boot environments a mandatory part of the OS, which was an obvious choice with the transition from
UFS to ZFS for the root fs. This came into Solaris development a bit before Solaris 11, so it was present in OpenSolaris and lives on in many forms of illumos.
Datacenter storage will generally not be using M.2 client drives. They employ optimizations that win many benchmarks but sacrifice on consistency multiple dimensions (power loss protection, write performance degrades as they fill, perhaps others).
With SSDs, the write pattern is very important to read performance.
Datacenter and enterprise class drives tend to have a maximum transfer size of 128k, which is seemingly the NAND block size. A block is the thing that needs to be erased before rewriting.
Most drives seem to have an indirection unit size of 4k. If a write is not a multiple of the IU size or not aligned, the drive will have to do a read-modify-write. It is the IU size that is most relevant to filesystem block size.
If a small write happens atop a block that was fully written with one write, a read of that LBA range will lead to at least two NAND reads until garbage collection fixes it.
If all writes are done such that they are 128k aligned, sequential reads will be optimal and with sufficient queue depth random 128k reads may match sequential read speed. Depending on the drive, sequential reads may retain an edge due to the drive’s read ahead. My own benchmarks of gen4 U.2 drives generally backs up these statements.
At these speeds, the OS or app performing buffered reads may lead to reduced speed because cache management becomes relatively expensive. Testing should be done with direct IO using libaio or similar.
This article is talking about SATA SSDs, not HDDs. While the NVMe spec does allow for MVMe HDDs, it seems silly to waste even one PCIe lane on a HDD. SATA HDDs continue to make sense.
And I'm saying assuming that m.2 slots are sufficient to replace SATA is folly because it is only talking about SSDs.
And SATA SSDs do make sense, they are significantly more cost effective than NVMe and trivial to expand. Compare the simplicity, ease, and cost of building an array/pool of many disks comprised of either 2.5" SATA SSDs or M.2 NVMe and get back to me when you have a solution that can scale to 8, 14, or 60 disks as easily and cheaply as the SATA option can. There are many cases where the performance of SSDs going over ACHI (or SAS) is plenty and you don't need to pay the cost of going to full-on PCIe lanes per disk.
> And SATA SSDs do make sense, they are significantly more cost effective than NVMe
That doesn't seem to be what the vendors think, and they're probably in a better position to know what's selling well and how much it costs to build.
We're probably reaching the point where the up-front costs of qualifying new NAND with old SATA SSD controllers and updating the firmware to properly manage the new NAND is a cost that cannot be recouped by a year or two of sales of an updated SATA SSD.
SATA SSDs are a technological dead end that's no longer economically important for consumer storage or large scale datacenter deployments. The one remaining niche you've pointed to (low-performance storage servers) is not a large enough market to sustain anything like the product ecosystem that existed a decade ago for SATA SSDs.
In addition to my other comments about parallel IO and unbuffered IO, be aware that WS2022 has (had?) a rather slow NVMe driver. It has been improved in WS2025.
Microsoft has the KEK or passphrase that can be used to derive the KEK. The KEK protects the DEK which is used to encrypt the data. Rotating the KEK (or KEKs if multiple slots are used) will overwrite the encrypted DEK, rendering the old KEK useless.
Or does BitLocker work differently than typical data at rest encryption?
reply