If you have to cross the kernel boundary every time you want to access something in your cache, your "cache" is now much, much slower. Note that this applies even if the OS keeps the file in memory, and doesn't require going to disk.
Sure, but if accepting that cache slowdown makes the rest of the system more responsive and more useful for background tasks, it may be worthwhile. It's a trade-off, and the browser makers have every incentive to be as selfish as possible.