Having built production apps on SQLite I can say it's not all sunshine and roses, the complexity explodes the moment you need multiple workers that can all write.
You better hope you dont have any big indexes or your writes will queue and you start getting "database is locked" errors. Good luck queueing writes properly at the application layer without rebuilding a full distributed queue system / network interface / query retry system on top of SQLite.
I really really really wish SQLite or PGLite supported multiple writers, especially if the queries are not even touching the same tables.
Oh I didn't say that, but nor is postgres. Use the right tool for the job I agree with, just people are making their lives difficult (expensive as well) by just blindly picking postgres.
SQLite is great until you try to do any kind of multi-writer stuff. Theres no SELECT FOR UPDATE locking and no parallel write support, if any of your writes take more than a few ms you end up having to manage queueing at the application layer, which means you end up having to build your own concurrent-safe multi-writer queue anyway.
I really wish I could but it's hard to embed in local-first apps and packages without forcing users to set up docker.
PGlite would be perfect if only it allowed multiple writer connections. SQLite is ok but I want PG extensions and I want true parallel multi-writer support!
Unrelated to parent thread: Sorry to ping you here @Reedlaw but it appears the email on your personal site is rejecting incoming mail.
I'm trying to reach you regarding an older rubygems package you maintain. We're trying to offer you some $ to rename it because we maintain a large project with the same name in other languages and want to start offering it in ruby. I sent you a connect request + message on Linkedin from https://www.linkedin.com/in/nicksweeting/
Yeah, I don't see how Python is fundamentally different from JavaScript as far as dynamicism goes. Sure Python has operator overloading, but JavaScript would implement those as regular methods. Pyrhon's init & new aren't any more convoluted than JavaScript's constructors. Python may support multiple inheritance but method and attribute resolution just uses the MRO which is no different than JavaScript's prototype chain.
ArchiveBox open source does not, but I have set it up for paying clients in the past using TLSNotary. This is actually a very hard problem and is not as simple as saving traffic hashes + original SSL certs (because HTTPS connections use a symmetric key after the initial handshake, the archivist can forge server responses and claim the server sent things that it did not).
There is only 1 reasonable approach that I know of as of today: https://tlsnotary.org/docs/intro, and it still involves trusting a third party with reputation (though it cleverly uses a zk algorithm so that the third party doesn't have to see the cleartext). Anyone claiming to provide "verifyable" web archives is likely lying or overstating it unless they are using TLSNotary or a similar approach. I've seen far to many companies make impossible claims about "signed" or "verified" web archives over the last decade, be very critial any time you see someone claiming that unless they talk explicitly about the "TLS Non-Repudiation Problem" and how they solve it: https://security.stackexchange.com/questions/103645/does-ssl...
If web pages were signed the way emails were, it would authenticate if an archived copy of a web page is indeed authentic, but good luck getting such a major change all the way across the entire web. Why would anyone who would gladly retract / redact information on a whim even subscribe to this technology? Would be nice if they all did though.
Its not for authenticity but instead to prevent tampering of adblock,ad and tracking removals more if there is history in search many complains in this
I've been mulling over how to take ArchiveBox in this direction for years, but it's a really hard problem to tackle because of privacy. https://docs.sweeting.me/s/cookie-dilemma
Most content is going behind logins these days, and if you include the PII of the person doing the archiving in the archives then it's A. really easy for providers to block that account B. potentially dangerous to dox the person doing the archiving. The problem is removing PII from logged in sites is that it's not as simple as stripping some EXIF data, the html and JS is littered with secret tokens, usernames, user-specific notifications, etc. that would reveal the ID of the archivist and cant be removed without breaking page behavior on replay.
My latest progress is that it might be possible to anonymize logged in snapshots by using the intersection of two different logged-in snapshots, making them easier to share over a distributed system like Bittorrent or IPFS without doxxing the archivist.
btop is good, I like 'glances' the best though because like 'atop' it actually highlights whatever problem is most likely to be causing lag at the moment, and it breaks out docker containers into a separate section and labels them properly.
You better hope you dont have any big indexes or your writes will queue and you start getting "database is locked" errors. Good luck queueing writes properly at the application layer without rebuilding a full distributed queue system / network interface / query retry system on top of SQLite.
I really really really wish SQLite or PGLite supported multiple writers, especially if the queries are not even touching the same tables.
reply