Translation: I have written buggy, racy software that has specific dependencies on thread timing. Please do not make significant improvements to Python because it will reveal these bugs in my software, and I will be forced to fix the bugs and use proper synchronization.

If you make something that works because of an explicit memory and concurrency model (and not like there are other options at the time), it is indeed legit to worry about a major shift to those models that would cause problems.

Even if those changes are better for other ways of solving problems.

Is that how you call things that have been working flawlessly and solving people's problems for over 10 years? Is needlessly breaking things that work an improvement to you?

You could probably simply lock the Python version you use for such code. No breakage there. If you must upgrade to a newer Python version, then you will have to repair broken code.

This went really well for the Python 2 -> 3 upgrade

It did buy a decade or so (or more really) - not like the python2 distribution you downloaded and distribute with your program back then is going to get tracked down and shot in the head by Guido anytime soon.

If you’re relying on whatever python version is distributed with whatever machine it happens to be on, there are a huge number of problems you’re already going to have.

Isn't that.. Already the case? The gil doesn't prevent thread switching in the middle of python code.

It's concurrent not parallel. The switch won't happen inside the execution of one opcode including some dictionary update operations so it's safe in many cases where parallel execution isn't.

Yes, single-instruction operations would be fine, but if you're writing multithreaded code you are probably doing things that the GIL doesn't protect all the time. Like dict-updates on classes that implement __set__, or `if not a[x]: a[x] = y` sorts of two-phased checks, or just like, anything else. You can't get very far with global state without reckoning with concurrency, GIL or not.

I assume that a change to relax the GIL will both allow you to opt-out of it, and allow you to use locking versions of primitive data-structures, anyway; it's not like it's going to just vanish overnight with no guardrails.

Currently, dictionary updates are atomic only if the keys are primitive types.

`dict.update()` will call methods like `__eq__`, and those methods (if implemented in Python) may temporarily release the GIL.

Unfortunately the Python docs says that dict.update() is atomic. It's being fixed though... It came out during these discussions.

It is probably a bad practice to not acquire a mutex for that concurrent dictionary update. The code should be improved in that regard, with or without any potential Python language change.

If needed, I'll probably just change it to asyncio, probably saner than inspecting everything for new parallelism bugs which can be incredibly subtle.

If performance isn't hugely important you could make blanket-locking wrappers around common data structures and swap them in-place for all of your global state.

.. but, as I said, removing the GIL will almost certainly be opt-in.

Seems like checking the docs for atomicity of every operation is a huge pain in the ass.

Best to avoid sharing data, and never mutating shared data!

Rust has a great rule: Sharing XOR Mutation.

Python is higher level, so message passing and passing "owned" values between threads is all the more feasible and sensible.

It really is, they don't make it clear at all. Every time I have to ask the question of "is this atomic under the GIL" I struggle to find the right answer.

That was my first thought as well. I use Python to whip up quick scripts and enjoy not having to worry about shared memory, even when I'm using concurrency. I'd hate to lose that.

Don’t you still have to worry about this since OS threads can be arbitrarily preempted?

That's fine. You don't have to worry about pausing, you have to worry about multiple threads getting at some memory at the same time.