I've been burned too many times by embracing open source products like this.
We've been fed promises like these before. They will inevitably get acquired. Years of documentation, issues, and pull requests will be deleted with little-to-no notice. An exclusively commercial replacement will materialize from the new company that is inexplicably missing the features you relied on in the first place.
For what it's worth, I understand this concern. However, I want to emphasize that pyx is intentionally distinct from Astral's tools. From the announcement post:
> Beyond the product itself, pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever. Nothing changes there. Instead, we'll offer paid, hosted services that represent the "natural next thing you need" when you're already using our tools: the Astral platform.
Basically, we're hoping to address this concern by building a separate sustainable commercial product rather than monetizing our open source tools.
I believe that you are sincere and truthful in what you say.
Unfortunately, the integrity of employees is no guard against the greed of investors.
Maybe next year investors change the CEO and entire management and they start monetizing the open source tools. There is no way of knowing. But history tells us that there is a non-trivial chance of this happening.
The uncertainly over a future rug pull is always real, but in reality I wonder if the actual reason for people's hesitancy is more than just that. In reality I suspect it's closer to one of simply identity and the ownership model itself. Just the very idea that core tooling you depend on is in the hands of a commercial company is enough to many back off in a way one might not be when the tooling is in the hands of a broader community that one can support on more equal terms.
@woodruffw I love your quote above that commits you to your open source base and I'm rooting for you. But how about an approach that commits you to this sentence in a more rigorous and legal way, and spin off your open source tooling to a separate community-based entity? Of course, upon that you can continue to maintain sufficient representation to make Astral's commercial products the natural progression and otherwise the model remains the same. That would be a significant transfer of control, but it is that very transfer that would get a overwhelming response from the community and could really unblock these great tools for massive growth.
I work a lot with LLVM/Clang and whilst i know Apple and Google are significant contributors I feel confident that LLVM itself exists outside of that yet accept that e.g. Apple's contributions afford them weight to steer the project in ways that match their interests in e.g. Swift and Apple tooling.
It makes sense, but the danger can come when non-paying users unwittingly become dependent on a service that is subsidized by paying customers. What you're describing could make sense if pyx is only private, but what if there is some kind of free-to-use pyx server that people start using? They may not realize they're building on sand until the VC investors start tightening the screws and insist you stop wasting money by providing the free service.
(Even with an entirely private setup, there is the risk that it will encourage too much developer attention to shift to working within that silo and thus starve the non-paying community of support, although I think this risk is less, given Python's enormous breadth of usage across communities of various levels of monetization.)
Conda said all this as well, and solved the same issues you’re trying to - namely precompiled versions of difficult to build packages. It then went commercial.
In the HPC space there are already Easybuild and Spack which make all the compiler tool chain and C and Fortran library dependency stuff very easy. They just haven’t taken off outside as they aim to solve cluster management problems but Spack is easy to self serve.
Yes, it went commercial. But Conda is still free. And the community had built conda-forge to maintain packages. Conda + conda-forge can completly replace Anaconda for most people.
So even though they went commercial, they left pretty good things behind for the Open source community.
The entire reason people choose "permissive licenses" is so that it won't last forever. At best, the community can fork the old version without any future features.
I don't think this is true -- a license's virality doesn't mean that its copyright holders can't switch a future version to a proprietary license; past grants don't imply grants to future work under any open source license.
Correct; however, without a CLA and assuming there are outside contributors, relicensing the existing code would be mildly painful, if not downright impossible.
You're saying that would be more painful in a viral license setting, right? If so I agree, although I think there's a pretty long track record of financially incentivized companies being willing to take that pain. MongoDB's AGPL transition comes to mind.
But, to refocus on the case at hand: Astral's tools don't require contributors to sign a CLA. I understand (and am sympathetic) to the suspicion here, but the bigger picture here is that Astral wants to build services as a product, rather than compromising the open source nature of its tools. That's why the announcement tries to cleanly differentiate between the two.
Yeah, only the (viral license + no CLA) combo has this effect of preventing you from changing license down the road. In your case (permissive license, no CLA) you still can relicense uv or Ruff. Not saying that you will do it, of course :-)
They have no need to, current repos show everything is under MIT/Apache. They could close the source at any time and not worry about CLA.
>bigger picture here is that Astral wants to build services as a product
What services? pyx? Looks nice but I doubt my boss is going to pay for it. More likely they just say "Whatever, package is in PyPi, use that."
UV, Ruff, Ty. Again, maybe they can get some data/quant firm who REALLY cares about speed to use their products. Everyone else will emit a long sigh, grab pip/poetry, black and mypy and move on.
> I think there's a pretty long track record of financially incentivized companies being willing to take that pain. MongoDB's AGPL transition comes to mind.
MongoDB had a CLA from the start, didn't it?
> Astral's tools don't require contributors to sign a CLA.
yeah? who's going to court for multiple years to fight giants? Maybe they'll pull a RedHat and put the code behind a subscription service. Still OSS right?
By viral, do you mean licenses like GPL that force those who have modified the code to release their changes (if they're distributing binaries that include those changes)?
Because FWIW CPython is not GPL. They have their own license but do not require modifications to be made public.
This is just plain false and honestly close-minded. People choose permissive licenses for all sorts of reasons. Some might want to close it off later, but lots of people prefer the non-viral nature of permissive licenses, because it doesn't constrain others' license choice in the future. Still others think that permissive licenses are more free than copyleft, and choose them for that reason. Please don't just accuse vast groups of people of being bad-faith actors just because you disagree with their license choice.
They've been the only game in town for a while, and their pricing reflects it. But this project is only for Python (for now?) so JFrog is not _immediately_ in danger.
Will pyx describe a server protocol that could be implemented by others, or otherwise provide software that others can use to host their own servers? (Or maybe even that PyPI can use to improve its own offering?) That is, when using "paid, hosted services like pyx", is one paying for the ability to use the pyx software in and of itself, or is one simply paying for access to Astral's particular server that runs it?
I might not be following: what would that protocol entail? pyx uses the same PEP 503/691 interfaces as every other Python index, but those interfaces would likely not be immediately useful to PyPI itself (since it already has them).
> or is one simply paying for access to Astral's particular server that runs it?
pyx is currently a service being offered by Astral. So it's not something you can currently self-host, if that's what you mean.
> pyx uses the same PEP 503/691 interfaces as every other Python index
... Then how can it make decisions about how to serve the package request that PyPI can't? Is there not some extension to the protocol so that uv can tell it more about the client system?
Interesting. I'll have to look into this further. (I've bookmarked the entire thread for reference, especially since I'm a bit attached to some of my other comments here ;) )
The hosting and administration part is what’s expensive and can’t be free and open source except when someone pays for it. So is npm open source? Whether it is or isn't doesn't matter as much as whether Microsoft continues to pay the bill.
I think the only way you can alleviate people's concerns here is by legally binding the company in a way that would adequately address those concerns. Which I don't recall ever seeing any company attempt in such a scenario, probably because it obviously devalues the company as it restricts what a potential future buyer could do with it. Which is exactly why some people don't buy such pinky promises.
I love using uv, but having worked for a VC funded open source startup, your concerns are spot on.
As soon as there is a commercial interest competing with the open source project at the same company the OSS version will begin to degrade, and often the OSS community will be left in the dark about this. The startup I was at had plenty of funding, far too many engineers, and still removed basically every internal resource from the oss project except one person and drove out everyone working on the community end of things.
I would also recommend avoiding working for any open source startup if your goal is to get paid to contribute to a community project. Plenty of devs will take a reduced salary to work on a great community project, but most of the engineers I saw definitely got the "bait and switch" and moved immediately to commercial projects.
This is a valid concern, but astral just has an amazing track record.
I was surprised to see the community here on HN responding so cautiously. Been developing in python for about a decade now- whenever astral does something I get excited!
I agree in principle, but in this case uv is open source and SO MUCH better than pip it would be insane not to use it on those grounds.
With uv the worst case is it goes closed source in a few years and we all switch to a fork.
With pip the best case is that maybe in 10 years they have improved it to fix all the issues that uv has already fixed, and you only spend 10 years using shitty software and putting up with endless bugs and paper cuts.
Frankly, it’s weird. You can find this business model all over the open-source world but for some reason Astral in particular is singled out for way more criticism on this than anything else I’ve seen, despite being unambiguously great contributors who have never put a foot wrong as far as I can tell.
Microsoft – who invented embrace, extend, and extinguish – own NPM, but I don’t see people wringing their hands over them in every thread that mentions NPM. But you mention Astral here or on Reddit and people line up to tell you it’s only a matter of time before they fuck people over. Why the disparity?
NPM has always been commercial (rather than managed by a foundation), and it was nominally acquired by GitHub rather than Microsoft, so at some level as long as GitHub is not causing issues (noting the recent GitHub changes should maybe also imply some consideration of problems for NPM), NPM is "safe".
Astral on the other hand has basically been rewrites in Rust of existing community-based open source tools, for which there is always the question of how such work is funded. PYX (which is an interesting choice of name given the conflicts with pyrex/cython filenames) from what we can see here appears to be in a similar vein, competing with PyPI and making changes which seemingly require their client (uv) be used.
Anaconda/ContinuumIO was also treated with similar suspicion to Astral, so I don't think it's Astral in particular, it's more they both are operating in the part of the ecosystem where it is comparatively easy to lock out community-based open source tools (which the Python ecosystem appears to have been better at setting up and maintaining than the JS ecosystem).
pyx doesn't compete with PyPI; it's a private registry that companies can use e.g. to host internal-only packages, or to provide curated views of things like PyPI for compliance reasons.
> making changes which seemingly require their client (uv) be used
That's an explicit non-goal: "You won't need to use pyx to use uv, and you won't need to use uv to use pyx."
Astral are not causing issues though. Why does “as long as Astral is not causing issues” not apply?
> Anaconda/ContinuumIO was also treated with similar suspicion to Astral
I haven’t observed this. I have seen condo talked about a fair amount but any issues have always revolved around it being awkward to use. But practically every discussion here or on Reddit about Astral has FUD.
Sorry, the quotes around safe were supposed to imply GitHub is not that safe in my opinion, but it's possibly why other people aren't concerned about NPM (also, being for a different programming language and community may help).
Anaconda/ContinuumIO (the company) was absolutely treated with suspicion, see e.g. https://www.mail-archive.com/numpy-discussion%40scipy.org/ms... (and you'll find many such threads around that time on mailing lists/forums that were where the scientific python community was), and while the sky didn't fall in, their history hasn't been spotless. In many ways Astral is the more "webby" version of Anaconda/ContinuumIO, and so assuming Astral will behave (and evolve) in a similar way to Anaconda/ContinuumIO seems to me at least to be a rational thing to do?
It’s great that astral are doing what they do, but it’s important to hedge against their success. We must maintain ecosystem fragmentation, or Astral becomes a choke point where a bad actor can seize the entire Python community and extract rent from it. Like Anaconda but successful. So keep using pylint, keep using mypy, keep using PyPI. Not exclusively, but as a deterrent against this kind of capture.
Has that ever happened in the Python ecosystem specifically? It seems like there would be a community fork led by a couple of medium-size tech companies within days of something like that happening, and all users except the most enterprise-brained would switch.
Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI.
Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). That has a lot to do with why I'm making PAPER.
For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process. It remembers entire wheels, but finding them in that cache requires knowing the URL from which they were downloaded. Because PyPI organizes the packages in its own database with its own custom URL scheme, pip would have to reach out to PyPI across the Internet in order to figure out where it put its own downloads!
> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original).
FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
I'm not saying here that pip doesn't have some gnarly internal details, just that the bigger thing holding it back is the lack of maintainer resources.
> For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process.
I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.
That said, I do think there are other good examples to your point, like uv's ability to prefetch package metadata, I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.
> FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
Fair enough. I'm sure if someone were paying me a competitive salary to develop my projects, they'd be getting done much faster, too.
> I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.
I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files.
> uv's ability to prefetch package metadata,
You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays.
> I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.
Ugh, yeah. I know the resolution logic has been extracted as a specific package, but it's been challenging trying to figure out how to actually use that in a project that isn't pip.
> I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files.
I'm talking about the fact there is no `uv pip download` or `uv pip wheel`.
I'm sure we'll discuss when you add this feature to uv, but I personally find uv's cache already grows too big too fast, so adding more to it makes me concerned.
> You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays.
Yeah, one of the way uv speeds up resolving is it pre-downloads .metadata files and checks if their requirements are identical to versions it had already checked, so it can quickly rule them out. It's a clever use of a collector and an advanced resolver.
>FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
No. This something people tell each other, that it's a lack of resources, but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage.
There are features in FreeCAD that are straight up just broken that you hit every single day of using FreeCAD. When a simple low cost fix is suggested you get immense pushback, because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance.
Progress happens when you sidestep those who block progress. That's what realthunder was doing and that's what Astral is doing with uv. No more bullshit excuses.
This is what forks are for. Alternately, money would probably help. Otherwise it's a bit rude to impose your priorities on others.
Perhaps some larger projects lose track of what they intended to prioritize. This is also a resources problem. There's nobody available to do management (and probably also not a social structure internally that would allow for it).
> because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance.
The developers will always be in a better position than end users to assess the long-term maintenance impact.
> The developers will always be in a better position than end users to assess the long-term maintenance impact.
At best developers may be in a better position to assess long-term consequences. The history of major software projects is also the history of many projects making technical decisions that look good in the short-term but which turn out to impede actually achieving the desired features/performance/etc, and thus lead to major refactors/rewrites.
imtringued is right though. Obstinate maintainership is a real failure mode of open source (and other) projects and it's often true that patch rejections involving phrases like "maintenance burden" are often pretextual. It's a shame to have to resort to creative destruction to improve things. Forks have a low success probability.
> Obstinate maintainership is a real failure mode of open source
I agree, but I also think sometimes that people think it's obstinate maintainership and it's not. For example, large PRs are hard to merge from a contributor who is fresh to the project because there is so much careful review that needs to be done, it's very resource intensive.
That said, one of my goals of becoming a maintainer has to be to try and making submitting a PR more friendly. Feel free to submit a PR to pip, I will be happy to help you get over any teething issues.
> No. This something people tell each other, that it's a lack of resources,
Well uv, as an OSS project, has come along with around 2 orders of magnitude more manpower than pip and has solved a lot of problems
> but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage.
I wasn't making a statement about OSS projects in general, but I agree that pip has prioritization problems but I would argue that it stems from the lack of resources.
The people who do donate their spare time are in fact only likely to donate that time on things that interest them. If there were resources to have someone act like a project manager, and other to follow their lead, then the prioritization problems could be fixed, but there aren't those resources.
> No more bullshit excuses
Then contribute your full time hours to pip, or other OSS that needs fixing, rather than arm chair commenting on hacker news.
Good to know. "which in turn are not all original" may have been an understatement. I don't know a lot of these details, because my interest in these things is more recent, and not so much driven by the complex multi-language projects.
This doesn't generalize: you could have said the same thing about pip versus easy_install, but pip clearly has worthwhile improvements over easy_install that were never merged back into the latter.
Pip is broken and has been for years, they're uninterested in fixing the search. Or even removing the search or replacing it with a message/link to the package index.
imo, if pip's preference is to ship broken functionality, then what is/is not shipped with pip is not meaningful.
This is not a charitable interpretation. The more charitable read is that fixing search is non-trivial and has interlocking considerations that go beyond what pip's volunteer maintainers reasonably want to or can pick up.
(And for the record: it isn't their fault at all. `pip search` doesn't work because PyPI removed the search API. PyPI removed that API for very good reasons[1].)
That was 7 years ago. If it's not coming back, the CLI should make that clear, instead of giving a temporary "cannot connect" message that implies it could work, if you wait a minute and try again.
It was three years ago; 2018 is when they considered removing the command, not when the search API was actually removed from PyPI.
And this is part of the interlocking considerations I mentioned: there are private indices that supply the XML-RPC API, and breaking them doesn't seem justifiable[1].
Does that seem like a real solution to you? That it's ok to represent a never-functional operation as one that might maybe work? ...because it could work of you jump through a bunch of undocumented hoops?
It's so wild to me that so many people are apparently against making a user-friendly update. The whole thing seems very against pep8 (its surprising, complicated, non-specific, etc)
I don't know what to tell you; I just gave an example of it being functional for a subset of users, who don't deserve to be broken just because it's non-functional on PyPI.
Nobody wants anything to be user-unfriendly. You're taking a very small view into Python packaging and extending it to motives, when resources (not even financial ones) are the primary challenge.
Is code to detect when the user is not in that subset and say that it doesn't work really really hard for some non-obvious reason? If the default case for the vast majority of users doesn't work, it doesn't seem like printing a more useful error message to them should be that hard.
> it doesn't seem like printing a more useful error message to them should be that hard.
I think the existing error message is useful:
$ pip search foo
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.
It says (1) what failed, (2) why it failed, and (3) links to a replacement, and (4) links to a deprecation explanation. That last link could maybe then link back to the pip issue or include some more context, but it's a far cry from being not helpful.
That is a nicer message than I remember. I don't have an issue with that. It let's you know it doesn't work, that it will never work, and provides links to an alternative and more information.
It either used to do fewer of those, or my memory is Swiss cheese, but I remember needing to search for the why's and alternatives.
There was a period when the error was a stack trace. But that was pretty brief, and happened before PyPI moved more "officially" towards not supporting the search API.
I think part of the confusion here (which is reasonable IMO) is that pip has a huge version tail, and so people report error messages and behaviors that have been fixed for years. This is one of the challenges that I think gets under-appreciated about pip's maintenance.
Your comment gave me an flashback to when I started programming and drag and dropped downloaded python packages into the packages folder instead of installing them.
I'm assuming by pip you mean pypi (the package registry) - I think you're making the mistake of thinking it has a lot more resources than it does, because of how much it punches above its weight.
Pypi is powered by warehouse which has around 3 developers maintaining it[0]. They're doing an awesome job, but the funding and resource available to them are probably substantially less than Astral could have with a paid offering.
We've been fed promises like these before. They will inevitably get acquired. Years of documentation, issues, and pull requests will be deleted with little-to-no notice. An exclusively commercial replacement will materialize from the new company that is inexplicably missing the features you relied on in the first place.