> Then came the pricing. The last quote I got for one of the tools on the market was $6,000/year for a team of 16 people in a use-it-or-loose-it way. For a tool we use maybe 2–3 times per sprint.
I could be wrong, but the idea of ending backend and front end reactivity (or sync) is not a good idea. As far as I understand FE reactivity occurs when there is an action but the backend reactivity doesn’t require an action there could be backend jobs that sync state between two components.
Pretty much the entire Data Science/Machine Learning landscape from numpy, etc. to tensorflow and alike are thin wrappers over C code and if you want performance, you better batch structure your operation beforehand and minimize back and forth from Python.
The GIL is only a problem if you’re trying to access the same memory.
If you take the time to split up your data into chunks you can avoid the GIL entirely with multiprocessing. Or by handing it off to a library that does it for you. just as long as they’re not using the same python objects.
Using multiprocessing adds some overhead for because of serialization, so it's slower than it would be to just hand off the Python objects directly to the workers (as you can with threads) because the process doing the parsing also has to spend time on serializing them again. So you can avoid the GIL, but it has a cost.
For example, if I parse an XML document with ElementTree, as a quick experiment, parsing the document takes ~1 second and serializing all the elements names and attributes to JSON takes an additional ~0.5 seconds. Serializing the whole ElementTree object using pickle takes ~4 seconds. Serializing it as XML takes roughly as long as parsing it.
Plus, [de]serialization often needs to be done under GIL (although under non-shared GIL per process, it interferes with any other async thing that may be done in each process).
It also screws up C extensions that don't support fork and does not solve the problem of increasing TLS handshake/fanout in outbound requests by the number of cores.
multiprocessing has more overhead even without serialization. I just brought it up to expand on why the GIL would force someone to thinking about being data-parallel.
What I was trying to say, I guess, is that the additional serialization overhead can't be parallelized, which means that parallelization with multiprocessing doesn't help much in some cases where GIL-free threading would.
Pyflink seems promising, I love vanilla flink but as soon as you need to debug your pyflink job pyflink becomes a hurdle. That translation layer between Python and Java can be opaque.
I enjoy building frameworks and libraries for fun, so kudos to you for that. At first glance, BlackSheep seems to combine the aspects of Flask, Django, and FastAPI. However, I’m a bit skeptical about its ability to coexist alongside them. The one feature I believe could drive the success of the next Python framework is the ability to operate in nogil mode.
How is the no-gil performance compared to other languages like - javascript (nodejs), go, rust, and even java? If it's bearable then I believe there is enormous value that could be generated instead of spending time porting to other languages.
No-GIL Python is still interpreted - single-threaded performance is slower that standard Python, which is in turn much slower than the languages you mentioned.
Maybe if you’ve got an embarrassingly parallel problem, and dozen(s) of cores to spare, you can match the performance of a single-threaded JIT/AOT compiled program.
How do companies like Instagram/OpenAI scale with a majority python codebase? Like I just kick it on HN idk much about computers or coding (think high school CS) why wouldn’t they migrate can someone explain like I’m five
Python may well have been the right choice for companies like that when they were starting out, but now they're much bigger, they would be better off with a different language.
However, they simply have too much code to rewrite it all in another language. Hence the attempts recently to fundamentally change Python itself to make it more suitable for large-scale codebases.
<rant>And IMO less suitable for writing small scripts, which is what the majority of Python programmers are actually doing.</rant>
What tool was this?