It's easy to dismiss our efforts, but Ruby is just as dynamic if not more than Python. It's also a very difficult language to optimize. I think we could have done the same for Python. In fact the Python JIT people reached out to me when they were starting this project. They probably felt encouraged seeing our success. However they decided to ignore my advice and go with their own unproven approach.
This is probably going to be an unpopular take but building a good JIT compiler is hard and leadership matters. I started the YJIT project with 10+ years of JIT compiler experience and a team of skilled engineers, whereas AFAIK the Python JIT project was lead by a student. It was an uphill battle getting YJIT to work well at first. We needed grit and I pushed for a very data-driven approach so we could learn from our early failures and make informed decisions. Make of that what you will.
Yes Python is hard to optimize. I Still believe that a good JIT for CPython is very possible but it needs to be done right. Hire me if you want that done :)
> whereas AFAIK the Python JIT project was lead by a student.
I am definitely not leading the team! I am frankly unqualified to do so lol. The team is mostly led by Mark Shannon, who has 10+ years of compiler/static analysis experience as well. The only thing I initially led was the optimizer implementation for the JIT. The overall design to choose tracing, to use copy and patch, etc. were other people.
> However they decided to ignore my advice and go with their own unproven approach.
Your advice was very much appreciated and I definitely didn't ignore your advice. I just don't have much say over the initial architectural choices we make. We're slowly changing the JIT based on data, but it is an uphill battle like you said. If you're interested, it's slowly becoming more like lazy basic block versioning https://github.com/python/cpython/issues/128939
You did great work on YJIT, and I am quite thankful for that.
Thanks for the response Maxime, your work on YJIT is astounding. The speedup from YJIT was a huge improvement over cRuby or MJIT, and the work was done relatively quickly compared to Python which seems to always be talking about this JIT but we are never seeing a comparable release.
Having had no experience in JIT development but having followed the faster cpython JIT progress on a weekly basis, I do find their JIT strategy a bit weird. The entire decision seemed to revolve around not wanting to embebed an external JIT/compiler with all that entails...
At first I thought their solution was really elegant. I have an appreciation for their approach, and I could have been captivated myself to choose it. But at this point I think this is a sunk cost fallacy. The JIT is not close to providing significant improvements and no one in the faster cpython community seems to be able to call the shot that the foundational approach may not be able to give optimal results.
I either hope to be wrong or hope that faster cpython managment has a better vision for the JIT than I do.
YJIT is optimized primarily for web workloads. We look at rails performance a lot, but also at various other libraries that are used in that context. If you look at the headline benchmarks at https://speed.yjit.org, it will give you an idea of what we're mostly focused on. This is in contrast with academic compiler project, which often focus on microbenchmarks and code that is very different from the code users actually run in practice.
YJIT or TruffleRuby: I'm biased being that I work on YJIT. The nice thing about YJIT is that it's likely to work out of the box, and just be a matter of calling `ruby --yjit` to turn it on. It will probably use a lot less memory than TR, and it's probably more likely to deliver the result you're looking for at this time (speed boost, no hassle).
That being said, for some small or specialized applications, TruffleRuby could deliver much higher peak performance. If you don't restart your server often and you have a lot of memory available, then maybe you don't care about warm-up time or memory usage, and TruffleRuby could be the right tool for you. Feel free to run your own benchmarks and also to blog about the results (though if you do, please share as much details about your setup as possible).
Yes, Marc Feeley was my PhD advisor. We came up with the original idea together. I also see it as a development of the work I did in my M.Sc. thesis on type-driven versioning of functions. Basic block versioning is lazy, type-driven tail splitting of code.
From the blog post, it sounded like their benchmark relied heavily on a gem with a C extension (nokogiri). It's hard to imagine a performance improvement on code that YJIT has no control over.
Yes, we ended up replacing Nokogiri by Nokolexbor, our own port of lexbor parser with like almost full compatibility with Nokogiri APIs while being around 5x faster: https://github.com/serpapi/nokolexbor
It is enough iterations for these VMs to warm up on the benchmarks we've looked at, but the warm-up time is still on the order of minutes on some benchmarks, which is impractical for many applications.
On the flip side, YJIT is probably one of the most memory-efficient JIT compilers out there (for any language). I say this having spoken to other JIT implementers.
We've worked really hard to reduce the memory overhead and at Shopify it's now down to less than 10% in our flagship production deployment.
Regardless, if memory usage is a legitimate concern for you, you can very easily remove the Rails initializer that turns on YJIT. You can choose between memory usage and response time. The choice is yours.
Oh yeah, please don't get me wrong. My original reply isn't to take away what YJIT accomplished. It was mainly to reference that in the past YJIT used more memory but maybe starting with Ruby 3.3 you won't have to think about that aspect anymore, even for apps running on a low spec machine.
I can't even begin to fathom at how much work and testing must have went into YJIT to make it work as well as it does. Thank you!
Author here. If you've grown up in a "normal", functional family, with two loving parents, and you enjoy talking to them on the phone, you should consider yourself very lucky.
I only take calls from my mother when I feel up to it. The reason for that is that she has no concept of boundaries. Her mental illness prevents her from grasping that concept. Her default behavior tends towards what a normal person would call harassment. If you can't relate or understand a situation like that, it might just be because you've had a relatively safe, coddled, privileged life.
