I briefly checked the source code, and I think we can't use this for our models, because we have lots of pre-processing and post-processing in python.

using this framework will require us to do everything in rust?

We are currently using torch serve. Although it's in java, it runs python code as workers.