Something to keep in mind - experienced devops and security experts are extremely valuable, at the same time ML beginners are in oversupply. Do you really want to put yourself at a massive disadvantage in the near to medium term, especially if the future of ML is kind of uncertain while with your other skillset you can have a pretty stable career?
As someone working in ML (a couple of years of experience), I'd much rather be in your position than mine.
There's a huge oversupply everywhere until expert ML levels.
Just about everybody was able to tweak some parameter in models, some can explain themselves, others can't, the end result doesn't differ that much.
The way I see it, there's 2 ways you can walk around this. By being a software engineer that deals with the underlying ingestion/infra (that's increasingly a solved problem too), or be the guy that actually write the ML package themselves. Only the latter will be anywhere near secure, but that's almost 0 percent of the current supply.
I'm glad I got out of machine learning/data engineer role for a pure software engineer.
As someone working in ML (a couple of years of experience), I'd much rather be in your position than mine.