Good question. I don't think there's an easy answer, except for something like:

A statement is provable if it's possible to write a proof for it.

But you're unlikely to find that satisfactory...

Spelling out the bounds of what can possibly constitute a proof is a big question in mathematical logic, model theory, and proof theory, and I'm far from an expert in these subjects.

I think any proof has to be based on some set of axioms, but Godel showed that not all provable statements can be formally derived from a single set of axioms.

Another way to put it might be that the space of all possible proofs can't be defined by a single axiom system with its rules of inference, but I'm not sure about this, because as I understand it we should always be able to tell mechanically, given a putative proof, whether it really proves the theorem.

In any case, I believe all these subtleties and difficulties also have to be faced if we try to spell out precisely what we mean by mathematical truth. In my view, this is the same question: mathematically true = it's possible to prove it.