Exactly, fuel should be more expensive because the negative externalities of its consumption should be priced in.
The extra value extracted from pricing in those externalities should be directed by the state towards offsetting the damage, it shouldn’t just be pocketed by the companies causing the damage.
The problem is that we no longer have the time to let market forces work that slowly. Things like home heaters, stoves, vehicle engines, etc. have service lifespans measured in decades so we need everyone buying electric now. Things like EVs or heat pumps often have higher upfront cost so we need to stop having the situation where people feel like they have to pay more to do the right thing because the fossil fuel prices are subsidized so low that many people don’t feel much pressure to change.
That's a valid concern and is the reason to scale the tax in over a many year period. That tax revenue doesn't have to be consumed by a swelling government, but part of it could be directly distributed as a dividend to each member of the society on an equal per-capita basis or biased towards lower income members of the society.
The counterfactual challenge is brutal, too: if we don’t do anything, the impact on food production will be far worse but if we don’t let it happen first we’re going to be plagued with people saying it wouldn’t have been so bad.
To clarify, we've been minimally subsidizing clean energy while not only significantly subsidizing oil exploration and development, but also using the largest military + intelligence budgets (US, UK) to "stabilize" oil producing regions and transportation.
The side-effect of this is, yes, unfortunately increased profits for oil companies.
I don't know if these could be cut down using some windfall tax scheme.