I've noticed that some things can make the price suddenly skyrocket. Certain PCB specifications for one (though couldn't name anything in particular off hand). The one thing that caught me by surprise was that if I wanted to have JLCPCB solder the ESP32 for me the cost would jump by more than double, and the PCB would have to be physically larger. Something about their "Economic" vs. "Standard" PCB assembly where, for reasons I'm not clear on, the ESP32 could only be done under the latter. I opted to solder that component myself, of course.
I set ESPHome to use IDF under the hood, you can check out my full ESPHome config for this: https://git.ellis.codes/e/esphome-configs/src/branch/main/vo...