Just grab any shared hosting for $5-10/mo (A Small Orange for example). They’ll keep PHP and MySQL updated for you, and many of them provide one click WP installs as well—and even if they don’t, WP is very easy to install, and it has a built-in one-click updater.
Especially easy to install because often, for the bigger providers, there are provider specific instructions to install stuff. Sometimes even from the provider themselves.
