I remember being completely impressed by my Note 3 (10 years ago almost exactly) playing games like Dead Trigger 2 and NFS: Underground. From then on, GPU performance / graphics on these tiny displays seemed good enough for me with only minor subjective improvements.

I'll second another comment, that another big challenge is the simple lack of ergonomics on any phone.