So, with the latest iOS 14 (and related backports to 13.7), the contact is now activating Apple Map each and every time you search for it.

But there is no configuration setting to disable this.