I don't think it automatically starts up. You have to run fish-config to start it yes?
$ fish_config
Web config started at 'file:///Users/bjorn/.cache/fish/web_config-6VIDFM.html'. Hit enter to stop.
::1 - - [12/Jul/2015 11:13:31] code 403, message Forbidden
When I kill this and refresh the config in the browser doesn't load anymore. So I don't see the problem.