Lower power density implies larger size, which implies higher cost. Since the parts outside the reactors themselves (the turbines, generators, heat sink, etc.) will be similar, this means a fusion power plant will cost more to build that a fission power plant.

The other problem is that size is also related to reliability. A fusion reactor will have many more parts than a fission reactor. It will also be much more complex, and operate at higher radiation and thermal stresses. This is particularly important because it is very difficult to repair something that is so radioactive that hands-on access is not possible.