The answer is, it depends. You have correctly answered your question.
If you DO add more water, then you are watering down the final product and so you end up with more beer of less potency. That may be a good thing or a bad thing, depending on what you are aiming for. If you are trying to brew a barley wine, then leave it alone and just go with less of the stronger beer.
If you are brewing a lower gravity session beer, then by all means, boil up some water and top it off. The idea there is to have a flavorful beer that will not fill you up or get you drunk, so if there's a bit less alcohol, no problem with that either.
IMO, the biggest thing though, is that you MUST boil it or treat it to remove chlorine. I would bet that the vast majority of the off flavor phenolic beers that I judge in competitions are made by newbies who simply top up with tap water, or if they are boiling, their water has chloramine in it which boiling will not remove.
In those cases use a bit of campden tablet to remove the chlorine. Campden tablets are about the size of an aspirin and they cost only about two or 3 dollars for a package. 1 tablet is enough to treat 20 gallons of water, so you can even use a pill splitter to divide the tablet down even further. it is well worth the minor cost and your beer will much less prone to phenolic off flavors if you do so.
"Lunch Meat. It's an acquired taste....." -- Mylo