Exactly. In the case of connectedness, you have to traverse all paths. You're looking for a global property of which objects connect to each other. Any particular path becomes irrelevant in that case, and so there is no weighting of paths or distance/weight estimates.
For pathfinding with A*, there may be multiple paths, and there may even be multiple paths with the same minimum distance. You don't really care about alternate paths though. You just want one path of minimum distance between two specific points. In that case, it makes sense to end early once you have a path of minimum distance.
As a fun little aside, the proof that A* returns a path of minimal distance depends on a property of the heuristic function that estimates remaining distance. The heuristic function must be admissible, which basically means it needs to be optimistic about the remaining distance, or rather, than it never overestimates the remaining distance. If you choose a pessimistic heuristic function which overestimates the remaining distance, A* can return a path which is not optimal, since it doesn't prioritize searching in the direction of an overestimated distance, and so might not find a path which is actually shorter than the estimate. In that case, A* is still able to find a path when a path exists (just maybe not an optimal one), and correctly determine when no path exists.
