Most of the time, yes, but sometimes a part of the golf course - say, a seasonal creek, or a ditch - might be designated a water hazard even though there is rarely (or never) water in it.
Golfers can try to play out of a water hazard, and sometimes that's easy to do. If a ball crosses the margin of a water hazard (designated by the yellow stakes or yellow lines, which are themselves considered part of the hazard), but is not actually in water, it might be easily playable.
If a ball is under water, however, it's almost always best to take the penalty and put a new ball into play.
The penalty is one stroke. There are two options for putting a new ball into play. One is to return to the spot from where the previous stroke was played and hit it again. The second, and more commonly chosen option, is to take a drop.
When a golfer takes a drop out of a water hazard, he must drop behind the point where his ball crossed the margin of the hazard. The drop can be made at any point, as far back as the golfer wishes, so long as the point where the ball crossed into the hazard is kept between the point of the drop and the hole. A ball is considered in the hazard when it lies within the hazard or when any part of it touches the hazard (remember, stakes and lines are themselves part of the hazard).
Rules covering water hazards can be found in Rule 26.