Sand's is a pretty popular fly fishing spot. It's up north, right on coal oil point. Another spot is the sandbar at the end of the breakwall for the harbor, and the base of the pier as Frank suggested. That's pretty much the extent of my knowledge about fly fishing around here, so maybe somebody else has something to add.
-Brian