If you are mainly going to use it to target calicos with lures such as swimbaits you want a rod that is fairly stiff. There are two reasons for this: 1) Pulling the fish away from cover as quickly as possible (you'd be amazed how fast a 2 lb. boiler calico can rock you) and 2) to get the hook set in deeper water.

Sounds like the problem with your rod is not the action but the sensitivity of the rod. I'm not too familar with the Calico Special rods but the Shimano Calcutta XFAs (in my opnion the best swimbait rods you can buy) are very stiff and yet extremely sensitive.