Kayaking
There are plenty of beaches through the coast that rent kayaks (typically per hour)
This links is a guide to various routeshttp://www.ibiza.travel/img/descargas/35_en_kayak.pdf
As an example, From Santa Eulalia you can kayak NE up the coast, calling in at the various beaches (and bars), or SW down the coast to Cala Llonga. Just past the Cala Llonga bay the water has made loads of caves out of the limestone cliffs .. take your snorkel for great caves and fish.
–STOP PRESS–
On 1 October 2013 two british tourists (Ang and Chris) were seen obstructing the Santa-Eulalia ferry by kayaking into the international shipping lanes
Later the same day they were stopped by the coastal police and towed out of danger when seen heading towards North Africa (for some strange reason) and interfering with the Tornado catamaran world cup racing week taking place off the waters of Santa Eulalia