Hi
I'm afraid you're unlikely to find anyone who will hire you anything in the port unless you've got a licence. Nortinort who hire out the boats by Stay insist on seeing a current RYA2 and won't let you near a boat unless you've got one.
Sensible really, because it's a busy port with lots of power and sail boats so you could get into some serious difficulties if you don't know the basics. (Try mooring at Formentor when you've got the glass bottom boat coming up alongside you!!!)
If you've got two days to spare during your holiday you could spend two (exhausting) days getting your licence at PP. You'd then be qualified to hire anything up to 10m which is big enough to take 10-12 people around the bay and out to CSV etc.
There's a brit who runs a company called Aurora based between PP and Alcudia. From memory I think it costs around 250 Euros (but to put it in perspective, it'll cost you 400 Euros to hire a 5.7m boat for the day). I think he advertises on the site. So if you're interested, give him a call.
Good luck. And have fun!