Pondicherry is one of the former French colonies and you'll be mistaken for thinking you weren't in India even now. The city is full of quiet, clean, shady cobbled streets with colonial-era townhouses. Quiet beaches stretch north and south from town, perfect for sunrise strolls. Although, the city lacks tourist destinations as such, you won't feel the absence. The place is full of excellent food joints that serves French, Italian and what not! During the course of my 3 day stay, I tried 8 restaurants and all of them turned to be better than expected.
If pizza is your bae, Cafe Xtasi is the place you should seriously consider. For an authentic French breakfast, do consider Baker Street. For that French colonial, ancient era kind of feeling (without breaking the bank), you should definitely checkout Le Dupleix.