I am fond of dancing to music

This sentence is correct

of is a preposition, followed by a noun or pronoun as the object, so dance needs to change It becomes the gerund form of dancing.

dance with sb. Dance with someone; dance to music and dance to the beat of music. So use dance to