There are two simultaneous celebrations going on, but he player cannot attend both. Affection can be raised with more females at the Rose Square Dance compared to the Inn.
Rose Square Celebration
Upon entering the square you'll see the young villagers talking at each other. Speak to any of the girls to ask one of them to dance with you. If their affection toward you is not high enough, they will decline the invitation.
The Inn Celebration
Festivities at the Inn consist mainly of adults, drinking and eating. Most everybody will want to drink with you. Drinking too much alcohol will make you pass out, triggering a cutscene where you wake up on your bed.