It's nothing that hasn't already been said to him before--in fact, it's pretty tame in comparison to what's usually thrown in his direction--so while he might not have wanted to hear it, it's something he was quite able to take in stride. Rory just sort of hunched his shoulders inwards to see if he could make himself smaller, and tried to scoot past without too much trouble.

"Sorry," he repeated softly, and just in the hopes of getting out of the situation without anything getting slammed headlong into any walls, added, "I'll stop looking at the floor."