It can cause contact dermatitis and flushing with water will help to an extent. It can cause damage I've seen it in the ER.
Pepper spray?
Just saying that if that's your basis for comparison, it's not the same ballpark as a little potato chip residue on a finger.
I have yet to see anyone have a visible skin reaction to any pepper.
Contact dermatitis can be a severe
allergic reaction, but un-concentrated capsaicin is not going to cause a severe (medical attention necessary) irritant reaction, even in someone with sensitive skin like mine.
Water flushing is common-sense medicine for chemical eye injuries, but in this case I fear will make things worse.