it should be ok, as long as your firewall is blocking them.
there's always someone port scanning somewhere, it seems... at work, i always see port scans from almost everywhere in our firewall logs.
just double check to make sure your firewall rules are ok
you could also traceroute that...