The "Safely Remove Hardware" icon also shows up if you have a USB printer or when you insert a USB thumbdrive or any other USB device also.
If you really don't want it to ever show:
Right click an empty spot of the Taskbar and select Properties.
At the bottom of the dialog put a check in the box next to Hide Inactive Icons.
Also click the Customize button.
Select - Safely Remove Hardware - from the Current Items list and click the dropdown arrow and select - Always Hide. Click OK and OK again.
I hope this helps.