What do you store on it?
I store files which come from all different type of sources.
Irrelevant. I could purposefully download gigabytes of malware and stash it all on the server and it would pose no more threat to that server than if all I did was download plain boring text files.
Downloading
anything is harmless. Data is inert.
Execution is the thing to watch out for, and that's why, if you're using the server as a server and not as a client, there's nothing to worry about. And when something is executed, it hoses the system that executes it, not the system that stores it. So running a piece of malware on a network drive doesn't hose the system that hosts that network drive, but instead the system that did the running. (That having been said, once the system that runs the malware is compromised, it could be used as a springboard from which to attack the server, but that's a possibility regardless of whether the malware is hosted locally on the client or remotely on the server.)
People have taken really opposite positions: "Yes, you should use AV-ware; No-- you don't need it and shouldn't."
Except one of those positions is couched in ignorance and the other is not.
Again, the threats faced by a server are very different than those by a client, because if the user is not doing things, then that means undesired code execution on a server comes from the exploit of security holes, which, if you keep your server OS and software patched, are 0day affairs, and AV is f---ing useless against those kinds of things. And there's a non-zero cost to AV, and I don't mean financially, in the form of false-positives that more often than not do more damage than the (very limited set of) things that they do protect against.
However, something like
EMET would help in the kinds of attack scenarios a server would encounter.