Since I only use one account, security wise there is no difference whether I have a password set or not, correct?
Simple question, complicated answer!
In terms of using a computer like a normal user would for desktop applications, no (assuming there isn't a concern regarding physical on-site security).
In terms of the computer being accessed via the network, maybe yes. If you want someone to access a Windows share over the local network without a password, then beware three scenarios that spring to mind it could allow people outside the local (presuming Internet-connected) network:
1: Most local Internet-connected networks have a private IP range provided by NAT by a router. It's possible for an Internet-based attacker to circumvent the protection offered by NAT to access information on your local network. If you have a share without a password, that's obviously at risk in that scenario. I would consider this scenario to be more likely if you have data that someone could conceivably intentionally target (though I suppose some tech-savvy black-hats could come up with a ransomware attack that involves accessing via SMB and encrypting everything accessible).
2: Same as the first point but through wifi (e.g. someone outside the building).
I can't remember what my third point was going to be!
If we're talking really casual use of SMB (ie. not remotely private/sensitive information at all), then passwordless logins might be worth considering. However, it's not a great trial to use the credentials for the user who typically signs on to a given workstation as the network access credentials too, then when the computer tries to access that share for the first time it'll throw up a login box, type the details and save them. For casual users, that's the way I normally set it up.
One other thing, SMB really isn't worth a shit in terms of hardened network security. Never leave an SMB share open to Internet access intentionally, it's not designed to take the pounding that it would get if bots on the net could see it. It would be a heck of a lot more secure to use some kind of cloud storage provider instead (e.g. Google Drive, OneDrive, Dropbox, Cloud Drive, etc).