Select Properties on the folder, Choose tab Security. Then remove Everyone and add only the user(s) you want to give access and specify their right to the folder.
NB! To change the permissions of a folder you first have to remove "Allow inheritable permissions from parent to propagate to this object"