Just file the claim. You don't have to go to fedex office, the claim form is available online or just call fedex and you only need to fax the information. Don't be so lazy, let fedex handle it. If fedex think it's a fraud then let your buyer know that fedex said it's a fraud. See how easy that is rather than passing judgement that he's trying to scam you?
I had to file a claim for one of my buyer that fedex said it was delivered to him. Turns out the driver swiped it and said that it was signed by my buyer, fedex knew that after the investigation. You never know who's telling the truth so let fedex handle it. You have to help your buyer because only you can initiate the claim and trace. So, just do it