The strict answer is that when the compiler sees...
if (expr) { }
... it evaluates expr to see if it is true, and if it is enters the conditional block of code. Your expression...
(function(a),false)
... is perfectly legal. The comma operator is used in C++ to separate expressions, so these two expressions are evaluated in order of precedence, i.e. left to right. Therefore the effect of the combined expression will be to, in order:
a) execute the function, passing a, and discarding the return value
b) return false to the if operator