If you read Plato you'll see how Socrates uses a method of conceptual analysis based on Parmenides.
Then Aristotle's logics basically create a relationship between what we would today think of as practical science and theoretical science.
Of course, mathematics is going on at the same time.
But to answer your question--it's considered philosophy because people like Bertrand Russell, WVO Quine, Frege, and other famous peeps made great advances there, and discussed relationships between math and logic, including set theory.