If there's one mouse, there are 10 more you've not seen.
Get some traps, block the holes in your walls (under the kitchen sink is a likely place) and check off places where they might be storing food (we have an IKEA bookshelf that the mice were storing misc seeds and whatnot in the front corners underneath).