You're not going to get that floor smooth again without some SERIOUS elbow grease. Concrete grinder MIGHT work, but they may also not do well with the glue. It will just gum up the pads/knives. I would just install a subfloor over it like Dricore. That's what I ended up doing in my basement as there was nothing I could do with the floor (glue from a previous carpet too). It was too rough/dirty to paint, and because of the roughness it was not really cleanable. I could vacuum, brush, use compressed air, do anything, it was always dirty from dust etc. So putting a thin layer of self leveling concrete or tiles or something would not have held together very well. I had tried a small area with a heat gun and was able to somewhat scrape the glue off, but to do the whole basement would have been an insane job not to mention the cost of running that gun for days on end.
If it's not a moist area you could probably skip the drycore and do plywood right over. You can either use a ton of glue and hope for the best, or nail it in. For plywood type products I prefer regular nails than powder actuated, what I do is drill a hole with a masonry drill bit, drop a couple pieces of rebar wire, and nail it in with a regular hammer. It ain't coming out.