It starts getting creepy or dangerous when people start interfering with your life and causing you harm with personal data used against you. This hasn't happened in the 20+ years of public internet data collection, and I don't see it happening anytime in the future. Unless someone has a vendetta against a particular person, the resources to cause this type of harm would be enormous.
Advertising in general isn't a terribly friendly thing. Its goal is to manipulate you into doing something you would otherwise not be inclined to do. Good old DeBeers: One of the best advertising campaigns ever. "If you don't receive a diamond, your man does not actually love you. And here's how much he should spend: $$$" So now we buy shiny crystalline carbon gems because a company told us that that we should.
But in general, people are easy to manipulate. Stage magicians rely on this. So do politicians. (At least one of them makes an honest living though. Har har.) And the more you know about them, the better a job you can do. Gerrymandering is another example of this, and it's gotten more sophisticated over the years as you're able to generate statistical heat maps of where your voters are.
Companies with big advertising budgets can use things like eye-tracking technology to see which things a potential customer is most likely to look at, and adjust their ads to be more effective/manipulative. Supermarkets can employ vision systems with their cameras to monitor where people are most likely to walk. Smart TVs keep an eye on what you're watching, how long you watch it, and some will even inventory your files on any attached network devices; all of this data is then offered up to advertisers.
Statistical analysis, when done with a good sample size and plenty of data, can start to eat into what you think is free will. Eventually, the computer can say, "Yeah, you have free will, I guess. But I've determined that there's a 97.8% chance that you're about to do <action>."
"Harm" as the benchmark? Estimates for the amount of lost productivity due to spam is
billions of dollars a year, even tens of billions. There's a small industry dedicated to attempting to filter it out. Time is wasted sifting through junk. Emails are lost to overzealous filters, leading to additional lost time.
Billions of dollars.
That's wiping out the economic value of quite a few lifetimes' worth of work. Is that not harmful?
Advertising isn't much better. It's just another way of attention-whoring for a business. Now just imagine if advertisers could really get, statistically speaking, inside your head.
People keep saying this---WHY is it creepy that a piece of software that learns to present relevant and useful information around your habits?. I find it incredibly useful. I think people overestimate what can be done with this data.
It's not just software that knows this stuff. People look at the data as part of their jobs. And some people might gain access to that data - people who perhaps
shouldn't have access to it.
We're also creatures of habit, like it or not. Advertisers would absolutely love to be part of your habits. Or hell, if they can convince you to
start some new habits that include them, that'd be even better, right?
There is a
lot of money to be made from acquiring very detailed consumer data - therefore there is motivation to gather as much as possible. The more that's gathered, the greater the probability of abuse.
This showed up in another thread:
Hmm, your sleep cycle seems odd.
You don't need medical help though.
You need homeopathic remedies, or some other kind of crap. Either way, the right solution just happens to be what some shady Internet company is selling, and they
just happen to be datamining your sleeping patterns.
Or your Google-owned Nest thermostat notices you awake at an unexpected time, so it recommends that you go get something at the nearest 24hr McDonalds.
That's just advertising though.
Plenty of other people would love your info. Car insurance companies would love to know more about your driving habits. Some already offer discounts if you agree to put a recording device in the car to monitor how you drive. Health insurance providers would love to know what you eat, how much you drink and what you drink (urinary tract health), how much you exercise, how much you sleep, or if you've ever researched health problems online.
(I suppose that I'll add that I would love to have this sort of helpful technology -
IF it could be done without the risk of abuse.)