The part you find troubling is likely the most exciting part---once harnessed and turned to an individual's utility. "Something that knows more about me than myself" was the dream of the 'personal digital assistant' craze in the '90s, but the technology wasn't there yet. It's here now, but it leverages network advantages and scalability to work its magic.
I imagine the synthesis of these forces in the future is technologies that aggregate, but that the indivudal can buy into. Do you want help deciding what to wear based on the weather or time of day, beyond what your assistant can do for you? Then there's a collective data channel you can opt-in to joining, and you'll trade some of your personal private information for access to a digital "collective unconscious" of aggregate data. I liken it to the 23andme approach of collecting genetic profiles.
There is a difference between "knowing more about you" and "helping you find out something new about you". The entity that knows more about everyone than they themselves do concentrates power. A tool that you use to find out something new about yourself, only for yourself, does not.
I imagine the synthesis of these forces in the future is technologies that aggregate, but that the indivudal can buy into. Do you want help deciding what to wear based on the weather or time of day, beyond what your assistant can do for you? Then there's a collective data channel you can opt-in to joining, and you'll trade some of your personal private information for access to a digital "collective unconscious" of aggregate data. I liken it to the 23andme approach of collecting genetic profiles.