Social Network Technology

We all know what the real value of social networks is - the data.  The connections between individuals and other entities.  The micro updates, the kind that share little tidbits of information.  These things, aren't all that valuable on their own.  In fact, they produce a lot of noise.  The real value in social networks, is the aggregate data, the information we cannot see at a glance.  I think the ramifications of this information, the quest for it's possession, can be attributed to the current social bubble.  The prospect of mining this data is tantalizing - of course anyone with the necessary means will pay top dollar for it.  But is it just the data they're after?  There must be some valuable technology here, right?  Something is responsible for organizing all this information, for providing the lay user with tools to say what they want, when they want, and more importantly, as much as they want.  Data, data, and more data.  It's where it's at.

Why?  Why is the oodles of social information so important?  Statistics.  Competitive edge.  The collective opinions of consumers.  This is why we're looking for a better way to extract meaningful information from all this social noise.  Social noise.  I like that term.  It makes me think of riding home on the train and a group of over-galvanized teenagers hop on board.  Its like I signed into Facebook for three stops.  I filter it out, the best I can anyway, by reading, or listening to music.  This filtering is relatively easy to achieve on a subway car with few people - not a lot of noise.  But with social networks, and their big data, it'd be like standing in the middle of general admission at an Iron Maiden show with my headphones on.  Kinda difficult to filter that out.

It's technology like this - separating noise from meaning - that social networks have inspired.  It's like NASA, and the various spin-off thechnologies they've developed.  Going into outer-space is an enormous undertaking, requiring thousands if not millions of smaller technological solutions.  These smaller innovations probably never would have seen the light of day if not for the desire to visit distant worlds.  This is how I see the role of technological innovation in the social realm playing out.  We've got a pile of data, and we're only discovering now the interesting things we can do with it, other than the original use case of social interaction on the web.

Take Google for instance.  They've already been doing R & D for a long time now, despite the fact that labs is winding down.  They started doing R & D as a consequence of having more data than anyone previous.  The result?  Stuff like Docs and App Engine.  Could Google have hammered out these products had they not been the worlds largest, most relied-upon search engine?  Probably.  But would the idea to build such products have been prevalent if they didn't build such technology to support their web index?  See, data at this scale inspires new technology, innovative new tools that have wider applications than for what they were originally designed.

Obviously Google doesn't have the last word on innovative technology with regard to big data.  Big data is becoming the norm.  It seems that today you can't be taken seriously unless you own and operate multiple data centers.  Sure, the information were storing is interesting, I have no doubt about that.  Some insights about social information come naturally, without the aid of technology.  Even in the past three or four years, I'm sure there have been a few light-bulbs popping up while digging around in the ever-connected social landscape.  What I'm really looking forward to is not so much how much data the social domain can accumulate - they've already flexed their data center muscles and there is no end in sight.  I'm more interested in the spin-off technology these social networks are going to give to the world - hopefully in the form of open source software.