Social Network Technology

We all know what the real value of social networks is - the data.  The connections between individuals and other entities.  The micro updates, the kind that share little tidbits of information.  These things, aren't all that valuable on their own.  In fact, they produce a lot of noise.  The real value in social networks, is the aggregate data, the information we cannot see at a glance.  I think the ramifications of this information, the quest for it's possession, can be attributed to the current social bubble.  The prospect of mining this data is tantalizing - of course anyone with the necessary means will pay top dollar for it.  But is it just the data they're after?  There must be some valuable technology here, right?  Something is responsible for organizing all this information, for providing the lay user with tools to say what they want, when they want, and more importantly, as much as they want.  Data, data, and more data.  It's where it's at.

Why?  Why is the oodles of social information so important?  Statistics.  Competitive edge.  The collective opinions of consumers.  This is why we're looking for a better way to extract meaningful information from all this social noise.  Social noise.  I like that term.  It makes me think of riding home on the train and a group of over-galvanized teenagers hop on board.  Its like I signed into Facebook for three stops.  I filter it out, the best I can anyway, by reading, or listening to music.  This filtering is relatively easy to achieve on a subway car with few people - not a lot of noise.  But with social networks, and their big data, it'd be like standing in the middle of general admission at an Iron Maiden show with my headphones on.  Kinda difficult to filter that out.

It's technology like this - separating noise from meaning - that social networks have inspired.  It's like NASA, and the various spin-off thechnologies they've developed.  Going into outer-space is an enormous undertaking, requiring thousands if not millions of smaller technological solutions.  These smaller innovations probably never would have seen the light of day if not for the desire to visit distant worlds.  This is how I see the role of technological innovation in the social realm playing out.  We've got a pile of data, and we're only discovering now the interesting things we can do with it, other than the original use case of social interaction on the web.

Take Google for instance.  They've already been doing R & D for a long time now, despite the fact that labs is winding down.  They started doing R & D as a consequence of having more data than anyone previous.  The result?  Stuff like Docs and App Engine.  Could Google have hammered out these products had they not been the worlds largest, most relied-upon search engine?  Probably.  But would the idea to build such products have been prevalent if they didn't build such technology to support their web index?  See, data at this scale inspires new technology, innovative new tools that have wider applications than for what they were originally designed.

Obviously Google doesn't have the last word on innovative technology with regard to big data.  Big data is becoming the norm.  It seems that today you can't be taken seriously unless you own and operate multiple data centers.  Sure, the information were storing is interesting, I have no doubt about that.  Some insights about social information come naturally, without the aid of technology.  Even in the past three or four years, I'm sure there have been a few light-bulbs popping up while digging around in the ever-connected social landscape.  What I'm really looking forward to is not so much how much data the social domain can accumulate - they've already flexed their data center muscles and there is no end in sight.  I'm more interested in the spin-off technology these social networks are going to give to the world - hopefully in the form of open source software.

The Social Protocol

What are social networks? They're websites people from around the globe use to communicate with one another. What is a website? They're a bunch of related resources linked together, and requested by web browsers. How are resources requested and returned? The browser sends an HTTP request to the website, and displays the result. This is how the overabundance of social networks work in their most basic sense – like websites do. Users around the globe, social network or otherwise, are connected via HTTP. I wonder, is HTTP an inherently social protocol, or does it appear that way because of the social phenomenon on the web? There are some interesting properties of the protocol that give it the appearance of being inherently social.

Just in case you don't think social networks are pervasive in the web space, try visiting any major site. You probably can't find many that don't have social network buttons embedded in the interface. The idea with these widgets isn't to build more incoming links. No, the idea is to be social about the subject matter. To talk about it. To rant and rave about it for that matter. Being social is about spawning new ideas through collaboration. Whether the majority of the discussions that take place over social networks add anything of value is subjective but nevertheless, powerful. We have networks of opinions we can traverse and mull over if we're so inclined. We even have tools that will search the these opinions, assessments, and wild theories, extracting the stuff we want. Think about the way a search engine works on the web - the same idea, only the data set we're querying is created by social networking tools. The data is linked together the same way the rest of web is, discussions about sirloin steak are probably linked to discussions about seasoning salt. Likewise on the rest of the web, only its pages that are linked, not discussions.

The world is a big place. The advent of the web made it an order of magnitude smaller. The web was designed for connectedness, hence the name. Collaboration, asking questions, getting answers, expressing opinions. What exactly is sent over the wire? We don't want to send raw text to our collaborators. We need content to be readable. We need to identify headings, and images. We need to emphasize certain points of interest in our content. A web browser parses a markup language and presents formatted information to the receiving user. This is the ML in HTML. Hyper-text is text with links to other URLs. The user supplies the link, and they're taken to the new location on the web. Addressability, a formatting language, a web browser that does all the underlying work. What else do we need to communicate over the web in a meaningful way?

Fast-forwarding a little, back to where social networks are the driving force of the web. How did the social phenomenon start? We used to have websites for sought-after information. Good luck finding an answer to your question should a web page dedicated to the topic not already exist. Of course, forums solved this problem originally, but there was something missing. Something that needed a personal opinion couldn't be done here. However, for general topics, forums are a great way to share knowledge. Opinions are expressed, knowledge conveyed, and not only is my question answered, but the thread lives on and helps thousands of people in the same situation I was in two months ago. Social networks took the concept of forums, groups, threaded-discussions and changed two things. First, you're no longer having a threaded-discussion with people you don't know. You're talking about stuff with friends, colleagues, people you've allowed access. This is your social circle, and your supposed to value their opinion over that of anonymous entities on the rest of the web. Second, the idea of a discussion being centered around a topic as been largely diminished in social networks. Essentially, in social networks, the user is the topic. Everything else is centered around the what they decide is relevant.

Back to the pre-social network web, HTML pages, web browsers, and the glue holding it all together – HTTP. The hyper-text transfer protocol is how we get from one page to another, load the page initially, and interact with the servers that store these pages. More importantly, HTTP gives the web a unified interface for these interactions. Browsers send HTTP requests to a location, a URL. An HTTP request has a method that tells the server what to do with the resource when it receives the request. For instance, if I want to retrieve an HTML page from a given URL, I send a GET HTTP request to that URL. GET means retrieve the HTML page and send it back to my browser. Say I'm on a site the lets me review cars. I sill out a review form and submit it. When I do this, I'm posting a new resource. For this I use the POST method. These two methods are the most common. When you're browsing the web, you might visit 50 to 100 pages. All of these requests are GET requests.

There are other, less-common HTTP methods as well - PUT, and DELETE. These methods also modify resources. If I want to delete a web page, I send a DELETE request to the page URL. If I want to update an existing resource, say change the text, I send a PUT request. Together, these four methods represent the unified behavior interface of the web. These are the actions of the web, create (POST), read (GET), update (PUT), and delete (DELETE). 

 

The social networks of today use HTTP because they're websites. HTTP has a unified interface, a small set of methods that tell the server what you want to do with the resource. Social networks generalized the concept of discussion topic and externalized their tools that allow readers of the web to point social network users to that web resource. Just like the TCP/IP protocol is general, you use it to send or receive data over the Internet, websites are general, you visit them seeking information or to perform whatever task you need. Just like HTTP provides a way for browsers to work with resource representations, social networks provide a way for users to connect with one another, regardless of the topic.

URLs are essential for sharing things on the web. Users point their browsers to URLs. In return, they get whatever the person on the other end is sharing. Generating new web resources, HTML pages and URLs is very easy today. Not just creating new resources, but managing their entire life cycle is trivial for most people. Social network or no social network, web applications make it easy to post new content, modify it, or get rid of it entirely. This is the social aspect of HTTP - the unified interface of GET, POST, PUT, and DELETE. Sometimes, you're required to supply specific parameters in your post to create a new recipe. You might need to specify a sorting column when retrieving a list of hockey teams. The question is, how do we get these parameters and how do we know which method to use?

Most resources on the web are simple. That is, they're meant to be read. This is why the default method used by a browser when pasting a URL is GET. The web is just like the real world in that you need to read before you write. This means you head to a website and see what it does, who owns it, does it interest you, is it safe, etc. Only afterward do we decide that we want to provide our own insights, thoughts, images, video, status, etc. These are objects we post onto the web using a URL endpoint. We update these objects later on by altering their attributes and putting them back. When they become irrelevant, we delete them. Or they're deleted for us. The work-flow, read about the service, post something, update it, is all derived from the end user content. The content dictates which which direction to take, which HTTP method to use.

To get a better idea of how HTTP is the ideal protocol for social work-flows, we need to think about RESTful web applications. That is, hypermedia as the engine of application state. Resources on the web don't have any state in the web browser. This means that what you're viewing right now is only a copy, a representation of the resource. The real one is on the server. Your web browser, being the application, needs direction on how to proceed, which URL to visit next, which method to use. Links do a good job of this. All the user needs to do is think "I want to go here" and click the link pointing to the new location. The browser says "GET me resource xyz". Now I can read the content and make a decision, what I want to do next. In this sense the user is the application. Imagine a different kind of web browser, one that doesn't care about presenting content to the user, one that only cares about making decisions a human would otherwise make. This kind of browser would maintain application state by continuously reading directions sent back from resources on the server and making the next move. There are, however, two things that this browser would struggle with.

The first, how does the browser know what it's options are? How does it know what it's next potential moves are, the URLs and expected methods. It would have to do what a human does, read the content in order to decide what to do next. So our new browser will need a representation it understands, XML for example. Inside the XML, we have content, probably related to the application domain our browser was built for. We also have link fields embedded in these server resource representations. These links aren't like regular links, they also have the HTTP method we're supposed to use. The browser then knows what method to use, it knows whether it is posting a new object or retrieving something that exists already.

The second problem, how does the browser know what the expected parameters are for any given resource? If a list of account transactions supports a page size, how does the browser know about it? The answer in HTML browsers is that the page designer implements a page size selector. The human operating the browser makes the decision by choosing an appropriate size. Our new browser has no luxery of human thought, only resource representations, URLs, and HTTP methods. They can only help guide our application if they provide the full extent of what can be done, how else can our application make an informed decision? It can't very well create new objects without knowing what fields are required. 

 

One option for these issues is an HTML form. Forms can store both the URL and the method required for any resource. Additionally, the input fields within make it easy for our browser to learn what the acceptable parameters for any given resource are. It doesn't need to make guesses by attempting to parse natural language, it simply reads the language it understands and acts accordingly.

The HTTP protocol itself isn't so much social as are the representations that HTTP clients interact with. If you visit someone's profile on a social networking site, that is a representation of what that person is doing, thinking, attending, reading, whatever. Based on what the visitor sees here, they make their decision on what to do next. Social networks make it so easy to hop around inside, jump from profile to profile, event to event. This is because they link to one another. Hypermedia. The engine of application state. If you update your profile, you're probably not sending a PUT request. If you delete a photo, you're probably not sending a DELETE request. This doesn't really matter because the ideas that make social networks social came from the web, refined and put in a confined container. HTTP is the interface that makes the web social.

Why Use Email

Coding horror has a nice entry on why email is counter-productive. It takes a different view as to why this is so. Rather than assume that it is the need to respond to emails that is the counter-productive part, the entry states that it is the constant need to see if anything new has arrived. The chances of any important information arriving via email are quite slim. You might get one or to useful emails per day. So what is the contents of the other emails that arrive daily that you don't care about? Maybe smaller pieces of information that might be of some significance in the future, but not today. And being interrupted with these irrelevant pieces of information is what is counter-productive.

So how to deal with these irrelevant pieces of information? There are always social networks. Social networks are feature reach and are certainly more than common these days. But I find that even moving these smaller pieces of seemingly irrelevant data to social networks as the delivery medium does not solve the problem entirely. Sure, this would clean up your inbox and thus eliminate the need for you to constantly be checking it. But what about that nagging feeling that you are missing something on your social networks? They do suffer from the same productivity problems as email does. Something else that might be taken into account when considering social networks as a delivery medium for sending simple messages is the fact that they support many more features. These other bells and whistles can be even more distracting than email because email does one thing and one thing only.

Using a wiki for exchanging most types of information, both big and small pieces, often makes more sense. Larger, more static pieces of information can be created as wiki pages. Smaller pieces of information can be created as tickets in project management systems, most of which have wiki support. A threaded dialog can then be created on these tickets. The conversation is than public and persistent. It is also search ready. They key point here being that tickets and wiki pages for that matter, are not limited to a software development context. They are much more general purpose and do not require polling on the recipient's behalf.

Social Network Paranoia

The production of content on modern social networking sites is produced almost entirely by it's users. Additionally, this content accounts for a large portion of the content on the web. Jeff Atkins talks about the sharecroppers of the web. It is both interesting and terrifying to think about. People are working for free to benefit the corporations that enable this content to be published. Although the scary implications of leaving your data in the hands of others is nothing new, the reasons for doing so is interesting. People generally use social networking sites because they enjoy doing it, not because they want to occupy some tiny portion of the content available on the web.

But why does it matter that these big corporations are gaining from other peoples free labor? what else do you have to lose if all your data one day disappears? The time investment that could have been spent on yourself. Although, it is highly unlikely that any of these mainstream social networks are going anywhere soon. So, the odds of losing everything you have invested are currently not all that large.