It will be a new networking platform that builds on today’s world-wide web and makes possible new generations of more powerful and useful applications.
My vision is of a next-generation web that is just as simple and flexible as web 1.0, but more interconnected and powerful.
At its core, it will incorporate a universe of connections that reflects the real-life links between people, organisations, services, products, web sites, and other entities.
Picture this
- A browser that instantly shows you the content you’ll find most interesting
- Search engines that return fewer, better results - every time
- A marketplace that always tells you the best products and services, and lets you advertise anything you like
- A world with no more spam, phishing or online scams
- Being able to access your contacts’ current details all the time, without effort
- and being able to find the right contact wherever you are
I’d like to share my vision for how all this will work, which is low-tech, doesn’t rely on any new technology, and is nearer than you think. And I predict that Yahoo! will probably be the company to make this happen, in the next 2 years.
An evolutionary step
The web has given us some great applications that let us do some amazing things: communicate, publish, share, and shop with greater ease and speed than ever before. People, content, and data are more fluid and connected than ever before.
However, the world-wide web has now reached the limits of the platform. These great applications that we have are still fragmented and inconsistent when they could be interconnected and co-operative.
The new web will be a platform that simply joins up all the functions and data with a new network of connections below the application level, enabling a new generation of applications that will make life easier, more productive, safer. and more fun.
Searching today’s fragmented web
Think how often you use an internet-based application to find something: content, recommendations, shopping, services, contact details, research, comparison…
Remember: “The answer is out there!”
The right answer is nearly always out there - somewhere.
- That person’s contact details do exist online
- There is a piece of software that does just what you need
- There is a company that sells the item you need at the right price
- Someone is trying to sell the sofa that’s perfect for your room
Most web applications are trying to match information to where that information is needed
The major problem is intrinsic to the Internet’s architecture: there is no center. The lack of center is the reason why the web has become so useful so quickly - it has enabled free, organic growth.
But it also means that any time we try to compare information, we have to try to collect information in one place - such as on a single database. That’s difficult on the web, although there are partial successes, see below:
Examples of the fragmented web
- Ebay has a huge database of people selling stuff, which it matches to people looking for stuff
- Google has a massive index of web page content, which it matches to search phrases
- Friends Reunited has a database of where people went to school
- Car Harbor is a solution (currently in development) that will match people looking for car parking with people who have free space to rent
- Loads of dating sites have databases of people wanting to find people who want to find people
- LinkedIn has a database of businesspeople’s information, and links between them
I could go on. What these applications have in common is that:
- They all try to match informaiton with needs-for-information
- They all have their own databases that contain a small proportion of the information out there
- And their scope is limited to only the data they hold themselves
In other words, they all have part of the picture.
This means they can only try to find the best answer to what you need with the information available.
It doesn’t matter how much bigger or smarter these systems get, they’re limited by the fragmented web version 1.0. Google or Ask.com will never be able to know what you really want when you search for “home run”. This is because the current web is still locked into reductionism. Because all these applications are just part of a disconnected world of data, they’re forced to reduce everything to their basic component parts.
Case study
If you use a search engine to search for “Cleaner Chesterfield”, what do you get? Absolutely nothing of any use! Google returns about 248,000 results, which are useless to me!
I don’t want 248,000 results! I one one good one!
The search engine has taken the information I provided, and it has done its best. I don’t blame Google - its algorithms are very powerful. It’s just that power isn’t enough to solve this problem.
What’s needed is a new way of looking at the problem - a new system that appreciates the context of my query, not just the words I use.
One-dimensional search
Today’s search engines take the whole rich universe of information, which contains my problem (query), and a plethora of possible answers, all floating in a multi-dimensional space of context.
What they do is squeeze that universe of possibilities into a one-dimensional list of 248,000 results.
They treat all information as equal, which is missing the whole trick, because all information is not equal - to me, here, now!
Visions of a more connected world
What we need is to connect everything in such a way that, from a single entry point, we really can find the best answer to our information need.
I’m a software designer, and I think best in terms of how solutions feel to use. So let me describe the future web through user experience stories.
I’ll describe to you my vision through the following examples:
Future killer homepage
I open my web browser. Instead of showing me the latest news, or summaries of my RSS feeds, or some combination of these things with a bit of weather and stock data thrown in, my browser simply shows me: what I’m most likely to be interested in.
How about that? No messing around, no settings and options, just what I’m mostly likely to be interested in.
How does it know?
Well, the answer is out there, isn’t it? There are things on the web today, which may be new, that I’d be really interested in if I saw them. All the system has to do is work out what those things are, from what it knows of me.
Now, there are already all kinds of data about me on the network. The problem is that, because of the disconnected system architecture, these bits of data don’t know each other. In the connected web, we simply join all this stuff up in a way that magnifies the usefulness of each bit.
Let’s think what kinds of thing we might know:
- My profile - age, location, gender, socio-economic information
- My bookmarks - in my browser bookmarks, history or sites like del.icio.us which indicate what information I’m interested in, what sites and companies I prefer
- My connections - on MSN, Yahoo! Messenger, Skype, email, LinkedIn and Yahoo Groups etc. - which indicates who my friends and business contacts are
- My words - on my web site or blog
Let’s start with just these. Say we could join them all up into a single system where all the data could be viewed and used together. What might we learn about me?
The power of connections
Aside from the raw data about me directly, we can learn quite a bit about me from the sites, people and groups I’m affiliated with. If you know my favourite sites and favourite people, you can start to build a picture of what my favourite stuff is.
(I’m not talking about any fancy fuzzy logic or AI here, this is pure number-crunching at work, and I’m not talking about any futuristic technology.)
Think of it like a heat map of the web.
We’re only interested in the things I’m directly associated with, and the things my friends and contacts are directly interested in.
Once we have a heat map of the areas that most interest me, we can look at key hotspots on the web to find the hottest content according to my profile for today.
The things I’ve most recently expressed an interest in (such as new trends, new sites, new contacts) may be hotter, and these things might “cool off” over time.
Similarly, things that I’ve repeatedly connected to (through regular visiting), can be interpreted as more important.
The killer homepage will easily be able to predict the things that I’m most likely to find interesting: to do with interaction design, web design, the environment, politics etc. just by looking at a heat map of where my interests lie, and what’s hottest in those areas today.
We’re starting to see new things that become possible just by joining up our data.
A tool for collecting connecting information
Check out the toolbar below (which I’ve mocked up). It already has most of the tools we need to power the connected web.
- The “Rate” tool and Bookmarking let me record pages I like (although the browser can also simply record which pages I’ve been interested in by what I’ve looked at, for how long, and how much I scrolled)
- The Buddies tools will show me my list of contacts, tell me who’s online, and let me review the strength of links I have with them and my permission settings (see Contact Manager later).
- The green flag tells me that the community thinks the site I’m on is safe (see Security ideas later)
- “Auto-login” is another usability and security feature that saves me time and prevents phishing (see later).
Future killer search
Today’s search engines are not great at recommending the best results to a search query because they are limited in scope and reductionist in approach. They treat all information the same, however there’s an easy way to cut down the right results by half.
The easiest and best thing we can do to make searching for products and services far more powerful is simply to separate things that are offerings for products and services from everything else.
Just take all the pages with people offering e.g. fishing tackle, and deal with them on a separate Marketplace system (see later), and are left with everything that’s simply talking fishing tackle.
I’d guess that about half the searches done on the web are ultimately looking for solutions (eventually in the form of products and services). In other words, we’re looking for something that solves a need we have. The rest of the searches are for raw information.
The problem with most of today’s search engines is that they treat everything as raw information.
By removing the whole supply/demand dimension into a separate Marketplace, we improve the quality of searches for product by maybe 1000%. Search engines can exclude all information relating to products and services, and can concentrate on doing what they already do best - matching search terms to raw information.
However, once we start joining up our world, the tasks of figuring out what content is most relevant to a particular search become so much easier!
Remember what we said about predicting what content will most interest me for my killer homepage? Search uses the same technology.
- What have I looked at recently?
- What pages have I spent most time on?
- What content have I been reading?
- And what have my friends, contacts, and colleagues been looking at (people who are likely to be in similar businesses to mine)?
This kind of search isn’t even that smart, yet it will be able to give me a shorter, better list of results.
Note: This approach is already being called “Social Search”. The key difference that the connected web concept brings is joining together all the entry points, so that it doesn’t matter what means you use to access content (RSS subscription, del.icio.us, or your browser bookmarks), all the data can be accessed in your personal networked heat map when you do a search.
Future killer marketplace
A lot of web applications exist to help people find products and services to solve various needs. In this I include all e-commerce sites, all sites that basically market stuff, consumer review sites, yellow pages and other directories.
Where they fail is that none of them can recommend the best solution to what you need. Why? Because they’re limited in scope and they treat all information the same (i.e. without taking context into account).
Now, we’ve already separated products & services from raw information. Raw info is left to our search engine. The Marketplace becomes the single universal resource for finding products & services.
All we need to do to improve searching for products and services is to record relevant data on who uses what products and services, and how they rate their experience.
Creating feedback loops
One of ebay’s and Amazon’s best features is the “web of trust”. These interconnected webs, that simply report a summary of other buyers’ feedback about a provider, are key to the new web. Where they don’t go far enough is that they’re limited only to the data on each discrete island of information - the individual site - because of problem #1, the disconnected web!
Other limitations of the current state-of-the-art are:
- The feedback systems are relatively crude. They give an overall good/bad rating, but can’t positively identify the most suitable solution out of a good bunch.
- Not everything fits into auctions or e-commerce, or even one-off transactions. For example, I can’t see my cleaner selling his or her services on ebay.
All we need to do to get a killer marketplace is:
- Create a common framework that lets anyone promote anything anywhere, and let other people search for whatever they want (Google Base, Google Classifieds and Microsoft Fremont are starting to look into this area, although unclassified ads will be more useful than classified ads)
- Record feedback whenever someone uses an offering (product or service)
- Use contextual information on suppliers and searchers
Recording the results of matches (e.g. “You looked at Brigg’s Builders. Rate your service…”) will help the system build up a useful, rich picture that will help the system predict the best matches for future searches.
On one level, you get the crude “95% positive rating” metric. However, just by keeping and joining up a bit more information, we can do a whole lot more.
Remember, instead of trying to arrange our universe of information into an arbitrary single line of best-to-worst, the goal is to predict the best few matches to a particular need in context, i.e. at a particular point in the interconnected web in space and time.
Case study revisited
The process I’ve noted above is not at all complicated, it’s just simple analysis of raw data. However it lets us predict with a good degree of certainty which cleaner out of the 30 near me will be most likely to suit my needs.
Therefore, my search doesn’t need to return 30 or 300 results. Because it’s smarter, it can just give me the top 5 to try.
Again, there’s no new technology here. It’s just existing stuff, but joined up. And it can work, easily! (I was playing with a working design for this interaction over a year ago, which solves the issues of trust.)
Let’s consider how the system will learn this stuff about me in the first place. (We can’t expect every commercial web site out there today spontaneously to share their data together!)
Future killer contacts manager
Again, there’s nothing much new here. All our future killer contacts manager needs to do is collect some of the same information we do today, and let it move about more easily.
A few things on the market today point toward what’s possible:
- LinkedIn, Spoke and others are basically databases of people and the connections between people. They enable services such as finding contacts in particular areas or in particular companies, or connecting to new people with similar business interests to yours.
- Plaxo lets you store your contact information in a single, central place. If your contacts also use Plaxo, then when you update your contact details, their copies of your info is updated automatically.
These are great and well-designed web products, however they still suffer from the general disconnection disease.
Here’s a vision of a contacts manager that’s transparent to use and does a lot more than simply record who you know.
The qualitative differences are:
- You can connect to more than people
- Connections can vary in importance
- Some connections are only valid when reciprocal
This image represents a tiny area of the connected web.
It shows a number of elements, which may be of any type, connected with simple links of varying strength (and each having its own extensible database in XML format).
How you connect to people
You’d create connections through a number of entry points, each with a similar result:
- By sharing emails
- By adding a messenger buddy
- By right-clicking/option-clicking a person’s smart-tagged link on a web page (see below)
In this example, you could be reading a blog or post by someone.
Their name is marked up with their ID account identifier, which lets your browser offer you the option to connect, with a “connection strength” that reflects the value you place on your connection with that person.
Note that this doesn’t create any connection from that person to you.
Here’s how it might work in a desktop email package (Outlook in this example).
You can connect to more than people
The power of connections is key the connected web. The reason it’s so powerful is that it represents the connections in real life. I cannot stress enough how important this point will be.
In real life, I can be connected to more than other human beings. I can also be affiliated with:
- Companies and brands - e.g. I may choose to express my connection to Apple computers, SitePoint books or my business coach Ali in terms of trust, support, and fidelity.
- Products and services - As we’ve already seen, if I rate a product or service, that’s real data that can be useful in building a picture of individuals and the links between them.
- Groups and organisations - membership or support of any political parties, clubs, societies, religious groups etc. are also connections.
- Web sites - If I bookmark and rate a web site (as good or bad), I’m creating data, which could be useful knowledge.
All this information is real and, taken together, can be used to get a quick heat map of my place in the connected web.
In the connected web, all these entities can exist alongside people. You can connect to them, and they can connect to you (recognising you as a member, family member, affiliate, subscriber etc.).
Connections can vary in strength
Most existing systems simply collect links, saying “Ben is connected to Thomas. Thomas is connected to 2,000 other people. Therefore, Ben has a million potential friends.”
This has limited use, because it doesn’t reflect the quality of the relationships. In real life, the relationships and links that connect us to other entities can be weak or strong, and also positive or negative. Therefore, links on the connected web must also reflect strength.
Let’s say I get an email from a designer working in China, who asks me some interesting things. Using my connected email client, I may think, “You know, I like the sound of this guy.” and simply right-click > Link > 3/10. We’re done! I have now expressed my link to Chen, my new contact in China. That link will be recorded on my central identity-account, where it can be used in all kinds of ways to benefit me. It means that:
- I have recorded a low level of recognition and synergy with another person
- His heat-map will affect my heat-map (a bit)
- My account can talk to his account so that I can always find his contact info (possibly), read his blog, see his homepage etc.
- Email from this guy always gets into my Inbox.
At a future point, I may change my mind about this contact, and may strengthen my link to him (if he grows on me), or set it to zero (if he turns out to be a weirdo). The important thing is that I have control.
Some connections are only valid when reciprocal
Let’s say that my friend, Chen, wants to send me an instant message.
I may set up my IM software to allow only people with whom I share a 20%+ link to make contact with me. This is fine, because I trust Chen 30%.
But some functions in social networking rely on mutual recognition, i.e. reciprocated links.
Let’s say Chen has a friend, Danny, who’s looking for a contact in Interaction Design. He may do a search on a connected web site, or through his IM client. This search will be effected through a peer-to-peer-style interaction.
Here’s how it works:
- His client contacts his ID account
- His ID account contacts the ID accounts of all his contacts (i.e. the people he trusts)
- If those account recognize him (i.e. there’s a reciprocal link), they may respond to his request. In this case, he’s contacted Chen’s account, which reveals that Chen is interested in Interaction Design. Because Chen trusts Danny 80% and Danny trusts Chen 60%, there is a 60% link between them (simply the lowest value). And because Chen allows his account to reveal general information about him to anyone with a total 10% link or more, the accounts talk, and Danny instantly sees a result: “Chen”
- Now, Chen’s account passes on the request to its contacts.
- Chen’s account contacts my account, because Chen recognizes me (50% link).
- My account recognizes Chen - with a link strength of 30% (the lowest link strength). Chen’s account says “I have a contact, value to me 60%, who’s looking for contacts in Interaction Design”
- My account calculates that the link from Chen’s contact to me is 60% x 50% = 30% (a composite connection, reflecting the remote nature of my link with Danny).
- I permit my general contact info to be passed to people with a 20%+ path strength, so my account passes my general contact info to Danny’s account, which passes it back to Danny’s software client, and I appear as a result on Danny’s search.
All the above would happen automatically in just a moment, and illustrates why some transactions in a social network rely on mutual recognition.
Future killer security features
We can use the power of the connected web to counter “un-neighborly” online activity.
Phishing and online scams
The system would work in a similar way to connected social bookmarking, but instead of marking favourite sites, smart users would click a different toolbar button to mark a site as “dodgy”.
The system can keep a central record of bad sites, and any time someone clicks a link in an email that purports to be from their bank, their browser toolbar will find the address/IP on the bad-list, and will pop up a big red “Warning! This site is NOT a real banking site!”.
Note the red flag also warning me that the community rates this site as malicious.
Auto-login
Automatically logging users into safe web sites both improves usability and safety.
If I visit a web site, my browser toolbar could indicate that the site offers automatic log-in (by being affiliated with the connected web).
Here’s how it might work:
- I click the “Autologin” button.
- The web site asks my browser for my unique ID address, and then contacts my ID account to request a new auto-login function.
- My ID account pings my authorized client software (e.g. my instant messenger), which pops up a message confirming the request.
- I confirm the request to my ID account, also confirming what data I wish to share with the web site.
- On future visits, my web browser tells the site my unique ID, the site contacts my ID account to get confirmation of my credentials, which are transferred automatically, and I’m logged in automatically.
Spam
There’s no doubt about the wasteful cost of spam, and a robust solution to the problem is needed to prevent the whole email system from grinding to a halt.
In a similar method to phishing, the connected worldwide web can keep a score of untrustworthy email addresses.
However, we can also use the connected web positively to enhance the basic blacklisting/whitelisting functionality, reducing the margin for error and adding a dose of rich content.
Through polling our peer-to-peer network of connections, it’s possible to ask a couple of thousand other entities connected in our web neighborhood whether an email address we don’t recognize is valid or not.
Here’s how it might work:
- You give your email server permission to run a certain kind of “Email-check” search via your ID account
- When the mail server receives an email from an unrecognized source, it pings your ID account with a request for an ID check.
- The ID account forwards the request to all its connections.
- Your connections’ ID accounts reply with either a positive (I trust this address), null (I don’t know this address) or negative (I don’t trust this address) response.
- Your email server quickly collates the polled results and decides what to do with the message.
Other applications for the connected web
Here are a few more obvious applications that will use the new architecture.
Dating
Dating will benefit from the ability to search your connections and broader network, as it’s beneficial to find someone who is already connected to you in some way (which traditional dating databases wouldn’t pick up on) - your interests & affiliations heat map!
Ride share
Ride sharing is an ideal web-based application. Think how many people live in the same suburban areas, and commute in their own vehicles to the same downtown locations for a day’s work, before crawling back in traffic the same way.
The big challenge to getting people to share their vehicles is personal security. How can you be sure that the person you’re picking up in your car (or the person who’s picking you up) is who they say they are?
The connected web helps by finding and showing the connecting path between you and them. If you know that they’re in the same knitting group as your next-door neighbor Molly, you can easily verify and be secure in their identity.
The world is full of these links - much more full than we think! When we meet someone with whom we share a connection, we say “Small world!”. But maybe the world is much smaller than we think!
Groups
Groups fit naturally into the connected web structure.
- A group is an entity.
- The members of the group are other entities.
- Membership means there is a reciprocal relationship between the group and each member - the group recognizes and trusts the individual, and the individual recognizes and trusts the group. So membership is a mutual recognition, and therefore under the control of both the group and the member.
- Group-level functions, like login to forums or receiving email newsletters, are dependent on the reciprocal relationship being in place.
- As an entity with links, the group connects each member to each other through itself. That reflects the real-world mesh of connections.
How might the connected web come about?
I can imagine two channels that could create the connected web:
- Commercial interest
- The Open-source community
Commercial interest
There are massive potential benefits for a company that can create a more connected web architecture. This architecture doesn’t replace web 1.0 - it works through it and uses the same technology as is in use today.
The difference is reach. There are only a few companies with long enough reach to build a connected web that could reach a critical mass: Yahoo!, Google, AOL and Microsoft/MSNBC are obvious candidates.
Out of these, I think Yahoo! is by far the strongest candidate, because it has always been in the community-building business, with a great human-centered directory, Groups and Messenger, plus recent acquisitions like Flickr, del.icio.us, and WebJay.
Yahoo! has also recently introduced its 360 suite (currently in beta), whose tag line is “Keep your friends and family connected to you.” Sounds like connected web to me!
How long could it take Yahoo! to get the connected web up and running? Probably months. It could start running existing and new services running on the connected architecture within a year.
The challenge to Yahoo! will be internal. For its entire history, Yahoo! has expanded mainly through acquisition, plugging new services into its web-based systems, which has delivered fast growth.
Yahoo has only engineered its own systems where there was nothing suitable available on the open market. However, it has done this very well, with a focus on ease-of-use and usefulness, maintaining a coherent user experience.
It already has a common login mechanism, and good APIs across all its services, and has driven innovations like single check-out for all stores in its marketplace. Plus, Yahoo is making good noises about social search (which is almost the connected web!).
I don’t know whether Yahoo has ever really had to re-architect its fundamental architecture, which has remained much the same (i.e. Web 1.0). The challenge is whether Yahoo’s shareholders will judge that it can afford to hold its expansion for long enough to turn into a butterfly. If it can grasp the nettle, I think that Yahoo is ideally positioned to reinvent itself as the dominant online brand for the next 5 or 10 years! I hope they can grasp the scope of the opportunity.
The Open-source community
We shouldn’t under-estimate the open-source route. Because the connected web is so simple in architecture (I haven’t detailed it all in this article), it would be possible to set up a consortium and agree an initial design and protocols within a year.
It would then be up to individuals, organizations and corporations to co-operate to build and host the infrastructure required to facilitate the connected web. I am convinced that this is possible, with different interests running the identity servers that store ID accounts and the applications that interact with them, and I have developed a model for this.
However, I don’t believe that the open-source movement can move quickly enough to win out against commercial interests, because there is too much to gain. I can’t honestly see the World-Wide Web Consortium (W3C) or other bodies being able to mobilize its affiliates in the same time Yahoo! could motivate its board into action. This is the challenge to the public community.
Only time will tell.
You want to find someone to come in and help clean your house.
Now, there’s someone living half a block away who would be perfect for the job.
But…
So there is a right answer out there, but how do you find that person?