The 2nd edition of Drupal(Camp)^2 took place over this weekend, with many presentations covering the most exciting topics in the Drupal community: Open Atrium, Features, Drush, Aegir, Simpletest, Coder, Devel, Views, Zen, Localize and, of course, ISS.
I look forward to DrupalCon Brazil 2010!
The Real-time Web seems to be the buzz of the moment, and there has been quite a debate comparing HTTP, XMPP and related technologies (Comet, Web sockets). Generally HTTP is associated with the Pull model, while XMPP is associated with the Push model. But it’s very well possible to design an architecture that follows the Push model using HTTP.
Let’s see an example to illustrated the point: Nick and Debbie are friends and they have subscribed to each other’s feed to receive updates. Their feeds are hosted on different servers.
In the Pull model, Nick has to poll Debbie’s server every time to check for updates from Debbie, and vice-versa.
In the Push model, the flow goes something like this:
Notice that the second step is a Push implemented in HTTP. Nick’s server didn’t have to poll Debbie’s server every time to check for updates.
In essence, this is what RSSCloud and PubSubHubBub are trying to accomplish: promoting an architectural design that follows the Push model using HTTP. ISS implementation on top of Drupal is also promoting this.
Yes, the third step is still the Pull model, but so what? Polling is only performed when the user is online and active, and strategies like the sessionLink can help determine how frequently the client has to poll.
What impact on usability, performance and scalability would there be if the third step could be implemented using Push technology? Does it justify all the effort it would take to push this Push technology into all browsers? Wouldn’t it be easier to just bring the clients closer to the heart of the Internet (i.e. providing end-to-end connectivity by making them addressable) and let them benefit from Push technology using HTTP as usual?
Regarding the ISS implementation on top of Drupal, when a user syndicates an entry to a channel, all friends that are subscribed to this channel are notified (even if they are from a different server). The notification to a different server is done via the Services module.
Since ISS encourages a more decentralized social network and the ISS policies limits who gets notified, this helps in terms of performance and scalability. The speed at which information propagates will only be limited by the speed of the social network to filter information, which guarantees a high quality and very personalized stream of information for each individual.
I’ve been working with JSON for quite a while now, and the more I use it, the more I love it.
The great majority of standards use XML, and ISS is no different. But we need to evolve.
As such, this is an example of how the tagcloud and taglink would be in JSON:
{"data":{
"iss":{
"science":{
"tagcloud":{
"year":["2006","2007","2008"],
"syndicated":["0,0,0,0,0,0,4,23,45,32,34,31",
"32,44,53,23,43,32,34,64,34,21,35,23",
"43,23,34"]},
"taglink":{
"base":"http:\/\/nick.iss.im\/category\/science",
"incoming":["http:\/\/debbie.iss.im\/category\/environment",
"http:\/\/ribas.iss.im\/category\/work",...],
"outgoing":["http:\/\/penny.iss.im\/category\/science",
"http:\/\/ribas.iss.im\/category\/nature",...]}
},...
}
}}
What do you think? Feedback is welcome!
Photos from Congonhas, Ouro Preto, and Belo Horizonte:
The Ext JS 3.0 was just released! To me, this is the best JavaScript library ever! And I’m glad that it’s open source.
The most important improvement of this release was a clear separation of the Widgets from the Core. The Ext Core provides object-oriented mechanisms for handling the DOM, Events and Ajax. The Widgets contemplate Windows, Layouts, Toolbars, Buttons, Menus, Trees, Grids and Forms. New to this release was support for Charts, which is done via Flash. There is no official support for Audio and Video, but many interesting Flash media players fill in the gap.
So congratulations to the Ext team! Thank you for the outstanding work! This is truly a step forward towards a richer Web!
The 10th edition of FISL (Internation Free Software Forum) was a huge success, with over 8000 participants, including the presence of the president of Brazil, Luiz Inácio Lula da Silva!
Our presentation entitled ISS (Instant Syndicating Standards) was great! We gave a quick overview of ISS and many attendees got interested in our work. One of the highlights of the presentation was when we used tennis balls to explain how the information would travel in the social network. It was really fun to see the balls been thrown from one side of the room to the other. Some photos of the presentation are shown below:
Bruna Griebeler at FISL 10
Tiago Rosa da Silva at FISL 10
Daniel Schmidt da Silva at FISL 10
Introduction
In the 90s, after Tim Berners-Lee created the World Wide Web at CERN research center, scientists around the world were able to share information with their colleagues by setting up their own personal Websites. The key concept behind the Web was the hyperlink, which allowed users to browse Web pages across different networks.
With the popularization of the Web, browsing Web pages just wasn’t enough. So by the turn of the millennium, the advent of search engines came to people’s assistance, most notably from a search engine created by Larry Page and Sergey Brin while at Stanford University. Powered by complex algorithms, search engines would rank and present Web pages to users based on keywords.
With further popularization of the Web, searching again wasn’t enough. The top-down presentation of Web pages selected from a huge collection by an algorithm has created a very undemocratic way of sharing information. In this next decade, users will rely less on monolithic algorithms and more on their own personal social network for sharing information.
While the first decade of the Web was all about browsing and the second decade was about searching, the next decade will be about syndicating. Users will connect with their personal social network to receive and disseminate information in a bottom-up manner. Information, services and interactions will all be syndicated, allowing users to share not only information but also rich and profound experiences.
Sharing Information
As of today, there is no existing technology that allows individuals to share information in a bottom-up manner on a global scale. ISS (Instant Syndicating Standards) is a proposal to create just that: a distributed worldwide recommender system perfectly tuned to output a very personalized stream of information for each individual, where information flows from the personal social network towards the whole wide world. This is accomplished by allowing each individual to create their own broadcasting channels and to connect these channels with the ones created by their personal social network. This trustful and cascading network of syndicated streams filters out irrelevant information, while still letting good information pass through at each level.
The key concept behind ISS is the tagLink. The tagLink is a semantic link created by individuals showing how their friends’ channels are connected with their own channels. If a user becomes interested in a particular channel from a friend, he may subscribe to this channel and add it to his own channel. Thus, each individual receives exactly what he wants based on these subscriptions, and all information that reaches them goes through friends’ approval first.
Sharing Services
ISS is being developed as a set of services on top of Drupal. This service-oriented architecture promotes interoperability and allows individuals from different networks to share information with each other. The ISS services englobes user, channel and content management. These services together with services that provide file, language, search and session management will transform Drupal from a Content Management System into a Web Operating System.
The key concept behind the Drupal WebOS is the serviceLink, which is a structured format that links services together, including services from different networks. For example: a user may browse his way to a friend’s profile and become aware of her interests by visualizing her channels. If there is a common interest, the user may subscribe to a channel and create a taglink that connects her remote channel with his local channel. The fact that the friend is from another network is totally transparent to the user. This is possible when these systems follow a set of open standards called IOS (Instant Operating System). Users will be served by multiple WebOS.
Sharing Interactions
The services provided by the Drupal WebOS can be accessed through a Webtop, i.e. a Desktop Environment that works on top of the Web. The Ext Webtop is a Webtop created using the Ext JavaScript Library following a set of open standards called ITOP (Instant Desktop Environment). From this Webtop, users can share interactions with each other.
The key concept behind the Ext Webtop is the sessionLink. The sessionLink is a service that follows the publish/subscribe pattern and provides users and applications with (almost) real-time updates for subscribed services. For example, when a chat application is loaded in the Ext Webtop, the Webtop subscribes to the user.im service to be kept aware of any updates. More sophisticated interactions may also be shared, including sharing the whole Desktop Environment (this is called an Instant Session). In an Instant Session, when a user opens a window in the Webtop, his friend sees the window being opened. Likewise, when his friend closes a window, the user sees the window being closed.
Conclusion
In this text, we have presented ISS (Instant Syndicating Standards), an open set of standards that challenges the top-down model of information-sharing and gives place to a bottom-up model, where each person has a unique voice and equal opportunity to contribute and benefit. In this way, we hope to bring people closer together to discuss common interests and share information in a more open and democratic manner.
Also, we have presented the IOS and ITOP open standards, which we believe will help people to have more rich and profound experiences. We want to bring the Instant from Instant Messaging to the Web. And by Instant, the most important aspect that we want to exploit is not so much the When, but the Who. We want to empower individuals to “exchange” their Operating Systems and Desktop Environments with friends much the same way that they exchange Messages with friends when using Instant Messaging applications.
Along these two decades, the Web has evolved tremendously. The Web’s influence in democratizing access to information is evident. Yet, there is still a long way to go before reaching a truly democratic Web, where information flows freely in all directions. Also, there is still a long way to go before reaching a truly interactive Web, where people can connect with each other to create rich and profound experiences. We hope that the work here presented will help shape the way we share information, services and interactions in the next decade, as we believe that this will fundamentally shape us into better individuals and into a better society as a whole.
Acknowledgements
I would like to thank the following organizations for sponsoring my work:
I would also like to thank the following open source communities for their contributions:
Further Information
References
The 10th edition of the FISL (International Free Software Forum) will be held at Porto Alegre, Brazil on June 24-27th. Last year the FISL had nearly 7500 participants, thus it’s one of the biggest Free Software events in the world. It’s a great honor to know that this event debuted about 10 years ago as a joint effort between local organizations, companies and universities (including our beloved UFRGS). This shows the strength of open source and free software in this region.
I’m glad to announce that the Instant Syndicating Standards project was selected from over 600 applicants. We will be presenting on the 25th at 9AM. Other Drupal related projects will also be presented (check the full schedule). By the way, FISL’s site is powered by Drupal and the artwork is beautiful!
The Drupal WebOS provides several services to the Ext Webtop. The structure of these services are well described and they can either be static or dynamic in nature. The difference between static and dynamic services is that the dynamic services can be seem more as a stream of information and they can be used to provide users with (almost) real-time updates. If there is a great number of dynamic services being requested at a given time, however, the performance of the application can suffer substantially. To solve this issue, we present the sessionLink.
The sessionLink is a service that follows the publish/subscribe pattern and provides users and applications with updates for subscribed services. For example, when a chat application is loaded in the Ext Webtop, the Webtop subscribes to the user.im service to be kept aware of any updates.
The following is a simplified description of the user.im service:
{"method": "user.im",
"args": [
{"name": "uid", type: "int", "desc": "User ID", "required": "1"},
{"name": "msg", type: "string", "desc": "Message", "required": "1"}
],
“type”: “data”,
"sessionlink": "payload",
"time": "2000",
"sid": "14"
}
The “sessionlink” can either be set to none, payload or timestamp. Services by default are static and there is no need to define sessionlink. A dynamic service on the other hand must have “sessionlink” set to payload or timestamp. When payload is set, the session.link service will return the payload for the subscribed method. Otherwise, if timestamp is set, it will only return a (unix) timestamp, thus requiring the Webtop to make an additional call to receive the update. For applications like chat, where speed is essential and the content is small, it’s recommended to set sessionlink to payload.
The time can also be set. This will help determine how often the Webtop will have to poll the session.link service. The lowest time of the current set of subscribed services is used. For example, if the Webtop is calling session.link every 30 seconds and the user loads the chat application, then the Webtop will start polling session.link every 2 seconds (2000 milliseconds).
The subscriptions are managed automatically by the Webtop by calling the session.subscribe service:
http://webos.iss.im/services/json
method=session.subscribe
sid=14
The user.im service has a sid (Service ID) of 14. The session.subscribe service actually handles sid as a string, thus enabling the Webtop to subscribe to several services at the same time (just use a comma to separate the sids).
To unsubscribe, there is the session.unsubscribe service that works the same way. If the user closes the chat application and he or she is not talking to anyone else, the Webtop can unsubscribe from the user.im service:
http://webos.iss.im/services/json
method=session.unsubscribe
sid=14
In conclusion, the sessionLink is a key component that helps the Webtop to be kept aware of any updates in an easy and straight-forward way. This component will help developers create applications that provide real-time collaboration and offer users a more seamless experience.
Google has announced the launch of a new service called Wave, which combines e-mail and instant messaging. Despite finding the service interesting, I still see fundamental problems in it’s architecture, specially regarding the flow of information. I cannot see how Google Wave will help people deal with information overload. Actually, I believe Wave will only create more information overload. Here is an interesting comment from the founder himself, Lars Rasmussen:
Well, what we are looking for now is for people to test it, try it out. We want to figure out what happens when lots of people use it. In the first several months, there were about 50 Wave users, and everything was all very controlled, calm, and quiet and all the Waves were super important to us. All of sudden there were 2000 users, all of them colleagues, all of them wanting to talk to us. And we learned — in a very exciting way — what it means to get lots of waves all of a sudden. And so we are learning lots from it.
To solve the information overload problem, I recommend taking a look at my proposal called ISS (Instant Syndicating Standards). It has been under the radar of Google for quite some time. Actually, it was part of Google’s Summer of Code in 2006. I have also commented it with the Google Reader and Google Talk team quite persuasively. I’ve been trying to reach people everywhere and talk about the advantages of adopting ISS. But unfortunately, no one has adopt it yet. Hey, it’s nobody’s fault, really. We have all being bombarded with information everywhere and it is hard to separate the wheat from the shaft. Too bad ISS doesn’t exist yet. It sure would help everyone in this sense.
I’m currently seeking to implement ISS on top of Drupal. ISS on top of Drupal has made me shift my focus from the original XMPP-based proposal to a Web-based proposal. The Web has such a momentum and it’s much easier to deploy. I just cannot see organizations and businesses contemplating XMPP and setting up Jabber Servers everywhere. Some will, but the great majority will just stick to the Web.
XMPP and Wave’s real-time collaboration reminds me of all the exciting things developed by my friend Massimiliano Mirra on SamePlace, specially the concept of Shared Web Applications. I believe real-time collaboration is very important. It’s not just about information sharing. It’s about interface sharing. People want to be able to share information with others and they want to be able to share interactions with others. And both SamePlace and Google Wave provide a good solution regarding sharing interactions. Inspired by SamePlace, I have also tackled this issue on top of the Web. For more details, please see sessionLink.
