Archive for the ‘iss’ Category

HTTP and the Push Model

Thursday, October 22nd, 2009

The Real-time Web seems to be the buzz of the moment, and there has been quite a debate comparing HTTP, XMPP and related technologies (Comet, Web sockets). Generally HTTP is associated with the Pull model, while XMPP is associated with the Push model. But it’s very well possible to design an architecture that follows the Push model using HTTP.

Let’s see an example to illustrated the point: Nick and Debbie are friends and they have subscribed to each other’s feed to receive updates. Their feeds are hosted on different servers.

In the Pull model, Nick has to poll Debbie’s server every time to check for updates from Debbie, and vice-versa.

In the Push model, the flow goes something like this:

  1. Debbie publishes a new entry on her server (Push);
  2. Debbie’s server lets Nick’s server know that Debbie has published a new entry (Push);
  3. Nick polls his own server to receive updates (Pull).

Notice that the second step is a Push implemented in HTTP. Nick’s server didn’t have to poll Debbie’s server every time to check for updates.

In essence, this is what RSSCloud and PubSubHubBub are trying to accomplish: promoting an architectural design that follows the Push model using HTTP. ISS implementation on top of Drupal is also promoting this.

Yes, the third step is still the Pull model, but so what? Polling is only performed when the user is online and active, and strategies like the sessionLink can help determine how frequently the client has to poll.

What impact on usability, performance and scalability would there be if the third step could be implemented using Push technology? Does it justify all the effort it would take to push this Push technology into all browsers? Wouldn’t it be easier to just bring the clients closer to the heart of the Internet (i.e. providing end-to-end connectivity by making them addressable) and let them benefit from Push technology using HTTP as usual?

Regarding the ISS implementation on top of Drupal, when a user syndicates an entry to a channel, all friends that are subscribed to this channel are notified (even if they are from a different server). The notification to a different server is done via the Services module.

Since ISS encourages a more decentralized social network and the ISS policies limits who gets notified, this helps in terms of performance and scalability. The speed at which information propagates will only be limited by the speed of the social network to filter information, which guarantees a high quality and very personalized stream of information for each individual.

TagCloud and TagLink in JSON

Wednesday, October 21st, 2009

I’ve been working with JSON for quite a while now, and the more I use it, the more I love it.

The great majority of standards use XML, and ISS is no different. But we need to evolve.

As such, this is an example of how the tagcloud and taglink would be in JSON:


{"data":{
  "iss":{
    "science":{
      "tagcloud":{
        "year":["2006","2007","2008"],
        "syndicated":["0,0,0,0,0,0,4,23,45,32,34,31",
          "32,44,53,23,43,32,34,64,34,21,35,23",
          "43,23,34"]},
      "taglink":{
        "base":"http:\/\/nick.iss.im\/category\/science",
        "incoming":["http:\/\/debbie.iss.im\/category\/environment",
          "http:\/\/ribas.iss.im\/category\/work",...],
        "outgoing":["http:\/\/penny.iss.im\/category\/science",
          "http:\/\/ribas.iss.im\/category\/nature",...]}
    },...
  }
}}

What do you think? Feedback is welcome!

Photos from FISL 10

Monday, June 29th, 2009

The 10th edition of FISL (Internation Free Software Forum) was a huge success, with over 8000 participants, including the presence of the president of Brazil, Luiz Inácio Lula da Silva!

Our presentation entitled ISS (Instant Syndicating Standards) was great! We gave a quick overview of ISS and many attendees got interested in our work. One of the highlights of the presentation was when we used tennis balls to explain how the information would travel in the social network. It was really fun to see the balls been thrown from one side of the room to the other. Some photos of the presentation are shown below:

Bruna Griebeler at FISL 10

Bruna Griebeler at FISL 10

Tiago Rosa da Silva at FISL 10

Tiago Rosa da Silva at FISL 10

Daniel Schmidt da Silva at FISL 10

Daniel Schmidt da Silva at FISL 10

Sharing Information, Services and Interactions in the Next Decade

Tuesday, June 16th, 2009

Introduction

In the 90s, after Tim Berners-Lee created the World Wide Web at CERN research center, scientists around the world were able to share information with their colleagues by setting up their own personal Websites. The key concept behind the Web was the hyperlink, which allowed users to browse Web pages across different networks.

With the popularization of the Web, browsing Web pages just wasn’t enough. So by the turn of the millennium, the advent of search engines came to people’s assistance, most notably from a search engine created by Larry Page and Sergey Brin while at Stanford University. Powered by complex algorithms, search engines would rank and present Web pages to users based on keywords.

With further popularization of the Web, searching again wasn’t enough. The top-down presentation of Web pages selected from a huge collection by an algorithm has created a very undemocratic way of sharing information. In this next decade, users will rely less on monolithic algorithms and more on their own personal social network for sharing information.

While the first decade of the Web was all about browsing and the second decade was about searching, the next decade will be about syndicating. Users will connect with their personal social network to receive and disseminate information in a bottom-up manner. Information, services and interactions will all be syndicated, allowing users to share not only information but also rich and profound experiences.

Sharing Information

As of today, there is no existing technology that allows individuals to share information in a bottom-up manner on a global scale. ISS (Instant Syndicating Standards) is a proposal to create just that: a distributed worldwide recommender system perfectly tuned to output a very personalized stream of information for each individual, where information flows from the personal social network towards the whole wide world. This is accomplished by allowing each individual to create their own broadcasting channels and to connect these channels with the ones created by their personal social network. This trustful and cascading network of syndicated streams filters out irrelevant information, while still letting good information pass through at each level.

The key concept behind ISS is the tagLink. The tagLink is a semantic link created by individuals showing how their friends’ channels are connected with their own channels. If a user becomes interested in a particular channel from a friend, he may subscribe to this channel and add it to his own channel. Thus, each individual receives exactly what he wants based on these subscriptions, and all information that reaches them goes through friends’ approval first.

Sharing Services

ISS is being developed as a set of services on top of Drupal. This service-oriented architecture promotes interoperability and allows individuals from different networks to share information with each other. The ISS services englobes user, channel and content management. These services together with services that provide file, language, search and session management will transform Drupal from a Content Management System into a Web Operating System.

The key concept behind the Drupal WebOS is the serviceLink, which is a structured format that links services together, including services from different networks. For example: a user may browse his way to a friend’s profile and become aware of her interests by visualizing her channels. If there is a common interest, the user may subscribe to a channel and create a taglink that connects her remote channel with his local channel. The fact that the friend is from another network is totally transparent to the user. This is possible when these systems follow a set of open standards called IOS (Instant Operating System). Users will be served by multiple WebOS.

Sharing Interactions

The services provided by the Drupal WebOS can be accessed through a Webtop, i.e. a Desktop Environment that works on top of the Web. The Ext Webtop is a Webtop created using the Ext JavaScript Library following a set of open standards called ITOP (Instant Desktop Environment). From this Webtop, users can share interactions with each other.

The key concept behind the Ext Webtop is the sessionLink. The sessionLink is a service that follows the publish/subscribe pattern and provides users and applications with (almost) real-time updates for subscribed services. For example, when a chat application is loaded in the Ext Webtop, the Webtop subscribes to the user.im service to be kept aware of any updates. More sophisticated interactions may also be shared, including sharing the whole Desktop Environment (this is called an Instant Session). In an Instant Session, when a user opens a window in the Webtop, his friend sees the window being opened. Likewise, when his friend closes a window, the user sees the window being closed.

Conclusion

In this text, we have presented ISS (Instant Syndicating Standards), an open set of standards that challenges the top-down model of information-sharing and gives place to a bottom-up model, where each person has a unique voice and equal opportunity to contribute and benefit. In this way, we hope to bring people closer together to discuss common interests and share information in a more open and democratic manner.

Also, we have presented the IOS and ITOP open standards, which we believe will help people to have more rich and profound experiences. We want to bring the Instant from Instant Messaging to the Web. And by Instant, the most important aspect that we want to exploit is not so much the When, but the Who. We want to empower individuals to “exchange” their Operating Systems and Desktop Environments with friends much the same way that they exchange Messages with friends when using Instant Messaging applications.

Along these two decades, the Web has evolved tremendously. The Web’s influence in democratizing access to information is evident. Yet, there is still a long way to go before reaching a truly democratic Web, where information flows freely in all directions. Also, there is still a long way to go before reaching a truly interactive Web, where people can connect with each other to create rich and profound experiences. We hope that the work here presented will help shape the way we share information, services and interactions in the next decade, as we believe that this will fundamentally shape us into better individuals and into a better society as a whole.

Acknowledgements

I would like to thank the following organizations for sponsoring my work:

  • The GIMSCOP research group from UFRGS university for sponsoring the development of the Drupal WebOS and Ext Webtop. Special regards to my mentor Dr. Jorge Otávio Trierweiler.
  • The PPGC (Computer Graduate Program) from UFRGS university and the CAPES brazilian federal agency, who provided me with a scholarship to develop ISS. Special regards to my mentor Dr. José Valdeni de Lima.
  • The Knight Foundation, for sponsoring the development of ISS on top of Drupal. Special regards to the UFRGSWeb team for helping me out and for giving me the opportunity to mentor them.

I would also like to thank the following open source communities for their contributions:

  • Dries Buytaert and the Drupal community. Special regards to Scott Nelson for the Services Module and Dmitri Gaskin for the JSON Server Module.
  • Jack Slocum and the ExtJS Community. Special regards to Todd Murdock for the Web Desktop extension and Thorsten Suckow-Homberg for the LiveGrid extension.

Further Information

References

ISS at FISL 10

Monday, June 8th, 2009

The 10th edition of the FISL (International Free Software Forum) will be held at Porto Alegre, Brazil on June 24-27th. Last year the FISL had nearly 7500 participants, thus it’s one of the biggest Free Software events in the world. It’s a great honor to know that this event debuted about 10 years ago as a joint effort between local organizations, companies and universities (including our beloved UFRGS). This shows the strength of open source and free software in this region.

I’m glad to announce that the Instant Syndicating Standards project was selected from over 600 applicants. We will be presenting on the 25th at 9AM. Other Drupal related projects will also be presented (check the full schedule). By the way, FISL’s site is powered by Drupal and the artwork is beautiful!

Bottom-up Wave Propagation

Thursday, May 28th, 2009

Google has announced the launch of a new service called Wave, which combines e-mail and instant messaging. Despite finding the service interesting, I still see fundamental problems in it’s architecture, specially regarding the flow of information. I cannot see how Google Wave will help people deal with information overload. Actually, I believe Wave will only create more information overload. Here is an interesting comment from the founder himself, Lars Rasmussen:

Well, what we are looking for now is for people to test it, try it out. We want to figure out what happens when lots of people use it. In the first several months, there were about 50 Wave users, and everything was all very controlled, calm, and quiet and all the Waves were super important to us. All of sudden there were 2000 users, all of them colleagues, all of them wanting to talk to us. And we learned — in a very exciting way — what it means to get lots of waves all of a sudden. And so we are learning lots from it.

To solve the information overload problem, I recommend taking a look at my proposal called ISS (Instant Syndicating Standards). It has been under the radar of Google for quite some time. Actually, it was part of Google’s Summer of Code in 2006. I have also commented it with the Google Reader and Google Talk team quite persuasively. I’ve been trying to reach people everywhere and talk about the advantages of adopting ISS. But unfortunately, no one has adopt it yet. Hey, it’s nobody’s fault, really. We have all being bombarded with information everywhere and it is hard to separate the wheat from the shaft. Too bad ISS doesn’t exist yet. It sure would help everyone in this sense.

I’m currently seeking to implement ISS on top of Drupal. ISS on top of Drupal has made me shift my focus from the original XMPP-based proposal to a Web-based proposal. The Web has such a momentum and it’s much easier to deploy. I just cannot see organizations and businesses contemplating XMPP and setting up Jabber Servers everywhere. Some will, but the great majority will just stick to the Web.

XMPP and Wave’s real-time collaboration reminds me of all the exciting things developed by my friend Massimiliano Mirra on SamePlace, specially the concept of Shared Web Applications. I believe real-time collaboration is very important. It’s not just about information sharing. It’s about interface sharing. People want to be able to share information with others and they want to be able to share interactions with others. And both SamePlace and Google Wave provide a good solution regarding sharing interactions. Inspired by SamePlace, I have also tackled this issue on top of the Web. For more details, please see sessionLink.

Drupal(Camp)^2

Wednesday, April 22nd, 2009

This saturdary I was at Drupal(Camp)^2, held at UNICAMP, Campinas, Brazil. The DrupalCamp was a huge success, with over 100 attendees. I gave a lecture about my work with Drupal, specially regarding ISS. It was a great opportunity to get to know the Drupal community in Brazil.

Knight Foundation sponsors ISS

Wednesday, March 4th, 2009

At this year’s DrupalCon it was announced the winners of the Knight Drupal Initiative, a partnership between the Knight Foundation and the Drupal Community to provide funding for grant proposals that explore digital publishing to transform their respective communities.

I’m glad to announce that our proposal has been approved! Our proposal aims to develop ISS support in Drupal. This will allow people to create and share a personalized stream of information within their social network, helping them to filter and recommend articles to others interested in the same issues.

I would like to thank Jose Zamora and Ken Rickard for all their effort in this process!

ISS UI Design

Wednesday, August 13th, 2008

Contacts

The ‘Contacts’ sidebar helps Nick discover new channels created by his personal social network.

The ‘List’ presents all aggregated entries from channels subscribed by Nick. Colors play an important role. Entries in blue are incoming entries, i.e. entries that have been received. Entries in red are outgoing entries, i.e. entries that have been sent. And entries in purple are entries that have been shared.

If Nick wants to share a specific entry, he may do so by tagging the entry appropriately. The Figure shows that this particular entry is already tagged with Science. He may add new tags by selecting them from his own tag cloud, or he may create new ones by typing them. When he syndicates this entry, he publishes this entry to the channels identified by these tags. Everyone subscribed to these channels will receive a notification.

Channels

Nick has associated Debbie’s Environment tag with his Science tag (i.e. he has created a taglink between the two channels). Every time Debbie publishes something tagged with Environment, these entries are automatically tagged with Science for Nick.

Thus, for users to share information, they don’t have necessarily to share the same concepts. All they have to do is to show how their friends’ concepts are associated with their own concepts.

It is important to remember that entries received are not published in Nick’s channels automatically unless they have been approved and shared by Nick himself.

Knight Drupal Initiative

Friday, August 8th, 2008

The Knight Drupal Initiative is a partnership between the Knight Foundation and the Drupal Community to provide funding for grant proposals that explore digital publishing to transform their respective communities.

I’m glad to announce that my proposal has been approved to the next phase of the selection process. This proposal’s main goal is to build ISS support in the Drupal platform. Our specific goal is to create a social network for our university to help students and professors discover and share information with colleagues and society.