Technogory

I recently starting playing around with Pipes - The Yahoo BETA of RSS mashing. Mashing is a way of aggregating many different sources of information into a single stream of data or information. In the case of Pipes, RSS is not the only thing you can mash!

The first thing you need to do to play with creating your own Pipes is create a Yahoo account if you don’t have one. Once you have your account, you can begin creating your own pipes. You can use anyone else’s Pipes without an account, as they can deliver in different formats to serve most needs.

Creating Pipes

So, let’s dive right in! The thing that helped me the most was realizing that the editor for Pipes is really geared towards tech heads - basically, if you are a developer or understand logic flow, the editor should be pretty easy for you to run with. I’m going to create a Pipe that will do a search on Yahoo and Google for Linux stuff, combine that with my own RSS feed, and then perform some filtering to get the final results.

The first thing you will notice is the side bar that has a number of tools. for now, we will concentrate on the Sources menu. We will be working with two different widgets from this menu. The first one is the Yahoo! Search. Grab the widget with your mouse and drag it into the editor pane. The widget will be drawn on the page with two entries initially.

The first entry field is for the thing you are searching for. In this case, I will search for all things that have Linux in them. Just that alone would return a lot of information. But I want to limit where I am looking for Linux. I want it to be news related items. So, in the second field titled Site Restriction, enter news.yahoo.com. In the lower grey pane you will see a debugger screen. This is very handy to make sure you are getting the results you want. So, click on the title bar of the widget, and the results of the search are shown below.

Now, I want to add another site restriction. That is, another site to search. So click on the green plus icon to add a Site Restriction. in this field, type news.google.com. You can include the http:// or not, the widget knows how to parse. Just those two searches alone will return a lot of results. Probably duplicates. We’ll deal with that shortly.

Now, let’s add a feed to this Pipe. Grab the Fetch Feed widget and drag it to the editor pane. I am going to enter the information for my sites feed, but you could enter any valid feed here. So, I enter www.velocitywebdev.com/?feed=rss2 to get the feed. I click on the title bar of the Fetch Feed widget, and see my results in the debugger pane below.

Now, I have to bring all this info together in one big bit bucket. This requires a new widget. In the left hand menu, click on the menu Operators. Here is where all of the logic operators can be found. In our example, I want to combine all of the data. To do this, I build a Union. So, drag the Union widget to the editor pane. The Union widget has a number of connectors on the top, and just one output on the bottom. So, as you can see, I can have multiple inputs into a union. But, how do we do this.

At the bottom of the Yahoo! Search and Fetch Feed widget, you will see a small circle - this is a connector. Click on the connector and crag down to any of the Union connectors. Release, and you have a connection. Repeat for the other widget. After connection the widgets, you can then click on the title bar of the Union to see the results in the debugger pane.

Okay, so we have a lot of stuff - what do we do? There is probably duplicate articles. There is a widget for that! In the Operators section, there are a probably a couple of widgets we could use; we will use Unique. The Unique widget allows us to specify the criteria based on a number of options; title, author, description, pub date, or link. We will look for uniqueness with the title. Now, connect the bottom Union widget connector to the top Unique widget connector. You can click the title bar to see your current output in the debugger pane.

All that is left is to connect our Unique widget connector to the Pipe Output connector on the editor pane. That’s it! We have created a Pipe that will search Google and Yahoo and take and RSS feed, mash it up, filter it on title for uniqueness, and provide the results. Great, but what do I do with the Pipe? Well, first we have to save it. Up on the Tab at the top near the Pipe Icon, it should say untitled. Change that to a title you would like. I called mine Linux Mashup. Next, select Save to the right of the screen.

Run The Pipe

Now that we have a Pipe we like, we need to test it out. So, at the top of the screen, click on the link Run Pipe. If everything went well, you should see the results of your Pipe! I will warn you, Pipes is Beta, and the browser I used for this example was Firefox 2.0.0.8 on a SLES 10.2 system. It has had trouble from time to time with the Pipes - taking way to long to show results.  I have not had trouble with Konqueror on the same system as you can see from the screen shot.  I also have had no trouble with Firefox 2.0.0.8 on Ubuntu.

What Do I Do With The Output?

Well - whatever you want! Pipes gives you a number of output options to your phone, email, use on your own web page or web app. It’s really got some good options. My use has been mostly as RSS feeds up to this point, but email and phone were verified and did work for me.

I think Pipes has a useful future. With the amount of information on the web, your mashups are limitless. Take the Pipe that sends a guy a listing of all apartments within a certain area and by certain landmarks? that’s pretty cool stuff. So, think about the kind of info you want to mashup and get to it!