river4

River4 is a node.js river-of-news aggregator that stores its lists and data in Amazon S3.

####Overview

We have a press backgrounder for River4 here. If you're wondering what it is, or why it's significant, this is the first place to go.

If you need help, we have a support mail list, with people who have successfully set up and are running River4 installations. If you're having trouble, this is the place to go.

If you're ready to install the software, you've come to the right place! :-)

What you'll need

A node.js installation.
An Amazon account, and an S3 bucket to store the JSON files, and a small HTML file.
One or more OPML subscription list files.

How to install

Create an S3 bucket to hold all your subscription lists, rivers, and data for the aggregator.
On the node.js system, set an environment variable, s3path, to contain the path to the bucket created in step 1.

export s3path=/river.mydomain.com/
Again, on the node.js system, set the two AWS environment variables. This allows the River4 app to write to your bucket.

export AWS_ACCESS_KEY_ID=12345

export AWS_SECRET_ACCESS_KEY=TUVWXYZ
Launch river4.js on a node.js system. Suppose that server is aggregator.mydomain.com.
Look in the bucket. You should see a data folder, with a single file in it containing the default value of prefs and stats for the app. There's also an index.html file, which will display your rivers in a simple way, providing code you can crib to create your own way of browsing (room for improvement here, for sure).
Create a folder at the top level of the bucket called "lists". Save one or more OPML subscription lists into that folder.
After a while you should see a new folder called "rivers" created automatically by the software. In that folder you should see one JSON file for each list. It contains the news from those feeds, discovered by River4. This format is designed to plug into the beautfiul" river displayer.
If you want to watch the progress of the aggregator, you can view this page.

http://aggregator.mydomain.com/serverdata

Notes

I edit code in an outliner, which is then turned into JavaScript. The "opml" folder in the repository contains the versions of the code that I edit. The comments are stripped out of the code before it's converted to raw JS, so there is information for developers in the OPML that isn't in the main files (though all the running code is in both).
The first released version is 0.79. They will increment by one one-hundredth every release. At some point I'll call it 1.0, then subsequent releases will be 1.01, 1.02 etc.
When you set up your S3 bucket, make sure that web hosting is enabled and index.html is the name of your index file. Here's a screen shot that shows how to set it up.

Thanks!

Thanks to two developer friends, Dan MacTough and Eric Kidd, who helped this Node.js newbie get this app up and running.

Specifically thanks to Dan for writing the excellent feedparser and opmlparser packages that are incorporated in River4.

Changes

v0.87 -- 6/09/14 by DW

Fixed a bug that would cause River4 to crash when processing an item with a null title.

v0.86 -- 6/09/14 by DW

Fixed a bug that would cause River4 to crash when reading an item from a subscription list that didn't have an xmlUrl attribute.

v0.85 -- 6/07/14 by DW

Two fixes, explained here.

v0.84 -- 6/06/14 by DW

Two fixes, explained here.

v0.83 -- 6/06/14 by DW

Now if there's an error in any JSON code we try to parse, we display an error message in the console, along with the path to the S3 file we were trying to read.

v0.81 -- 6/02/14 by DW

serverData.stats now has a copy of the last story added to the river. The dashboard page displays it.

v0.80 -- 6/02/14 by DW

New "dashboard" feature. If your server is running at aggregator.mydomain.com, if you go to:

http://aggregator.mydomain.com/dashboard

You'll get a real-time readout of what your aggregator is doing.

The HTML source for the dashboard page is in dashboard.opml in the opml folder in the repository.

DaveSeidel/river4