I have been hosting my own River4 server on Heroku, using Amazon S3 to store the data files. A year ago when I first set River4 up the Heroku free account did not have the shutdown requirements it has today. Heroku announced the changes to their plans last fall but did not start enforcing them in July.
I had forgotten about the changes Heroku was making until a month ago when I started noticing that my River4 server on Heroku was shutting down randomly unless I kept the River4 dashboard open. Apparently the free service is set up to shutdown if there is no inbound web traffic over a thirty minute span. I didn't have time to work on alternatives, so I decided to just sign up for the monthly hobby account, which at $7 per month isn't too bad.
Over the last couple of weeks I have been exploring different cloud hosting options for a couple of reasons, one being that I have been studying Docker and using River4 as the subject for building my own containers. Another reason is that Dave is working on a refresh to the EC2 for Poets he wrote several years ago, using River4 as the topic for the tutorial on setting up your own server on Amazon Web Service based on a custom image he built, and I did some testing of the tutorial.
EC2 for Poets got me wondering about other cloud services, and whether I ought to move my River4 server from Heroku to one of the other services. Yesterday I looked at Azure and Google Cloud Platform and found that Google has the most competitive compute prices. I figure you can probably run a River4 server on GCP for $5 to $6 per month, which is less than Heroku, AWS, and Azure.
Another, cheaper option, is to buy a virtual server at CloudAtCost. CloudAtCost has several plans, for which you can pay per month, or "buy" a virtual server for a one time charge. Last year I bought a Developer 1 server on which I am hosting Fargo Publisher, which is the back-end to my blog publishing environment. Right now CloudAtCost has a 50% off sale, so last night I decided to buy a Developer 2 server, which has much more horse power than any of the instances I tested on AWS, Azure, or Google.
I figure that while I can run River4 on this Developer2 server, I can also use it for any other DevOps software that I may tinker with in the future. Rather than install Node.js on the server, I have installed Docker to use the River4 container I created. I even went so far as to buy a domain to point at the server, MyWebStream.net. I had my new River4 server running in about 15 minutes, and I have switched my Heroku server back to the free service.
After using CloudAtCost for a year, I am comfortable endorsing the service. My server has been pretty reliable, although there has been some maintenance that has caused me to do some unexpected reboots. The servers I host with CloudAtCost are solely for my use, so there is no real pressure to keep these servers running non-stop.
My spelunking among the public clouds has lead me to the Google Cloud Platform, where I spied with my little eye the Node.js instance and it's $4.49 estimated monthly cost. The cost is not free, but I think it might be the lowest monthly cost for hosting River4 on a cloud server. Probably the only lower costs are either the "free" node on Heroku, with all its constraints, or the one time purchase of a server at CloudAtCost.
I launched the Node.js instance on GCP and found that it is Debian with node.js pre-installed. To install River4, one can skip the Install node.js portion of the Installing River4 on Ubuntu instructions and start with Install forever, then Install git and finally Install river4, before moving on to the River4 howto, starting at step 4.
You will need to open port 1337 in order to access the River4 site, it appears that something (perhaps node.js itself) is using port 80. I haven't done much research into disabling whatever is using port 80. In the Google Developers Console, click Networking, Firewall rules. Click New firewall rule, and enter the fields as follows:
Description: Allow port 1337 traffic to access the River4 home page (this is optional, you could leave it blank)
Source filter: Allow from any source (0.0.0.0/0)
Allowed protocols and ports: tcp:1337
Then click Create.
To find the public IP address for your site, on the Google Developers Console click Compute, Compute Engine, VM instances. You will see your instance name listed below a graph showing CPU utilization, and to the right is the External IP address. To access the River4 Home page enter http://[External IP address]:1337 in the address bar of your web browser.
The Google Developers Console provides a button for SSH access to your instance, which opens an SSH session in a browser window.
The GCP Node.js instance is based on the Google Compute Engine f-1 micro instance that can cost as little as $0.0056 per hour per month if you run the instance for a full month. Again, for comparison, the AWS t2.micro instance costs $0.013 per hour, so you can see the compute part of the cost is much lower. I expect there will be additional I/O charges add to the $4.49 estimated monthly price.
I provisioned an instance of Ubuntu 14.04 in Azure to host my River4 Docker container, and it has been running for just over 20 hours with no problems. I am not sure how long I will keep this instance running, I may take it down at any time. Update: I have shutdown the instance and removed the link to it.
There are a couple of things I like about Azure over Amazon. One is that they provide a relatively friendly DNS name for your site. While one can access the River4 home page via a public IP, I think DNS names are much more user friendly and Azure certainly makes that much easier. (Amazon will provide a public DNS but they tend to be really long, and in my opinion, un-friendly.)
The other thing I like about Azure is that they provide a lot of nice, easy to use monitoring tools. In fact, I think the user interface of the Azure portal is much more friendly than Amazon. If you are at all interested in monitoring your River4 instance, I think Azure makes that much easier.
On demand AWS Pricing for a t2.micro instance, which is their smallest, is $0.013 per hour. If you go with a reserved instance for one year, you can get the price down to $0.009 per hour. Azure's on demand pricing for a basic instance, which is their smallest, $0.018 per hour. Azure billing rounds up to the nearest minute, while AWS rounds up to the nearest hour.
I re-installed Docker Toolbox on a computer running Windows 7, following the instructions provided on the Docker web site. There are inconsistencies in what happens during the install and what is shown on the web site. For example, what is shown on the web site as the Docker CLI actually appears on my computer as Docker Quickstart Terminal.
Clicking the Docker Quickstart Terminal icon does not execute what is described on the web site. The issue appears to be that the icon is intended to execute a shell script, start.sh that on my computer is associated with Notepad++. After some fiddling I noticed a couple of things, one is that when I right-click the icon, I see options for Git Init Here and Git Bash. If I click Git Bash, a window opens with what appears to be a Bash shell with the folder that the icon is associated to opened as default.
After selecting Git Bash to open the "bash" window, I then ran start.sh and it appears to successfully connect to the default VM created in Virtual Box. I did a little more searching and found the folder C:\Program Files (x86)\Git\bin with a number of programs including sh.exe, which I assume is the bash shell. However, configuring the icon to use sh.exe to open start.sh results in errors. For now I can by with manually running start.sh.
I need to run start.sh in order to associate the bash window with the default virtual machine in which docker is hosted. I assume the script is setting the environment variables described on this page. I have been successful at running the docker "hello-world" container as well as my river4 container.
I have checked, and Kitematic is also using the default vm, so when I start Kitematic I can see the river4 container running. I can't open command lines in Kitematic, and I suspect this is because the environment variables are not being set as needed to access the virtual machine.
Actually, I think the problem is not having the ssh.exe in the PATH on my computer as described in the Using Docker from a Windows Command Line prompt section on this page. I've added c:\Program Files (x86)\Git\bin to the PATH, and I'll need to reboot to confirm that was the issue.
No joy after the reboot, clicking the Quickstart Terminal icon on my desktop generates the same result. I am starting to think I have multiple ssh.exe programs on my computer, and it is finding an incompatible one in my path.
For some reason the touchscreen on my Surface 3 is acting up. At what appears to be random occurrences touches become translated as right-clicks and the only way I can resolve the issue is by restarting the tablet.
Another issue, potential related, is that the July 23 firmware update is not being installed. After some fiddling, I think the issue is that I was trying to install the update with the Type Cover not attached. I've forced installation of the update, with the Surface 3 plugged in and the Type Cover attached and it appears the system update is finally being installed.
The following instructions assume the reader has access to an Ubuntu server and is somewhat familiar with how to function within the server. To build a functioning River4 server do the following (if you are already running Docker skip to step 3):
Step 1: Follow the instructions provided on Docker.com to install Docker Engine on the Ubuntu server
Step 2: Confirm you have Docker running by entering sudo docker run hello-world
Step 3: Run the River4 container by entering docker run --name river4 -d -p 1337:1337 fmcpherson/river4
--name provides a name to the container, making it easier to run subsequent commands
-d specifies to run the container in the background, similar to how forever works
-p specifies port mapping in the format [host computer port] : [container port]
Step 4: Confirm River4 is running by checking for console output by entering docker logs river4
You may wish to pipe the output to more for closer inspection: docker logs river4 | more
You should see: "River4 v0.X running on port 1337, file path == river4data/" without quotes and X representing a version number. (As of this writing the current version of River is 0.118.)
Step 5: If you don't already know it, find the IP address for the Ubuntu server
Enter ifconfig and look for inet addr: within eth0.
For example purposes, I will use 192.168.142.129 as the IP address of the server
Step 6: Open the River4 home page in a web browser
Enter [Your server ip address]:1337 in the address bar of your web browser. For example http://192.168.142.129:1337
The River4 Home page should load
Depending on the rate at which new articles are published, you may not see new article entries for some time.
Step 7: Check the River4 Dashboard
Step 8: To stop the River4 container enter docker stop river4
To confirm what Docker images are on your server enter docker images. When you run a container, Docker first checks to see whether there is an image stored on the host computer and if there is, runs it from there. If an image for the container is not found on the local computer, Docker then searches Docker Hub for the image. The River4 image is in the fmcpherson/River4 repository on the Docker Hub.
To remove the image from your computer enter docker rmi -f fmcpherson/river4
By default all of the data files that River4 uses are stored within the container. The River4 image is built with a default RSS subscription list called readinglist.opml and it is stored in /river4data/lists. Changes to data files do not persist within the container after the container is shut down. You can map a directory on the host computer to the /river4data directory in the container by adding the -v flag to the docker run command. For example:
Create a river4data directory in your home folder on the host computer. For example /home/frank/river4data.
To map the directory you created above to the River4 container enter the docker run command as follows:
docker run --name river4 -v /home/frank/river4data:/river4data -d -p 1337:1337 fmcpherson/river4
River4 will create the data, lists, and rivers directories in your /home/frank/river4 directory and the lists directory will be empty. You will need to copy a RSS subscription list OPML file to the lists directory.
If you are running your Ubuntu server in a public cloud, you may need to open port 1337 to access the River4 home page. The -p flag in the docker run statement specifies the port mapping in the following format [host computer port] : [container port]. The container port will always be 1337, but the host computer port can be whatever port you specify. For example to enable the River4 container to hear on port 80 enter -p 80:1337 in the docker run command. If you map port 80 you will not need to include that port number for the address to the River4 home page.
If you are interested in how the Docker image was built, you can look at the Dockerfile in my Github repository for the container.
I am doing more testing of my River4 Docker container on Linux and I think I have figured out the mapping of the /river4data volume in the container to a local directory on my Linux host server.
The change I had to make is to use the full directory path to a folder in my home directory. Here is what I have done:
Create a directory in my home directory called localstorage. (It could be called river4data)
Note the full path: /home/frank/localstorage
Executed the following command: docker run --name river4 -v /home/frank/localstorage:/river4data -d -p 1337:1337 fmcpherson/river4 to run the container.
The data, lists, and rivers directories were created by river4.js, and the lists directory is empty. I confirmed by entering docker logs river4 that river4.js was running with zero feeds.
I then copied the readinglist.opml file to the lists directory, but got a permission error, so I entered sudo cp readinglist.opml /home/frank/localstorage/lists/readinglist.opml.
I ran docker logs river4 a few times again and eventually river4.js picked up the readinglist.opml file and started processing the feeds. I then checked the /data/feeds directory and noted that all of the feed subdirectories were created.
The latest version of the River4 Docker image now supports volumes, which provides for data to persist across a container. NOTE: If you use Kitematic to run the River4 container, there are some slight differences to how this works, which I will write about in another article.
Create the river4data and subdirectories at build time (see Dockerfile)
Copy my full readinglist.opml to the /river4data/lists directory during build
Changed the CMD so that River4 is run from the root directory so that it finds river4data where it expects to find it as a subdirectory off root.
Mount /river4data as a volume during build.
You can run the container as before, it will now use the default readinglist without include.
docker run --name river4 -d -p 1337:1337 fmcpherson/river4
Confirm entering by docker logs river4
If you are running Docker Engine 1.8 or newer, you can use the docker cp command to copy your own readinglist.opml file in to the /river4data/lists directory in the container by entering the following (assuming the readinglist.opml is in the current directory, and you have named your container river4)
NOTE: The above change will not persist, if you shut down the container, the next time it starts will start up with the originally provided readinglist.opml. You can re-run the docker cp command after starting up the container to overwrite the file.
If you want the changes you make to readinglist.opml to persist, you need to "map" a local directory on your computer to the /river4data mount in the container. For example, assume you have a directory on your computer called /river, to start the river4 container in a way that data persists enter the following run command.
Now, if you overwrite the readinglist.opml with your own copy, it will persist after you shutdown the river4 container, provided you execute the run statement exactly as provided above.
After running the container, to overwrite enter
After this, the change you just made will persist, again provided you include the -v flag in the docker run statement as provided above.
If the -v flag is not included and you enter "docker run --name river4 -d -p 1337:1337 fmcpherson.river4" the original readinglist provided in the image will be used by the container.
The following recipe assumes the reader is familiar with Docker, knows how to create images and containers, and has an account on Docker Hub. Familiarity with Git is helpful if you want to use Github to obtain and get updates to the source files, but it is not required. I am also assuming that you are executing these instructions on a computer running some form of Unix.
I recommend following the instructions in Docker Docs to install Docker Engine on the computer that will host your River4 container. For reference, I used Ubuntu server, running as a guest in VMWare workstation on a computer running Windows 7 to create my image of River4. I've tested the image I created using Kitematic on Windows and OS X and I have used Boot2Docker on OS X. Kitematic appears to provide the easiest way to run Docker on a Windows or Mac computer.
Step 1, Obtain Source files
Obtain the River4 source files from the River4 Github repository. You can either use Git to obtain a copy or click the Download Zip button on the repository web page. Create a directory on the computer that has Docker installed and extract the Zip file to that directory. To use Git, CD to the directory you created to store the River4 files and enter git clone https://github.com/scripting/river4.git
Obtain the Dockerized River4 source files from the Github repository. You can either use Git to obtain a copy or click the Download Zip button on the repository web page. Create a directory on the computer that has Docker installed and extract the Zip file to that directory. To use Git, CD to the directory you created to store and enter git clone https://github.com/fmcpherson/DockerizedRiver4.git
Note that, if you are using git to obtain the source files, it is best to create separate directories for the River4 and Dockerized River4 files so to not confuse git. If you simple download the Zip files, you can extract the files into one directory.
Step 2, Package together the Docker Image source files
Create a new directory
Copy the River4 source files to that directory.
Copy the DockerizedRiver4 source files to that directory
After you complete step 2 your directory must have the following files, if any of these files are missing you will successfully create your Docker image.
Your directory may contain additional files, but the ones above are critical to building an image that will run River4. Next, we will create the river4data directory structure. While we River4.js will create these directories if they do not exist, we need to create them so that the SampleSubscriptions.opml file is in a location River4.js expects in order to look for news articles.
Step 3, Create river4data directory structure
Make sure you are in the directory that you created for step 2
Create river4data by entering mkdir river4data
Change to the river4data by entering cd river4data
Create the following directories: data, lists, and rivers by entering the following
The next step is to copy a OPML file to the /river4data/lists/ directory. This file will contain your subscriptions list. I recommend you use SampleSubscriptions.opml, which you copied in Step 2.
Step 4, Copy SampleSubscriptions.opml to /river4data/lists/
Return to the directory that contains the files you packaged together in Step 2. For example, if the folder you created in Step2 is named river 4, enter cd ~/river4
Enter cp SampleSubscriptions.opml ./river4data/lists/
At this point you have all the files placed in their correct locations and you are ready to build your Docker image for River4. The instructions to build the image are contained in Dockerfile, which is in the directory you created in Step 2. You must execute the commands from within that directory.
Step 5, Build your River4 Docker image
If you think you will want to share your River4 Docker image, you will want to create an account on Docker Hub, and you will then use the account name as part of the image name. For example, my account on Docker Hub is fmcpherson and therefore the name for my River4 image is fmcpherson/river4.
The command to build a Docker image is: docker build -t [image name] .
For example, the command that I execute is docker build -t fmcpherson/river 4 .
Note that the space and period at the end of the command is important, it indicates that the current directory contains the source files for the image. Again, be sure that Dockerfile is in the current directory as it provides the instructions for the docker build.
If all goes well, your River4 Docker image will be built and added to the list of images on your computer. If you enter docker images you should see your River4 image listed. At this point y0u have successfully created your Docker River4 image.
Run Your River4 Image
To run your River4 image, enter docker run --name river4 -d -p 1337:1337 [image name] for example, the command that I enter to run my River4 image is docker run --name river4 -d -p 1337:1337 fmcpherson/river4.
Here is a brief explanation for the run command:
--name provides a name for the container, if you don't provide this Docker will automatically create one.
-d specifies to run the container in the background. If you are familiar with it, this is similar to the forever utility.
-p maps external network ports to internal network ports, in the format external port : internal port. The internal port 1337 is the port that River4 is listening to, the external port specifies which port an external computer uses to communicate with the container. For example, to open your river you can enter localhost:1337 in the address bar in your web browser.
To confirm that your container is running you can use the docker logs command to see the River4 console output. Enter docker logs river4, assuming you named your container as in my example above.
At this point, River4 is running in a Docker container on your computer. You can open a web browser, enter localhost:1337 in the address bar, and the River4 home page will load and over time it will display the title of news articles obtained from the sites in your reading list. Note that if you are running the container in a virtual machine on your computer you will need to enter the IP address for the virtual machine and not use localhost. There may be a variety of reasons why y0u cannot access the River4 site if it is host3ed in a virtual machine, and troubleshooting them is beyond the scope of this article. The most likely cause of any access problems with virtual machines is that the networking is not properly set up for the virtual machine.
If you use Kitematic to run the River4 container it will display the ip address and port number that you need to enter in the web browser address bar to open the River4 home page.
To Stop Your River4 Container
As a tool to learn Docker, I have created a Docker image to create a container that hosts and runs the River4 RSS Agreggator. The image is available in a public repository in the Docker Hub. The container has everything needed to run River4, including a sample RSS reading list. The reading list OPML file within the container includes my personal RSS subscriptions that I use in my personal river. Unfortunately, the current iteration does not include a way to make changes to the reading list, nor is the application state saved when you stop the container. I have more studying to do about how persistence and data storage is done in containers.
Here are some notes for how to obtain and use the container for those who are familiar with Docker.
Obtain the image by running docker pull fmcpherson/river4
To run River4 as a background process using the default port 1337 enter docker run -p 1337:1337 -d fmcpherson/river4. Note that the -p 1337:1337 portion of this statement provides the port mapping, the left side of the colon indicates the host server port, the right side has the port the application is expecting for communication. If you want you can map River4 to port 80 by entering -p 80:1337 in the run command.
If you want to see the server output of the server running in the background, you need to first get the container ID using the docker ps command, then enter docker logs [container id]
If you want to run River4 interactively so that you see the server output enter docker run -p 1337:1337 -t -i fmcpherson/river4 /bin/bash
If you are running Docker on a physical computer, you can access the River4 app by entering http://localhost:1337 in a web browser
If you are running Docker on a virtual server, or if you want to access the River4 app from a computer on your network other than the one hosting Docker, you will need to obtain the IP address of the host and then enter http://[Host IP address]:1337
If you are not familiar with Docker, or just heard a little bit about it, but are interested in trying it on a Windows or Mac, the simplest way to get started is by using Kitematic. Kitematic provides a graphical interface to Docker, which is otherwise a command line tool. Be aware that Kitematic uses Virtual Box for hosting Docker and any containers.
If you are familiar with Docker and you want to look at the Dockerfile that I wrote to create this image, you will find it in my DockerizedRiver4 Github repository.