Refactoring ImagesOf

The Vision

As it sits right now, this is just a little collection of scripts that runs the ImagesOfNetwork on reddit. They do the job, but they're very fragile. There's no central location for configuration, it depends on various configuration files and settings defined ad-hoc within the source code. If you decide to switch hosts at some point, I suspect it'll be painful to get everything set back up. My initial goal is clean up the source, package the python, and make the installation repeatable.

The first PR creates a setup.py which will install the scripts as they are. It also serves to automatically install all dependencies for the scripts (at this point, PRAW and it's dependencies), with the correctly pinned versions. The version pinning is especially important since these scripts don't work with the most recent PRAW.

The next step is to pull configuration out, either into a configuration file, most likely configparser style, or maybe more straightforward, into a images_of.settings module.

Now, we have an images_of module to work with, we can consolidate common functionality. A simple example of something we could do is to extend the praw.Reddit class to automatically auth us from our settings.

Working with virtualenv

A common paradigm in writing python scripts is to have a shebang line that looks like #!/usr/bin/env python. There are two good ways to specify which python version to use: 1) call as python2 script.py or python3 script.py, or 2) Use a virtualenv.

The virtualenv option is especially nice for development because it won't clutter up your system's python installation, and you can be certain you've only installed the modules you asked for - if something's missing from your configuration, you'll find out real quick. You can activate and deactivate it at will, and you can start over whenever you need to. It's pretty straight forward to use.

First, make sure you've got virtualenv installed, either with apt or pip.

# apt-get install python3-virtualenv
   OR
# pip3 install virtualenv

Now, from the root of your repository (or really anywhere, but that's a pretty good spot, since it's what you're working with) we'll create a virtualenv called venv, and make sure we're using python3

$ virtualenv3 -v venv -p python3
$ source venv/bin/activate
(venv) $ python setup.py install

While you're inside the virtualenv, the it's name is displayed in parentheses in your prompt. If you run python now, it'll be the python you've set up the virtualenv with, not the system default. This is important, because as noted earlier, a lot of scripts start with #!/usr/bin/env python, and now we know we're using the right python. Also, as long as you are inside your virtualenv, the scripts and entry points installed by setup.py (copy_wiki_pages.py, etc) will be available in your $PATH, but when you deactivate your virtualenv (simply issue the command deactivate), they're no longer available.

Developing

Instead of using setup.py install, we can use setup.py develop. This just symlinks some things instead of copying the files into place. That way, when you make changes to the source, you don't have to reinstall or anything, just re-run the programs.

Running a virtualenv with supervisord

It looks like there are various ways of setting up supervisord to play nicely with virtualenv. This stack overflow post has a few solutions, or you (or I) could set up a wrapper script to source the virtualenv prior to executing the script.

The most straightforward way I can see is to adjust your path to the installed executable. You'll set your working directoy to where it was, then just use /path/to/venv/bin/imagesof_bot.py and that will use the right python and the virtual environment. It should look something like

[program:imagesof_bot]
directory=/path/to/code/or/configuration
command=/path/to/venv/bin/imagesof_bot.py

or more concrete, where all the configuration lives at the code root, and my virtualenv is named venv-imagesof (and my name is foo apparently):

[program:imagesof_bot]
directory=/home/foo/code/ImagesOfNetwork
command=/home/foo/code/ImagesOfNetwor/venv-imagesof/bin/imagesof_bot.py

Special bonus, it works with python setup.py develop as well as install

If there is more than one program that needs to be run out of the virtualenv, you could even do something like

[group:imagesof]
programs=imagesof_bot,subreddit_setup_and_copy
directory=/home/foo/code/ImagesOfNetwork
user=foo
...
[program:imagesof_bot]
command=/home/foo/code/ImagesOfNetwork/venv-imagesof/bin/images_of_bot.py
[program:subreddit_setup_and_copy]
command=/home/foo/code/ImagesOfNetwork/venv-imagesof/bin/subreddit_setup_and_copy.py

Other Note

This first PR, I've realized that I can refactor so that it won't break any of your configuration by simply skipping the move to scripts/. However, I'd also like to change scripts to entry_points at some point, and that would make the same changes as this does necessary. It does delay making breaking changes for the time being though.