Lightweight command-line process control
Procdog (as in "process watchdog"... get it?) is a simple command-line tool to start, stop, and check the health of processes. It works with any kind of process you can invoke from the command line (be it native, Java, Node, Python, or anything) on MacOS or Linux.
Why would you want another tool for this?
- For basic interactive situations, you can just run a process in a terminal or use job management in your shell (
- But you rapidly realize this won't work well once you have longer-lived processes or are
sshing to remote servers. Then you could use
nohupand manually checking with
ps, or using screen. But these don't script easily.
- You also might want to script starting and stopping. With a little effort you might write a custom Bash script (writing a PID file, using
pkill, etc.), but it's a hassle and gets messy quickly.
- Of course, you can just "do it right." Traditionally, in the Unix world, the way to control services are System V service scripts, start-stop-daemon, Upstart, or systemd. Using these is essential for production deployment, but they tend to be a bit arcane and highly OS-dependent, so aren't as convenient for casual use, and you can't easily develop and test on both MacOS and Linux (as many of us try to do).
Procdog is an alternative for developers that attempts to be easy to install, simple and obvious to use, and cross-platform. Processes are independent of the shell used to invoke them (i.e., detatched as with
nohup) and you can also check status or kill them at any time (as with a Unix service).
Probably the most simlar tool to Procdog is Supervisor. Although it has similar goals, Supervisor is a more complex, production-oriented tool with a long-lived, centrailized XML-RPC server that monitors processes. Procdog is intended to be a single, simple, easy-to-use command that needn't be set up and managed itself or run as root. Each use of Procdog is independent and monitors just one process. It does not require a configuration file unless you want to save typing. You can even check the Procdog script into your own project so developers can immediately use it locally, since it has no dependencies (besides Python 2.7).
You'll find it useful if you have servers, databases, or other processes you want to manage on your personal machine when developing, in build systems and test harnesses, test deployments, etc. Currently, it doesn't have restart logic, log file rotation, or some other features you may want for a production environments; for this consider Supervisor, Upstart, systemd, & co. Procdog is also way less mature than these alternatives.
No dependencies except Python 2.7+. It's easiest to install with pip:
sudo pip install procdog
Or, since it's just one file, you can copy the single
procdog script somewhere (perhaps within your own project) and make it executable.
Now you can start and monitor any process (here let's pick "sleep" -- not all that useful, but you already have it):
$ procdog start myprocess --command "sleep 100" running, pid=14969 $ procdog status myprocess running, pid=14969 $ procdog stop myprocess stopped $
Note you have to give your process an arbitrary name (
myprocess here), like a Unix service name, so you can refer to it. Once the process is done, the monitor daemon also exits.
procdog -h for help on all options.
A better example
Now, with a real server we'd like to know if it's actually up and doing something, like listening on a port. Procdog supports an arbitrary command to test health (e.g. running
curl to see if it returns a result) and can wait until a server is running and tell you.
$ procdog start backend --command="java -classpath my-backend.jar com.example.BackendServer server config.yml" \ --health-command="curl http://localhost:8080/ping" \ --health-count=10 --health-delay=2 \ --dir=$HOME/backend \ --stdout=backend.log --stderr=backend.log --append \ --ensure-healthy --strict running, health=0, pid=15240 $ procdog status backend running, health=0, pid=15240 $ procdog stop backend --strict stopped $ procdog stop backend --strict procdog: error: process 'backend' is not running $
Some notes on this:
- We can specify where to write stdout and stderr. They can be the same or different. Existing log files are appended to if you use
- The health command simply calls a shell command to see if the server is healthy. The return code of the health check command must be
0for the server to be considered healthy. In this case, we're callin
curlon a known health-check URL, which will have return code 0 on an HTTP 200
--ensure-healthyoption means the command will block until the process is healthy, or until the daemon gives up and kills the process (if necessary). In this example, it will try 10 times, sleeping 2 seconds each time, before giving up.
--diroption means process will run from that directory.
- We ask the client to be
--strict, so that it returns non-zero status code when we try to start a process that's already running or or stop one that is already stopped.
It's possible to avoid typing by putting most options in a configuration file:
# Procdog config file. Each section is a process name. [backend] command=java -classpath my-backend.jar com.example.BackendServer server config.yml health_command=curl http://localhost:8080/ping health_count=10 health_delay=2. dir=$HOME/backend stdout=backend.log stderr=backend.log append=False ensure_healthy=True strict=True
You have any number of sections, one section per process. Procdog reads options from
procdog.cfg (in the same directory the
procdog script resides). Any options given on the command line override those in the configuration file. Note that environment variables (like
$HOME) are allowed and expanded. Once you have the above section in your config file, you can run:
$ procdog start backend running, health=0, pid=15396 $ procdog stop backend stopped $
How it works
Procdog starts a small daemon that
popen()s and monitors the process. The daemon listens and accepts commands on a local Unix domain socket, making it possible to check the process is running or terminate it, and to do simple health checks. You can see these sockets at
/var/tmp/procdog.*.sock (where the * is the id of the process). For simplicity, there is a single Procdog daemon for each monitored process, so each process is handled compeltely separately.
We use Unix domain sockets so that we don't have the headaches of pid files or choosing and binding to TCP ports. They're also available on most platforms.
Daemon logs are sent to
/var/tmp/procdog.*.log. Usually you'll want to redirect your process stdout and stderr using the
Procdog is quite new so probably not stable. Bug reports and contributions are welcome!
All basic features are covered with a simple Bash-based regression test.
To run it, clone this repo, invoke
tests/run.sh then follow the directions to diff the output using