|
|
@ -15,26 +15,201 @@ pushd watchdog.go/ |
|
|
|
go run ./watchdog.go -c dog.json |
|
|
|
``` |
|
|
|
|
|
|
|
# Getting Started |
|
|
|
|
|
|
|
<details> |
|
|
|
<summary>How do I configure what to watch?</summary> |
|
|
|
|
|
|
|
### `name` |
|
|
|
|
|
|
|
This is an arbitrary name which can be used as `{{ .Name }}` in the template strings. |
|
|
|
|
|
|
|
In most cases I would probably set mine to the literal domain name (such as `git.rootprojects.org`), |
|
|
|
but you can do whatever makes sense to you. |
|
|
|
|
|
|
|
If you use it in a template, you probably shouldn't allow it to be arbitrary user input. |
|
|
|
|
|
|
|
### `url` |
|
|
|
|
|
|
|
This is the page the watchdog will check for the exact match of `keywords` you set. |
|
|
|
|
|
|
|
It's a get request. I actually want to be regularly check that contact forms are working, |
|
|
|
so I may very well add more functionality to this in the future. |
|
|
|
|
|
|
|
### `keywords` |
|
|
|
|
|
|
|
The `url` will be checked for a literal, exact match of keywords. |
|
|
|
|
|
|
|
Be careful of "smart quotes" and HTML entities: |
|
|
|
|
|
|
|
- `We’re Open!` is not `We're Open!` |
|
|
|
- Neither is `We're Open!` nor `We're Open!` |
|
|
|
|
|
|
|
### `webhooks` |
|
|
|
|
|
|
|
This references the arbitrary `name` of a webhook in the `webhooks` array. |
|
|
|
|
|
|
|
These webhooks will be run in order and the next will still be run even when the previous fails. |
|
|
|
|
|
|
|
### `recover_script` |
|
|
|
|
|
|
|
The full contents of this file will be passed to bash as if it were a file. |
|
|
|
|
|
|
|
You can run a single line, such as `ssh watchdog@my.example.com 'systemctl restart foo.service'` |
|
|
|
or an entire script. |
|
|
|
|
|
|
|
#### Pro-Tip™ |
|
|
|
|
|
|
|
Don't forget that you can add single-use ssh keys to run specific commands on a remote system. |
|
|
|
|
|
|
|
On the system that runs the watchdog you would create a new, single-use key, like this: |
|
|
|
|
|
|
|
```bash |
|
|
|
ssh-keygen -N '' -f ~/.ssh/restart-service-foo -C "For Foo's Watchdog" |
|
|
|
cat ~/.ssh/restart-service-foo.pub |
|
|
|
``` |
|
|
|
|
|
|
|
On the system that runs the service being watched you would add that key restricted to the single command: |
|
|
|
|
|
|
|
`/root/.ssh/authorized_keys`: |
|
|
|
|
|
|
|
``` |
|
|
|
command="systemctl restart foo.service",no-port-forwarding,no-x11-forwarding,no-agent-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5HKrWMTkrInDNGMeEWu/bSc2RCCpUPBbZCh1hWIASzMUlPDdTtqwe6ve4aozbeSVFm0mTfBlNPWPqf1ZAx80lQTbYIPNsPlQw0ktuFPWHqDGayQuPBwtk7PlzOcRah29EZ/gbz4vQCDk5G1AygBBt9S3US7Q+/xi5mk/bKkmenpYUwBrpaHx/hm4xY/6qThZDh+tf5CvTnbnb3tVS8ldOIOHMMBtxkzOcgxu12gJV3cHg/xvNd1fTrcshnjnldQphhW0/g068Ibz6aabjuy5h89uVvbEv74wV5CH7XS0TsuOIhv9/dKoi9XBRI9oM4RgPNLWxZETOGzKGYOHqxcmL For Foo's Watchdog |
|
|
|
``` |
|
|
|
|
|
|
|
</details> |
|
|
|
<details> |
|
|
|
<summary>{{ .Name }} and other template variables</summary> |
|
|
|
|
|
|
|
`{{ .Name }}` is the only template variable right now. |
|
|
|
|
|
|
|
It refers to the name of the watch, which is "Example Site" in the sample config below. |
|
|
|
|
|
|
|
</details> |
|
|
|
<details> |
|
|
|
<summary>How to use with Mailgun</summary> |
|
|
|
|
|
|
|
### `my_mailgun` |
|
|
|
|
|
|
|
Replace `my_mailgun` with whatever name you like, or leave it the same. |
|
|
|
|
|
|
|
It's an arbitrary name for you to reference. |
|
|
|
For example, you may have different mailgun configurations for different domains. |
|
|
|
|
|
|
|
### `my.example.com` |
|
|
|
|
|
|
|
Replace both instances of `my.example.com` with your domain in mailgun. |
|
|
|
|
|
|
|
### `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` |
|
|
|
|
|
|
|
Replace the HTTP Basic Auth "password" `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` |
|
|
|
with your API token. |
|
|
|
|
|
|
|
`api` stays, it is the expected "username". |
|
|
|
|
|
|
|
### `to`, `from`, `subject`, & `text` |
|
|
|
|
|
|
|
Hopefully it's obivous enough that you should send your alerts to yourself, |
|
|
|
not `jon.doe@gmail.com` but, just in case: change that to your email. |
|
|
|
|
|
|
|
The `from` address can be _any_ domain in your mailgun account. |
|
|
|
|
|
|
|
`subject` is the plain-text subject and `text` must be plain-text (not html) email contents. |
|
|
|
|
|
|
|
</details> |
|
|
|
<details> |
|
|
|
<summary>How to use with Twilio</summary> |
|
|
|
|
|
|
|
### `my_twilio` |
|
|
|
|
|
|
|
Replace `my_twilio` with whatever name you like, or leave it the same. |
|
|
|
|
|
|
|
It's an arbitrary name for you to reference. |
|
|
|
For example, you may have different twilio configurations for different domains. |
|
|
|
|
|
|
|
### `AC00000000000000000000000000000000` |
|
|
|
|
|
|
|
This is a placeholder for your `sid`, also called "Account SID". |
|
|
|
|
|
|
|
Replace both instances of `AC00000000000000000000000000000000` with the `sid` shown in your twilio dashboard. |
|
|
|
|
|
|
|
This is used both in the `url` as well as in the `auth`. |
|
|
|
|
|
|
|
### `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` |
|
|
|
|
|
|
|
Replace the HTTP Basic Auth "password" `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` |
|
|
|
with your API token. |
|
|
|
|
|
|
|
You'll find this on the twilio dashboard with your "Account SID", |
|
|
|
but you'll have to click to reveal it. |
|
|
|
|
|
|
|
### `To`, `From`, & `Body` |
|
|
|
|
|
|
|
All phone numbers should have the country code prefix (`+1` for USA) attached. |
|
|
|
|
|
|
|
`To` is the number that you want to send the text to. |
|
|
|
|
|
|
|
`From` must be a number listed in your twilio account. |
|
|
|
|
|
|
|
`Body` is a plain-text short message. Remember K.I.S.S: "keep it short 'n sweet". |
|
|
|
|
|
|
|
</details> |
|
|
|
<details> |
|
|
|
<summary>How to use with Nexmo, Mailjet, and other webhook-enabled services</summary> |
|
|
|
|
|
|
|
See the examples of Twilio and Mailgun. |
|
|
|
|
|
|
|
Look for "curl" in the documentation of the service that you're using. |
|
|
|
It should be fairly easy to just look at the headers that are being set and repeat. |
|
|
|
|
|
|
|
</details> |
|
|
|
|
|
|
|
## Sample Config |
|
|
|
|
|
|
|
You can set notifications for _any_ service that supports HTTPS webhooks. |
|
|
|
|
|
|
|
The examples below are shown with Twilio and Mailgun, as taken from their `curl` documentation. |
|
|
|
|
|
|
|
```json |
|
|
|
{ |
|
|
|
"watches": [ |
|
|
|
{ |
|
|
|
"name": "Example Site", |
|
|
|
"url": "https://example.com/", |
|
|
|
"webhooks": ["twilio"], |
|
|
|
"keywords": "My Site", |
|
|
|
"webhooks": ["my_mailgun", "my_twilio"], |
|
|
|
"recover_script": "systemctl restart example-site" |
|
|
|
} |
|
|
|
], |
|
|
|
"webhooks": [ |
|
|
|
{ |
|
|
|
"name": "twilio", |
|
|
|
"name": "my_mailgun", |
|
|
|
"method": "POST", |
|
|
|
"url": "https://api.mailgun.net/v3/my.example.com/messages", |
|
|
|
"auth": { |
|
|
|
"username": "api", |
|
|
|
"password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" |
|
|
|
}, |
|
|
|
"headers": { |
|
|
|
"User-Agent": "Watchdog/1.0", |
|
|
|
}, |
|
|
|
"form": { |
|
|
|
"from": "Watchdog <watchdog@my.example.com>", |
|
|
|
"to": "jon.doe@gmail.com", |
|
|
|
"subject": "{{ .Name }} is down.", |
|
|
|
"text": "The system is down. Check up on {{ .Name }} ASAP." |
|
|
|
} |
|
|
|
}, |
|
|
|
{ |
|
|
|
"name": "my_twilio", |
|
|
|
"method": "POST", |
|
|
|
"url": "https://api.twilio.com/2010-04-01/Accounts/AC00000000000000000000000000000000/Messages.json", |
|
|
|
"auth": { |
|
|
|
"user": "AC00000000000000000000000000000000", |
|
|
|
"pass": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" |
|
|
|
"username": "AC00000000000000000000000000000000", |
|
|
|
"password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" |
|
|
|
}, |
|
|
|
"headers": { |
|
|
|
"User-Agent": "Watchdog/1.0", |
|
|
|
}, |
|
|
|
"form": { |
|
|
|
"To": "+1 801 555 1234", |
|
|
@ -47,6 +222,9 @@ go run ./watchdog.go -c dog.json |
|
|
|
``` |
|
|
|
|
|
|
|
<!-- |
|
|
|
I may want to allow for an overlay of configs for cases like Twilio |
|
|
|
where it must be run once per contact. |
|
|
|
|
|
|
|
"webhooks": [ |
|
|
|
{ |
|
|
|
"name": "twilio", |
|
|
|