update docs, support headers, expand user/pass
This commit is contained in:
parent
dc59f44322
commit
e1209f36b7
186
README.md
186
README.md
|
@ -15,26 +15,201 @@ pushd watchdog.go/
|
||||||
go run ./watchdog.go -c dog.json
|
go run ./watchdog.go -c dog.json
|
||||||
```
|
```
|
||||||
|
|
||||||
|
# Getting Started
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>How do I configure what to watch?</summary>
|
||||||
|
|
||||||
|
### `name`
|
||||||
|
|
||||||
|
This is an arbitrary name which can be used as `{{ .Name }}` in the template strings.
|
||||||
|
|
||||||
|
In most cases I would probably set mine to the literal domain name (such as `git.rootprojects.org`),
|
||||||
|
but you can do whatever makes sense to you.
|
||||||
|
|
||||||
|
If you use it in a template, you probably shouldn't allow it to be arbitrary user input.
|
||||||
|
|
||||||
|
### `url`
|
||||||
|
|
||||||
|
This is the page the watchdog will check for the exact match of `keywords` you set.
|
||||||
|
|
||||||
|
It's a get request. I actually want to be regularly check that contact forms are working,
|
||||||
|
so I may very well add more functionality to this in the future.
|
||||||
|
|
||||||
|
### `keywords`
|
||||||
|
|
||||||
|
The `url` will be checked for a literal, exact match of keywords.
|
||||||
|
|
||||||
|
Be careful of "smart quotes" and HTML entities:
|
||||||
|
|
||||||
|
- `We’re Open!` is not `We're Open!`
|
||||||
|
- Neither is `We're Open!` nor `We're Open!`
|
||||||
|
|
||||||
|
### `webhooks`
|
||||||
|
|
||||||
|
This references the arbitrary `name` of a webhook in the `webhooks` array.
|
||||||
|
|
||||||
|
These webhooks will be run in order and the next will still be run even when the previous fails.
|
||||||
|
|
||||||
|
### `recover_script`
|
||||||
|
|
||||||
|
The full contents of this file will be passed to bash as if it were a file.
|
||||||
|
|
||||||
|
You can run a single line, such as `ssh watchdog@my.example.com 'systemctl restart foo.service'`
|
||||||
|
or an entire script.
|
||||||
|
|
||||||
|
#### Pro-Tip™
|
||||||
|
|
||||||
|
Don't forget that you can add single-use ssh keys to run specific commands on a remote system.
|
||||||
|
|
||||||
|
On the system that runs the watchdog you would create a new, single-use key, like this:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh-keygen -N '' -f ~/.ssh/restart-service-foo -C "For Foo's Watchdog"
|
||||||
|
cat ~/.ssh/restart-service-foo.pub
|
||||||
|
```
|
||||||
|
|
||||||
|
On the system that runs the service being watched you would add that key restricted to the single command:
|
||||||
|
|
||||||
|
`/root/.ssh/authorized_keys`:
|
||||||
|
|
||||||
|
```
|
||||||
|
command="systemctl restart foo.service",no-port-forwarding,no-x11-forwarding,no-agent-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5HKrWMTkrInDNGMeEWu/bSc2RCCpUPBbZCh1hWIASzMUlPDdTtqwe6ve4aozbeSVFm0mTfBlNPWPqf1ZAx80lQTbYIPNsPlQw0ktuFPWHqDGayQuPBwtk7PlzOcRah29EZ/gbz4vQCDk5G1AygBBt9S3US7Q+/xi5mk/bKkmenpYUwBrpaHx/hm4xY/6qThZDh+tf5CvTnbnb3tVS8ldOIOHMMBtxkzOcgxu12gJV3cHg/xvNd1fTrcshnjnldQphhW0/g068Ibz6aabjuy5h89uVvbEv74wV5CH7XS0TsuOIhv9/dKoi9XBRI9oM4RgPNLWxZETOGzKGYOHqxcmL For Foo's Watchdog
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<details>
|
||||||
|
<summary>{{ .Name }} and other template variables</summary>
|
||||||
|
|
||||||
|
`{{ .Name }}` is the only template variable right now.
|
||||||
|
|
||||||
|
It refers to the name of the watch, which is "Example Site" in the sample config below.
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<details>
|
||||||
|
<summary>How to use with Mailgun</summary>
|
||||||
|
|
||||||
|
### `my_mailgun`
|
||||||
|
|
||||||
|
Replace `my_mailgun` with whatever name you like, or leave it the same.
|
||||||
|
|
||||||
|
It's an arbitrary name for you to reference.
|
||||||
|
For example, you may have different mailgun configurations for different domains.
|
||||||
|
|
||||||
|
### `my.example.com`
|
||||||
|
|
||||||
|
Replace both instances of `my.example.com` with your domain in mailgun.
|
||||||
|
|
||||||
|
### `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
|
||||||
|
|
||||||
|
Replace the HTTP Basic Auth "password" `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
|
||||||
|
with your API token.
|
||||||
|
|
||||||
|
`api` stays, it is the expected "username".
|
||||||
|
|
||||||
|
### `to`, `from`, `subject`, & `text`
|
||||||
|
|
||||||
|
Hopefully it's obivous enough that you should send your alerts to yourself,
|
||||||
|
not `jon.doe@gmail.com` but, just in case: change that to your email.
|
||||||
|
|
||||||
|
The `from` address can be _any_ domain in your mailgun account.
|
||||||
|
|
||||||
|
`subject` is the plain-text subject and `text` must be plain-text (not html) email contents.
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<details>
|
||||||
|
<summary>How to use with Twilio</summary>
|
||||||
|
|
||||||
|
### `my_twilio`
|
||||||
|
|
||||||
|
Replace `my_twilio` with whatever name you like, or leave it the same.
|
||||||
|
|
||||||
|
It's an arbitrary name for you to reference.
|
||||||
|
For example, you may have different twilio configurations for different domains.
|
||||||
|
|
||||||
|
### `AC00000000000000000000000000000000`
|
||||||
|
|
||||||
|
This is a placeholder for your `sid`, also called "Account SID".
|
||||||
|
|
||||||
|
Replace both instances of `AC00000000000000000000000000000000` with the `sid` shown in your twilio dashboard.
|
||||||
|
|
||||||
|
This is used both in the `url` as well as in the `auth`.
|
||||||
|
|
||||||
|
### `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
|
||||||
|
|
||||||
|
Replace the HTTP Basic Auth "password" `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
|
||||||
|
with your API token.
|
||||||
|
|
||||||
|
You'll find this on the twilio dashboard with your "Account SID",
|
||||||
|
but you'll have to click to reveal it.
|
||||||
|
|
||||||
|
### `To`, `From`, & `Body`
|
||||||
|
|
||||||
|
All phone numbers should have the country code prefix (`+1` for USA) attached.
|
||||||
|
|
||||||
|
`To` is the number that you want to send the text to.
|
||||||
|
|
||||||
|
`From` must be a number listed in your twilio account.
|
||||||
|
|
||||||
|
`Body` is a plain-text short message. Remember K.I.S.S: "keep it short 'n sweet".
|
||||||
|
|
||||||
|
</details>
|
||||||
|
<details>
|
||||||
|
<summary>How to use with Nexmo, Mailjet, and other webhook-enabled services</summary>
|
||||||
|
|
||||||
|
See the examples of Twilio and Mailgun.
|
||||||
|
|
||||||
|
Look for "curl" in the documentation of the service that you're using.
|
||||||
|
It should be fairly easy to just look at the headers that are being set and repeat.
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
## Sample Config
|
## Sample Config
|
||||||
|
|
||||||
|
You can set notifications for _any_ service that supports HTTPS webhooks.
|
||||||
|
|
||||||
|
The examples below are shown with Twilio and Mailgun, as taken from their `curl` documentation.
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"watches": [
|
"watches": [
|
||||||
{
|
{
|
||||||
"name": "Example Site",
|
"name": "Example Site",
|
||||||
"url": "https://example.com/",
|
"url": "https://example.com/",
|
||||||
"webhooks": ["twilio"],
|
|
||||||
"keywords": "My Site",
|
"keywords": "My Site",
|
||||||
|
"webhooks": ["my_mailgun", "my_twilio"],
|
||||||
"recover_script": "systemctl restart example-site"
|
"recover_script": "systemctl restart example-site"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"webhooks": [
|
"webhooks": [
|
||||||
{
|
{
|
||||||
"name": "twilio",
|
"name": "my_mailgun",
|
||||||
|
"method": "POST",
|
||||||
|
"url": "https://api.mailgun.net/v3/my.example.com/messages",
|
||||||
|
"auth": {
|
||||||
|
"username": "api",
|
||||||
|
"password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
|
||||||
|
},
|
||||||
|
"headers": {
|
||||||
|
"User-Agent": "Watchdog/1.0",
|
||||||
|
},
|
||||||
|
"form": {
|
||||||
|
"from": "Watchdog <watchdog@my.example.com>",
|
||||||
|
"to": "jon.doe@gmail.com",
|
||||||
|
"subject": "{{ .Name }} is down.",
|
||||||
|
"text": "The system is down. Check up on {{ .Name }} ASAP."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "my_twilio",
|
||||||
|
"method": "POST",
|
||||||
"url": "https://api.twilio.com/2010-04-01/Accounts/AC00000000000000000000000000000000/Messages.json",
|
"url": "https://api.twilio.com/2010-04-01/Accounts/AC00000000000000000000000000000000/Messages.json",
|
||||||
"auth": {
|
"auth": {
|
||||||
"user": "AC00000000000000000000000000000000",
|
"username": "AC00000000000000000000000000000000",
|
||||||
"pass": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
|
"password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
|
||||||
|
},
|
||||||
|
"headers": {
|
||||||
|
"User-Agent": "Watchdog/1.0",
|
||||||
},
|
},
|
||||||
"form": {
|
"form": {
|
||||||
"To": "+1 801 555 1234",
|
"To": "+1 801 555 1234",
|
||||||
|
@ -47,6 +222,9 @@ go run ./watchdog.go -c dog.json
|
||||||
```
|
```
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
|
I may want to allow for an overlay of configs for cases like Twilio
|
||||||
|
where it must be run once per contact.
|
||||||
|
|
||||||
"webhooks": [
|
"webhooks": [
|
||||||
{
|
{
|
||||||
"name": "twilio",
|
"name": "twilio",
|
||||||
|
|
28
watchdog.go
28
watchdog.go
|
@ -260,15 +260,26 @@ func (d *Dog) notify(hardFail bool) {
|
||||||
}
|
}
|
||||||
|
|
||||||
if 0 != len(h.Form) {
|
if 0 != len(h.Form) {
|
||||||
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
|
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
}
|
}
|
||||||
|
|
||||||
if 0 != len(h.Auth) {
|
if 0 != len(h.Auth) {
|
||||||
user := h.Auth["user"]
|
user := h.Auth["user"]
|
||||||
|
if "" == user {
|
||||||
|
user = h.Auth["username"]
|
||||||
|
}
|
||||||
pass := h.Auth["pass"]
|
pass := h.Auth["pass"]
|
||||||
|
if "" == user {
|
||||||
|
pass = h.Auth["password"]
|
||||||
|
}
|
||||||
req.SetBasicAuth(user, pass)
|
req.SetBasicAuth(user, pass)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
req.Header.Set("User-Agent", "Watchdog/1.0")
|
||||||
|
for k := range h.Headers {
|
||||||
|
req.Header.Set(k, h.Headers[k])
|
||||||
|
}
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
resp, err := client.Do(req)
|
||||||
if nil != err {
|
if nil != err {
|
||||||
d.logger <- fmt.Sprintf("[Notify] HTTP Client Error: %s", err)
|
d.logger <- fmt.Sprintf("[Notify] HTTP Client Error: %s", err)
|
||||||
|
@ -313,11 +324,13 @@ type ConfigWebhook struct {
|
||||||
Method string `json:"method"`
|
Method string `json:"method"`
|
||||||
URL string `json:"url"`
|
URL string `json:"url"`
|
||||||
Auth map[string]string `json:"auth"`
|
Auth map[string]string `json:"auth"`
|
||||||
|
Headers map[string]string `json:"headers"`
|
||||||
Form map[string]string `json:"form"`
|
Form map[string]string `json:"form"`
|
||||||
Config map[string]string `json:"config"`
|
Config map[string]string `json:"config"`
|
||||||
Configs []map[string]string `json:"configs"`
|
Configs []map[string]string `json:"configs"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The default http client uses unsafe defaults
|
||||||
func NewHTTPClient() *http.Client {
|
func NewHTTPClient() *http.Client {
|
||||||
transport := &http.Transport{
|
transport := &http.Transport{
|
||||||
Dial: (&net.Dialer{
|
Dial: (&net.Dialer{
|
||||||
|
@ -332,20 +345,11 @@ func NewHTTPClient() *http.Client {
|
||||||
return client
|
return client
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// This is so that the log messages don't trample
|
||||||
|
// over each other when they happen simultaneously.
|
||||||
func logger(msgs chan string) {
|
func logger(msgs chan string) {
|
||||||
for {
|
for {
|
||||||
msg := <-msgs
|
msg := <-msgs
|
||||||
log.Println(msg)
|
log.Println(msg)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/*
|
|
||||||
request :+ http.NewRequest("POST"
|
|
||||||
curl -s --user 'api:YOUR_API_KEY' \
|
|
||||||
https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \
|
|
||||||
-F from='Excited User <mailgun@YOUR_DOMAIN_NAME>' \
|
|
||||||
-F to=YOU@YOUR_DOMAIN_NAME \
|
|
||||||
-F to=bar@example.com \
|
|
||||||
-F subject='Hello' \
|
|
||||||
-F text='Testing some Mailgun awesomeness!'
|
|
||||||
*/
|
|
||||||
|
|
Loading…
Reference in New Issue