add more CLI options and API scaffolding

This commit is contained in:
AJ ONeal 2020-10-10 18:03:16 -06:00
parent d244cbf589
commit 24c7720831
28 changed files with 1452 additions and 252 deletions

7
.gitignore vendored
View File

@ -1,5 +1,12 @@
/goserv /goserv
*.claims.json
*.jwk.json
*.jws.json
*.jwt.txt
xversion.go
*_string.go
*_vfsdata.go *_vfsdata.go
*.env *.env
.env .env

37
.goreleaser.yml Normal file
View File

@ -0,0 +1,37 @@
# This is an example goreleaser.yaml file with some sane defaults.
# Make sure to check the documentation at http://goreleaser.com
before:
hooks:
# You may remove this if you don't use go modules.
- go mod download
# you may remove this if you don't need go generate
- go generate ./...
builds:
- env:
- CGO_ENABLED=0
goos:
- linux
- windows
- darwin
goarch:
- amd64
- arm64
- arm
archives:
- replacements:
darwin: Darwin
linux: Linux
windows: Windows
386: i386
amd64: x86_64
arm64: aarch64
checksum:
name_template: 'checksums.txt'
snapshot:
name_template: "{{ .Tag }}-next"
changelog:
sort: asc
filters:
exclude:
- '^docs:'
- '^test:'

2
.ignore Normal file
View File

@ -0,0 +1,2 @@
public/vendor
vendor/

205
README.md
View File

@ -1,38 +1,207 @@
# goserv # [GoServ](https://git.coolaj86.com/coolaj86/goserv)
!["golang gopher with goggles"](https://golang.org/lib/godoc/images/footer-gopher.jpg)
> Boilerplate for how I like to write a backend web service > Boilerplate for how I like to write a backend web service
## Build ## Build
#### Goreleaser
See <https://webinstall.dev/goreleaser>
Local-only
```bash
goreleaser --snapshot --skip-publish --rm-dist
```
Publish
```bash
# Get a token at https://github.com/settings/tokens
export GITHUB_TOKEN=xxxxxxx
# Remove --snapshot to error on non-clean releases
goreleaser --snapshot --rm-dist
```
The platform, publish URL, and token file can be changed in `.goreleaser.yml`:
```yml
env_files:
gitea_token: ~/.config/goreleaser/gitea_token
gitea_urls:
api: https://try.gitea.io/api/v1/
```
#### Manually
```bash
git clone ssh://gitea@git.coolaj86.com:22042/coolaj86/goserv.git ./goserv
pushd ./goserv
bash ./examples/build.sh
```
```bash ```bash
export GOFLAGS="-mod=vendor" export GOFLAGS="-mod=vendor"
go mod tidy go mod tidy
go mod vendor go mod vendor
go generate -mod=vendor ./... go generate -mod=vendor ./...
go build -mod=vendor . go build -mod=vendor -o dist/goserv .
``` ```
To build for another platform (such as Raspberry Pi) set `GOOS` and GOARCH`:
```bash ```bash
./goserv run --listen :3000 --serve-path ./overrides GOOS=linux GOARCH=arm64 go build -mod=vendor -o dist/goserv-linux-arm64 .
``` ```
## Eamples and Config Templates #### Run
```bash
./dist/goserv run --listen :3000 --trust-proxy --serve-path ./overrides
```
## Examples and Config Templates
The example files are located in `./examples` The example files are located in `./examples`
- Caddyfile (web server config) - Caddyfile (web server config)
- .env (environment variables) - .env (environment variables)
- build.sh - build.sh
```bash
export BASE_URL=https://example.com
```
### Authentication
You can use an OIDC provider or sign your own tokens.
```bash
go install -mod=vendor git.rootprojects.org/root/keypairs/cmd/keypairs
# Generate a keypair
keypairs gen -o key.jwk.json --pub pub.jwk.json
```
Create an Admin token:
```bash
# Sign an Admin token
echo '{ "sub": "random_ppid", "email": "me@example.com", "iss": "'"${BASE_URL}"'" }' \
> admin.claims.json
keypairs sign --exp 1h ./key.jwk.json ./admin.claims.json > admin.jwt.txt 2> admin.jws.json
# verify the Admin token
keypairs verify ./pub.jwk.json ./admin.jwt.txt
```
Create a User token:
```bash
# Sign a User token
echo '{ "sub": "random_ppid", "email": "me@example.com", "iss": "'"${BASE_URL}"'" }' \
> user.claims.json
keypairs sign --exp 1h ./key.jwk.json ./user.claims.json > user.jwt.txt 2> user.jws.json
# verify the User token
keypairs verify ./pub.jwk.json ./user.jwt.txt
```
**Impersonation** can be accomplished by appending the token `sub` (aka PPID) to the url as
`?user_id=`.
## REST API
All routes require authentication, except for those at `/api/public`.
```txt
Authentication: Bearer <token>
```
Here's the API, in brief:
```txt
# Demo Mode Only
DELETE /public/reset Drop database and re-initialize
# Public
GET /public/ping Health Check
POST /public/setup <= (none) Bootstrap
# Admin-only
GET /admin/ping (authenticated) Health Check
# User
GET /user/ping (authenticated) Health Check
```
When you `GET` anything, it will be wrapped in the `result`.
#### Bootstrapping
The first user to hit the `/api/setup` endpoint will be the **admin**:
```bash
export TOKEN=$(cat admin.jwt.txt)
```
```bash
curl -X POST "${BASE_URL}/api/public/setup" -H "Authorization: Bearer ${TOKEN}"
```
Then the setup endpoint will be disabled:
```bash
curl -X POST "${BASE_URL}/api/public/setup" -H "Authorization: Bearer ${TOKEN}"
```
## GoDoc
If the documentation is not public hosted you can view it with GoDoc:
```bash
godoc --http :6060
```
- <http://localhost:6060/pkg/git.example.com/example/goserv/>
- <http://localhost:6060/pkg/git.example.com/example/goserv/internal/api>
- <http://localhost:6060/pkg/git.example.com/example/goserv/internal/db>
You can see abbreviated documentation with `go`'s built-in `doc`, for example:
```bash
go doc git.example.com/example/goserv/internal/api
```
<http://localhost:6060>
## Test
Create a test database. It must have `test` in the name.
```sql
DROP DATABASE "goserv_test"; CREATE DATABASE "goserv_test";
```
Run the tests:
```bash
export TEST_DATABASE_URL='postgres://postgres:postgres@localhost:5432/goserv_test'
go test -mod=vendor ./...
```
## Dependencies ## Dependencies
This setup can be run on a VPS, such as Digital Ocean, OVH, or Scaleway This setup can be run on a VPS, such as Digital Ocean, OVH, or Scaleway
for \$5/month with minimal dependencies: for \$5/month with minimal dependencies:
- VPS - VPS
- Caddy - Caddy
- PostgreSQL - PostgreSQL
- Serviceman - Serviceman
**Mac**, **Linux**: **Mac**, **Linux**:
@ -80,7 +249,7 @@ sudo env PATH="$PATH" \
caddy run --config ./Caddyfile caddy run --config ./Caddyfile
``` ```
See the Cheat Sheet at https://webinstall.dev/caddy See the Cheat Sheets at https://webinstall.dev/caddy
and https://webinstall.dev/serviceman and https://webinstall.dev/serviceman
### PostgreSQL (Database) ### PostgreSQL (Database)
@ -106,6 +275,14 @@ psql 'postgres://postgres:postgres@localhost:5432/postgres'
See the Cheat Sheets at https://webinstall.dev/postgres See the Cheat Sheets at https://webinstall.dev/postgres
and https://webinstall.dev/serviceman and https://webinstall.dev/serviceman
## License ## Licenses
Copyright 2020. All rights reserved. Copyright 2020 The GoServ Authors. All rights reserved.
### Exceptions
- `countries.json` LGPL, taken from <https://github.com/stefangabos/world_countries>
- `flags.json` MIT, taken from <https://github.com/matiassingers/emoji-flags>
These are probably also in the Public Domain. \
(gathering the official data from any source would yield the same dataset)

View File

@ -1,5 +1,5 @@
// +build !dev // +build !dev
//go:generate go run -mod vendor github.com/shurcooL/vfsgen/cmd/vfsgendev -source="git.coolaj86.com/coolaj86/goserv/assets".Assets //go:generate go run -mod vendor github.com/shurcooL/vfsgen/cmd/vfsgendev -source="git.example.com/example/goserv/assets".Assets
package assets package assets

View File

@ -1,5 +1,5 @@
// +build !dev // +build !dev
//go:generate go run -mod vendor github.com/shurcooL/vfsgen/cmd/vfsgendev -source="git.coolaj86.com/coolaj86/goserv/assets/configfs".Assets //go:generate go run -mod vendor github.com/shurcooL/vfsgen/cmd/vfsgendev -source="git.example.com/example/goserv/assets/configfs".Assets
package configfs package configfs

View File

@ -0,0 +1,3 @@
-- this is only used for the tests
DROP TABLE IF EXISTS "authn";
DROP TABLE IF EXISTS "events";

View File

@ -8,12 +8,12 @@ CREATE TABLE IF NOT EXISTS "authn" (
"id" TEXT PRIMARY KEY DEFAULT gen_random_uuid(), "id" TEXT PRIMARY KEY DEFAULT gen_random_uuid(),
"ppid" TEXT NOT NULL, "ppid" TEXT NOT NULL,
"email" TEXT NOT NULL, "email" TEXT NOT NULL,
"verified" BOOL NOT NULL DEFAULT FALSE, "verified_at" TIMESTAMPTZ NOT NULL DEFAULT ('0001-01-01 00:00:00' AT TIME ZONE 'UTC'),
"created_at" TIMESTAMP NOT NULL DEFAULT (now() AT TIME ZONE 'UTC'), "created_at" TIMESTAMPTZ NOT NULL DEFAULT (now() AT TIME ZONE 'UTC'),
"updated_at" TIMESTAMP NOT NULL DEFAULT (now() AT TIME ZONE 'UTC'), "updated_at" TIMESTAMPTZ NOT NULL DEFAULT (now() AT TIME ZONE 'UTC'),
"deleted_at" TIMESTAMP NOT NULL DEFAULT ('epoch' AT TIME ZONE 'UTC') "deleted_at" TIMESTAMPTZ NOT NULL DEFAULT ('0001-01-01 00:00:00' AT TIME ZONE 'UTC')
); );
--CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_slug" ON "authn" ("ppid"); --CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_ppid" ON "authn" ("ppid");
CREATE INDEX IF NOT EXISTS "idx_ppid" ON "authn" ("ppid"); CREATE INDEX IF NOT EXISTS "idx_ppid" ON "authn" ("ppid");
CREATE INDEX IF NOT EXISTS "idx_email" ON "authn" ("email"); CREATE INDEX IF NOT EXISTS "idx_email" ON "authn" ("email");
@ -26,8 +26,8 @@ CREATE TABLE IF NOT EXISTS "events" (
"table" TEXT NOT NULL, "table" TEXT NOT NULL,
"record" TEXT NOT NULL, "record" TEXT NOT NULL,
"by" TEXT NOT NULL, "by" TEXT NOT NULL,
"at" TIMESTAMP NOT NULL DEFAULT (now() AT TIME ZONE 'UTC'), "at" TIMESTAMP NOT NULL DEFAULT (now() AT TIME ZONE 'UTC')
); );
--CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_record" ON "events" ("record"); --CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_record" ON "events" ("record");
CREATE INDEX IF NOT EXISTS "idx_record" ON authn ("record"); CREATE INDEX IF NOT EXISTS "idx_record" ON "events" ("record");
CREATE INDEX IF NOT EXISTS "idx_by" ON authn ("by"); CREATE INDEX IF NOT EXISTS "idx_by" ON "events" ("by");

6
doc.go Normal file
View File

@ -0,0 +1,6 @@
// See also
//
// internal/api: http://localhost:6060/pkg/git.example.com/example/project/internal/api
//
// internal/db: http://localhost:6060/pkg/git.example.com/example/project/internal/db
package main

View File

@ -15,6 +15,8 @@ example.com {
# reverse proxy /api to :3000 # reverse proxy /api to :3000
reverse_proxy /api/* localhost:3000 reverse_proxy /api/* localhost:3000
reverse_proxy /.well-known/openid-configuration localhost:3000
reverse_proxy /.well-known/jwks.json localhost:3000
# serve static files from public folder, but not /api # serve static files from public folder, but not /api
@notApi { @notApi {
@ -22,6 +24,8 @@ example.com {
try_files {path} {path}/ {path}/index.html try_files {path} {path}/ {path}/index.html
} }
not path /api/* not path /api/*
not path /.well-known/openid-configuration
not path /.well-known/jwks.json
} }
route { route {
rewrite @notApi {http.matchers.file.relative} rewrite @notApi {http.matchers.file.relative}

1
examples/build.sh Normal file → Executable file
View File

@ -8,4 +8,3 @@ go mod tidy
go mod vendor go mod vendor
go generate -mod=vendor ./... go generate -mod=vendor ./...
go build -mod=vendor . go build -mod=vendor .

View File

@ -1,2 +1,22 @@
PORT="3000" PORT="3000"
#LISTEN=":3000" #LISTEN=":3000"
DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres
TEST_DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres_test
TRUST_PROXY=false
# Supports OIDC-compliant SSO issuers / providers
# (Auth0, Google, etc)
# (should provide .well-known/openid-configuration and .well-known/jwks.json)
OIDC_WHITELIST=https://mock.pocketid.app
# Public Key may be provided in addition to or in lieu of OIDC_WHITELIST
# can be RSA or ECDSA, either a filename or JWK/JSON (or PEM, but good luck escaping the newlines)
#
# go install -mod=vendor git.rootprojects.org/root/keypairs/cmd/keypairs
# keypairs gen -o priv.jwk.json --pub pub.jwk.json
#
#PUBLIC_KEY='{"crv":"P-256","kid":"kN4qj1w01Ry6ElG9I3qAVJOZFYLDklPFUdHaKozWtmc","kty":"EC","use":"sig","x":"SzzNgrOM_N0GwQWZPGFcdIKmfoQD6aXIzYm4gzGyPgQ","y":"erYeb884pk0BGMewDzEh_qYDB0aOFIxFjrXdqIzkmbw"}'
PUBLIC_KEY=./pub.jwk.json
#--demo is explicit

5
examples/genkeys.sh Executable file
View File

@ -0,0 +1,5 @@
# Install `keypairs`
go install -mod=vendor git.rootprojects.org/root/keypairs/cmd/keypairs
# Generate a keypair
keypairs gen -o key.jwk.json --pub pub.jwk.json

4
examples/setup.sh Executable file
View File

@ -0,0 +1,4 @@
#!/bin/bash
bash ./examples/build.sh
bash ./examples/genkeys.sh
bash ./examples/test.sh

45
examples/test.sh Executable file
View File

@ -0,0 +1,45 @@
#!/bin/bash
export BASE_URL=http://localhost:7070
#export BASE_URL=https://example.com
#CURL_OPTS="-sS"
CURL_OPTS=""
mkdir -p ./tmp/
# Sign an Admin token
echo '{ "sub": "admin_ppid", "email": "me@example.com", "iss": "'"${BASE_URL}"'" }' > ./tmp/admin.claims.json
keypairs sign --exp 1h ./key.jwk.json ./tmp/admin.claims.json > ./tmp/admin.jwt.txt 2> ./tmp/admin.jws.json
export ADMIN_TOKEN=$(cat ./tmp/admin.jwt.txt)
# verify the Admin token
#keypairs verify ./pub.jwk.json ./admin.jwt.txt
# Sign a User token
echo '{ "sub": "random_ppid", "email": "me@example.com", "iss": "'"${BASE_URL}"'" }' > ./tmp/user.claims.json
keypairs sign --exp 1h ./key.jwk.json ./tmp/user.claims.json > ./tmp/user.jwt.txt 2> ./tmp/user.jws.json
export USER_TOKEN=$(cat ./tmp/user.jwt.txt)
# verify the User token
#keypairs verify ./pub.jwk.json ./user.jwt.txt
EID=$(cat ./user.jws.json | grep sub | cut -d'"' -f 4)
echo ""
echo 'DELETE /api/public/reset (only works in --demo mode, deletes all data)'
curl $CURL_OPTS -X DELETE "${BASE_URL}/api/public/reset"
echo ""
echo ""
echo "Bootstrap with a new admin (only works once)"
curl -f $CURL_OPTS -X POST "${BASE_URL}/api/public/setup" \
-H "Authorization: Bearer ${ADMIN_TOKEN}"
echo ""
echo "Create a new user"
curl $CURL_OPTS -X POST "${BASE_URL}/api/users" \
-H "Authorization: Bearer ${USER_TOKEN}" \
-d '{ "display_name": "Jo Doe" }'
echo ""

7
go.mod
View File

@ -1,14 +1,17 @@
module git.coolaj86.com/coolaj86/goserv module git.example.com/example/goserv
go 1.15 go 1.15
require ( require (
git.rootprojects.org/root/go-gitver v1.1.3 // indirect
git.rootprojects.org/root/go-gitver/v2 v2.0.1
git.rootprojects.org/root/keypairs v0.6.3
github.com/go-chi/chi v4.1.2+incompatible github.com/go-chi/chi v4.1.2+incompatible
github.com/jmoiron/sqlx v1.2.0 github.com/jmoiron/sqlx v1.2.0
github.com/joho/godotenv v1.3.0 github.com/joho/godotenv v1.3.0
github.com/lib/pq v1.8.0 github.com/lib/pq v1.8.0
github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749 // indirect github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749 // indirect
github.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546 github.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546
golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346 // indirect golang.org/x/tools v0.0.0-20201001230009-b5b87423c93b
google.golang.org/appengine v1.6.6 // indirect google.golang.org/appengine v1.6.6 // indirect
) )

12
go.sum
View File

@ -1,3 +1,9 @@
git.rootprojects.org/root/go-gitver v1.1.3 h1:/qR9z53vY+IFhWRxLkF9cjaiWh8xRJIm6gyuW+MG81A=
git.rootprojects.org/root/go-gitver v1.1.3/go.mod h1:Rj1v3TBhvdaSphFEqMynUYwAz/4f+wY/+syBTvRrmlI=
git.rootprojects.org/root/go-gitver/v2 v2.0.1 h1:CdNfvlGDggFbyImxlqA2eFUVRKQKn1EJNk7w/3TQAfk=
git.rootprojects.org/root/go-gitver/v2 v2.0.1/go.mod h1:ur82M/jZcvr1WWihyVtNEgDBqIjo22o56wcVHeVJFh8=
git.rootprojects.org/root/keypairs v0.6.3 h1:dFuDjlg9KFXhRTIyVy3VfXzlX7hyyeC0J8PTUUZBzpo=
git.rootprojects.org/root/keypairs v0.6.3/go.mod h1:WGI8PadOp+4LjUuI+wNlSwcJwFtY8L9XuNjuO3213HA=
github.com/go-chi/chi v4.1.2+incompatible h1:fGFk2Gmi/YKXk0OmGfBh0WgmN3XB8lVnEyNz34tQRec= github.com/go-chi/chi v4.1.2+incompatible h1:fGFk2Gmi/YKXk0OmGfBh0WgmN3XB8lVnEyNz34tQRec=
github.com/go-chi/chi v4.1.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ= github.com/go-chi/chi v4.1.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ=
github.com/go-sql-driver/mysql v1.4.0 h1:7LxgVwFb2hIQtMm87NdgAVfXjnt4OePseqT1tKx+opk= github.com/go-sql-driver/mysql v1.4.0 h1:7LxgVwFb2hIQtMm87NdgAVfXjnt4OePseqT1tKx+opk=
@ -20,6 +26,7 @@ github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9dec
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/mod v0.3.0 h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
@ -37,10 +44,11 @@ golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346 h1:hzJjkvxUIF3bSt+v8N5tBQNx/605vszZJ+3XsIamzZo= golang.org/x/tools v0.0.0-20201001230009-b5b87423c93b h1:07IVqnnzaip3TGyl/cy32V5YP3FguWG4BybYDTBNpm0=
golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU= golang.org/x/tools v0.0.0-20201001230009-b5b87423c93b/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc= google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=

414
internal/api/api.go Normal file
View File

@ -0,0 +1,414 @@
package api
import (
"context"
"crypto/rand"
"database/sql"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"strconv"
"strings"
"time"
"git.rootprojects.org/root/keypairs"
"git.rootprojects.org/root/keypairs/keyfetch"
"github.com/go-chi/chi"
)
// TrustProxy will respect X-Forwarded-* headers
var TrustProxy bool
// OIDCWhitelist is a list of allowed issuers
var OIDCWhitelist string
var issuers keyfetch.Whitelist
// RandReader is a crypto/rand.Reader by default
var RandReader io.Reader = rand.Reader
var startedAt = time.Now()
var defaultMaxBytes int64 = 1 << 20
var apiIsReady bool
// Init will add the API routes to the given router
func Init(pub keypairs.PublicKey, r chi.Router) http.Handler {
// TODO more of this stuff should be options for the API
{
// block-scoped so we don't keep temp vars around
var err error
list := strings.Fields(strings.ReplaceAll(strings.TrimSpace(OIDCWhitelist), ",", " "))
issuers, err = keyfetch.NewWhitelist(list)
if nil != err {
log.Fatal("error parsing oidc whitelist:", err)
}
}
// OIDC Routes
if nil != pub {
fmt.Println("Public Key Thumbprint:", pub.Thumbprint())
fmt.Println("OIDC enabled at /.well-known/openid-configuration")
r.Get("/.well-known/openid-configuration", func(w http.ResponseWriter, r *http.Request) {
baseURL := getBaseURL(r)
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(fmt.Sprintf(
`{ "issuer": "%s", "jwks_uri": "%s/.well-known/jwks.json" }`+"\n",
baseURL, baseURL,
)))
})
fmt.Println("JWKs enabled at /.well-known/jwks.json")
r.Get("/.well-known/jwks.json", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
// non-standard: add expiry for when key should be fetched again
// TODO expiry should also go in the HTTP caching headers
exp := time.Now().Add(2 * time.Hour)
b := pubToOIDC(pub, exp)
// it's the little things
w.Write(append(b, '\n'))
})
}
r.Route("/api", func(r chi.Router) {
r.Use(limitResponseSize)
r.Use(jsonAllTheThings)
/*
n, err := countAdmins()
if nil != err {
log.Fatal("could not connect to database on boot:", err)
}
apiIsReady = n > 0
*/
// Unauthenticated routes
r.Route("/public", func(r chi.Router) {
r.Post("/setup", publicSetup)
r.Get("/ping", ping)
})
// Require admin-level permission
r.Route("/admin", func(r chi.Router) {
r.Use(errorUnlessReady())
r.Use(mustAdmin())
r.Get("/ping", ping)
})
// Any authenticated user
r.Route("/user", func(r chi.Router) {
r.Use(errorUnlessReady())
r.Use(canImpersonate())
r.Get("/ping", ping)
// TODO get ALL of the user's data
//r.Get("/", userComplete)
})
})
return r
}
// Reset sets the API back to its initial state
func Reset() {
apiIsReady = false
}
// utils
func getBaseURL(r *http.Request) string {
var scheme string
if nil != r.TLS ||
(TrustProxy && "https" == r.Header.Get("X-Forwarded-Proto")) {
scheme = "https:"
} else {
scheme = "http:"
}
return fmt.Sprintf(
"%s//%s",
scheme,
r.Host,
)
}
func mustAuthn(r *http.Request) (*http.Request, error) {
authzParts := strings.Split(r.Header.Get("Authorization"), " ")
if 2 != len(authzParts) {
return nil, fmt.Errorf("Bad Request: missing Auhorization header")
}
jwt := authzParts[1]
// TODO should probably add an error to keypairs
jws := keypairs.JWTToJWS(jwt)
if nil == jws {
return nil, fmt.Errorf("Bad Request: malformed Authorization header")
}
if err := jws.DecodeComponents(); nil != err {
return nil, fmt.Errorf("Bad Request: malformed JWT")
}
kid, _ := jws.Header["kid"].(string)
if "" == kid {
return nil, fmt.Errorf("Bad Request: missing 'kid' identifier")
}
iss, _ := jws.Claims["iss"].(string)
// TODO beware domain fronting, we should set domain statically
// See https://pkg.go.dev/git.rootprojects.org/root/keypairs@v0.6.2/keyfetch
// (Caddy does protect against Domain-Fronting by default:
// https://github.com/caddyserver/caddy/issues/2500)
if "" == iss || !issuers.IsTrustedIssuer(iss, r) {
return nil, fmt.Errorf("Bad Request: 'iss' is not a trusted issuer")
}
pub, err := keyfetch.OIDCJWK(kid, iss)
if nil != err {
return nil, fmt.Errorf("Bad Request: 'kid' could not be matched to a known public key")
}
errs := keypairs.VerifyClaims(pub, jws)
if nil != errs {
strs := []string{}
for _, err := range errs {
strs = append(strs, err.Error())
}
return nil, fmt.Errorf("invalid jwt:\n%s", strings.Join(strs, "\n\t"))
}
email, _ := jws.Claims["email"].(string)
ppid, _ := jws.Claims["sub"].(string)
if "" == email || "" == ppid {
return nil, fmt.Errorf("valid signed token, but missing claim for either 'email' or 'sub'")
}
ctx := context.WithValue(r.Context(), userPPID, ppid)
return r.WithContext(ctx), nil
}
func pubToOIDC(pub keypairs.PublicKey, exp time.Time) []byte {
exps := strconv.FormatInt(exp.Unix(), 10)
jsons := string(keypairs.MarshalJWKPublicKey(pub))
// this isn't as fragile as it looks, just adding some OIDC keys and such
// make prettier
jsons = strings.Replace(jsons, `{"`, `{ "`, 1)
jsons = strings.Replace(jsons, `",`, `" ,`, -1)
// nix trailing }
jsons = jsons[0 : len(jsons)-1]
// add on the OIDC stuff (exp is non-standard, but used by pocketid)
jsons = `{ "keys": [ ` +
jsons + fmt.Sprintf(`, "ext": true , "key_ops": ["verify"], "exp": %s }`, exps) +
" ] }"
return []byte(jsons)
}
// HTTPResponse gives a basic status message
// TODO sanitize all error messages and define error codes
type HTTPResponse struct {
Error string `json:"error,omitempty"`
Code string `json:"code,omitempty"`
Success bool `json:"success"`
Data interface{} `json:"result,omitempty"`
}
func ping(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
// as of yet, only known to be a user
ppid, _ := ctx.Value(userPPID).(string)
w.Write([]byte(fmt.Sprintf(
`{ "success": true, "uptime": %.0f, "ppid": %q }`+"\n",
time.Since(startedAt).Seconds(), ppid,
)))
}
func noImpl(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(
`{ "success": false, "error": "not implemented" }` + "\n",
))
}
func publicSetup(w http.ResponseWriter, r *http.Request) {
if apiIsReady {
// default is already 404, methinks
http.Error(w, "Not Found", http.StatusNotFound)
return
}
r, err := mustAuthn(r)
if nil != err {
userError(w, err)
return
}
ctx := r.Context()
ppid := ctx.Value(userPPID).(string)
if "" == ppid {
if nil != err {
userError(w, err)
return
}
}
if err := adminBootstrap(ppid); nil != err {
serverError("publicSetup.bootstrap", w, err)
return
}
apiIsReady = true
w.Write([]byte(fmt.Sprintf(
`{ "success": true, "sub": %q }`+"\n",
ppid,
)))
}
func reply(w http.ResponseWriter, msg interface{}) {
w.WriteHeader(http.StatusOK)
b, _ := json.MarshalIndent(&HTTPResponse{
Success: true,
Data: msg,
}, "", " ")
w.Write(append(b, '\n'))
}
func userError(w http.ResponseWriter, err error) {
w.WriteHeader(http.StatusBadRequest)
b, _ := json.Marshal(&HTTPResponse{
Error: err.Error(),
})
w.Write(append(b, '\n'))
}
func dbError(hint string, w http.ResponseWriter, err error) {
serverError(hint, w, err)
}
func serverError(hint string, w http.ResponseWriter, err error) {
// TODO check constraint errors and such, as those are likely user errors
if sql.ErrNoRows == err {
userError(w, fmt.Errorf("E_NOT_FOUND: %s", err))
return
} else if strings.Contains(err.Error(), "constraint") {
userError(w, fmt.Errorf("E_DUPLICATE_NAME: %s", err))
return
}
log.Printf("[%s] error: %v", hint, err)
w.WriteHeader(http.StatusInternalServerError)
b, _ := json.Marshal(&HTTPResponse{
Error: err.Error(),
})
w.Write(append(b, '\n'))
}
var errSetID = errors.New("you may not set the player's ID")
// Middleware
func errorUnlessReady() func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !apiIsReady {
http.Error(w, "Not Found", http.StatusNotFound)
return
}
next.ServeHTTP(w, r)
})
}
}
type adminCtx string
const (
adminPPID adminCtx = "ppid"
)
type userCtx string
const (
userPPID userCtx = "ppid"
)
func canImpersonate() func(next http.Handler) http.Handler {
return actAsAdmin(false)
}
func mustAdmin() func(next http.Handler) http.Handler {
return actAsAdmin(true)
}
func countAdmins() (int, error) {
return 0, errors.New("not implemented")
}
func adminBootstrap(ppid string) error {
return errors.New("not implemented")
}
func actAsAdmin(must bool) func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var err error
r, err = mustAuthn(r)
if nil != err {
userError(w, err)
return
}
ctx := r.Context()
// as of yet, only known to be a user
ppid, _ := ctx.Value(userPPID).(string)
//ok, err := isAdmin(ppid)
ok, err := false, errors.New("not implemented")
if nil != err {
serverError("actAsAdmin", w, err)
return
}
if !ok {
if must {
userError(w, errors.New("you're not an admin"))
return
}
next.ServeHTTP(w, r)
return
}
// we now know this is an admin, adjust accordingly
ctx = context.WithValue(r.Context(), adminPPID, ppid)
// also, an admin can impersonate
//uemail := r.URL.Query().Get("user_email")
uppid := r.URL.Query().Get("manager_id")
if "" == uppid {
uppid = ppid
}
ctx = context.WithValue(ctx, userPPID, uppid)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
}
func jsonAllTheThings(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// just setting a default, other handlers can change this
w.Header().Set("Content-Type", "application/json")
next.ServeHTTP(w, r)
})
}
func limitResponseSize(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, defaultMaxBytes)
next.ServeHTTP(w, r)
})
}

164
internal/api/api_test.go Normal file
View File

@ -0,0 +1,164 @@
package api
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"log"
mathrand "math/rand"
"net/http"
"net/http/httptest"
"net/url"
"os"
"strings"
"testing"
"git.example.com/example/goserv/internal/db"
"git.rootprojects.org/root/keypairs"
"git.rootprojects.org/root/keypairs/keyfetch"
"github.com/go-chi/chi"
)
var srv *httptest.Server
var testKey keypairs.PrivateKey
var testPub keypairs.PublicKey
var testWhitelist keyfetch.Whitelist
func init() {
// In tests it's nice to get the same "random" values, every time
RandReader = testReader{}
mathrand.Seed(0)
}
func TestMain(m *testing.M) {
connStr := needsTestDB(m)
if strings.Contains(connStr, "@localhost/") || strings.Contains(connStr, "@localhost:") {
connStr += "?sslmode=disable"
} else {
connStr += "?sslmode=required"
}
if err := db.Init(connStr); nil != err {
log.Fatal("db connection error", err)
return
}
if err := db.DropAllTables(db.PleaseDoubleCheckTheDatabaseURLDontDropProd(connStr)); nil != err {
log.Fatal(err)
}
if err := db.Init(connStr); nil != err {
log.Fatal("db connection error", err)
return
}
var err error
testKey = keypairs.NewDefaultPrivateKey()
testPub = keypairs.NewPublicKey(testKey.Public())
r := chi.NewRouter()
srv = httptest.NewServer(Init(testPub, r))
testWhitelist, err = keyfetch.NewWhitelist(nil, []string{srv.URL})
if nil != err {
log.Fatal("bad whitelist", err)
return
}
os.Exit(m.Run())
}
// public APIs
func Test_Public_Ping(t *testing.T) {
if err := testPing("public"); nil != err {
t.Fatal(err)
}
}
// test types
type testReader struct{}
func (testReader) Read(p []byte) (n int, err error) {
return mathrand.Read(p)
}
func testPing(which string) error {
urlstr := fmt.Sprintf("/api/%s/ping", which)
res, err := testReq("GET", urlstr, "", nil, 200)
if nil != err {
return err
}
data := map[string]interface{}{}
if err := json.NewDecoder(res.Body).Decode(&data); nil != err {
return err
}
if success, ok := data["success"].(bool); !ok || !success {
log.Printf("Bad Response\n\tURL:%s\n\tBody:\n%#v", urlstr, data)
return errors.New("bad response: missing success")
}
if ppid, _ := data["ppid"].(string); "" != ppid {
return fmt.Errorf("the effective user ID isn't what it should be: %q != %q", ppid, "")
}
return nil
}
func testReq(method, pathname string, jwt string, payload []byte, expectedStatus int) (*http.Response, error) {
client := srv.Client()
urlstr, _ := url.Parse(srv.URL + pathname)
if "" == method {
method = "GET"
}
req := &http.Request{
Method: method,
URL: urlstr,
Body: ioutil.NopCloser(bytes.NewReader(payload)),
Header: http.Header{},
}
if len(jwt) > 0 {
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", jwt))
}
res, err := client.Do(req)
if nil != err {
return nil, err
}
if expectedStatus > 0 {
if expectedStatus != res.StatusCode {
data, _ := ioutil.ReadAll(res.Body)
log.Printf("Bad Response: %d\n\tURL:%s\n\tBody:\n%s", res.StatusCode, urlstr, string(data))
return nil, fmt.Errorf("bad status code: %d", res.StatusCode)
}
}
return res, nil
}
func needsTestDB(m *testing.M) string {
connStr := os.Getenv("TEST_DATABASE_URL")
if "" == connStr {
log.Fatal(`no connection string defined
You must set TEST_DATABASE_URL to run db tests.
You may find this helpful:
psql 'postgres://postgres:postgres@localhost:5432/postgres'
DROP DATABASE IF EXISTS postgres_test;
CREATE DATABASE postgres_test;
\q
Then your test database URL will be
export TEST_DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres_test
`)
}
return connStr
}

37
internal/api/db.go Normal file
View File

@ -0,0 +1,37 @@
package api
import (
"encoding/base64"
"time"
"git.example.com/example/goserv/internal/db"
)
func newID() string {
// Postgres returns IDs on inserts but,
// for portability and ease of association,
// we'll create our own.
b := make([]byte, 16)
_, _ = RandReader.Read(b)
id := base64.RawURLEncoding.EncodeToString(b)
return id
}
// NotDeleted supplements a WHERE clause
const NotDeleted = `
( "deleted_at" IS NULL OR "deleted_at" = '0001-01-01 00:00:00+00' OR "deleted_at" = '1970-01-01 00:00:00+00' )
`
func logEvent(action, table, recordID, by string, at time.Time) (string, error) {
id := newID()
if _, err := db.DB.Exec(`
INSERT INTO "events" ("id", "action", "table", "record", "by", "at")
VALUES ($1, $2, $3, $4, $5, $6)`,
id, action, table, recordID, by, at,
); nil != err {
return "", err
}
return id, nil
}

View File

@ -3,10 +3,13 @@ package db
import ( import (
"context" "context"
"database/sql" "database/sql"
"fmt"
"io/ioutil" "io/ioutil"
"regexp"
"sort"
"time" "time"
"git.coolaj86.com/coolaj86/goserv/assets/configfs" "git.example.com/example/goserv/assets/configfs"
"github.com/jmoiron/sqlx" "github.com/jmoiron/sqlx"
// pq injects itself into sql as 'postgres' // pq injects itself into sql as 'postgres'
@ -15,21 +18,14 @@ import (
// DB is a concurrency-safe db connection instance // DB is a concurrency-safe db connection instance
var DB *sqlx.DB var DB *sqlx.DB
var firstDBURL PleaseDoubleCheckTheDatabaseURLDontDropProd
// Init returns a, you guessed it, New Store // Init initializes the database
func Init(pgURL string) error { func Init(pgURL string) error {
// https://godoc.org/github.com/lib/pq // https://godoc.org/github.com/lib/pq
f, err := configfs.Assets.Open("./postgres/init.sql") firstDBURL = PleaseDoubleCheckTheDatabaseURLDontDropProd(pgURL)
if nil != err {
return err
}
dbtype := "postgres" dbtype := "postgres"
sqlBytes, err := ioutil.ReadAll(f)
if nil != err {
return err
}
ctx, done := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second)) ctx, done := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second))
defer done() defer done()
@ -37,6 +33,29 @@ func Init(pgURL string) error {
if err := db.PingContext(ctx); nil != err { if err := db.PingContext(ctx); nil != err {
return err return err
} }
// basic stuff
f, err := configfs.Assets.Open("./postgres/init.sql")
if nil != err {
return err
}
sqlBytes, err := ioutil.ReadAll(f)
if nil != err {
return err
}
if _, err := db.ExecContext(ctx, string(sqlBytes)); nil != err {
return err
}
// project-specific stuff
f, err = configfs.Assets.Open("./postgres/tables.sql")
if nil != err {
return err
}
sqlBytes, err = ioutil.ReadAll(f)
if nil != err {
return err
}
if _, err := db.ExecContext(ctx, string(sqlBytes)); nil != err { if _, err := db.ExecContext(ctx, string(sqlBytes)); nil != err {
return err return err
} }
@ -45,3 +64,58 @@ func Init(pgURL string) error {
return nil return nil
} }
// PleaseDoubleCheckTheDatabaseURLDontDropProd is just a friendly,
// hopefully helpful reminder, not to only use this in test files,
// and to not drop the production database
type PleaseDoubleCheckTheDatabaseURLDontDropProd string
// DropAllTables runs drop.sql, which is intended only for tests
func DropAllTables(dbURL PleaseDoubleCheckTheDatabaseURLDontDropProd) error {
if err := CanDropAllTables(string(dbURL)); nil != err {
return err
}
// drop stuff
f, err := configfs.Assets.Open("./postgres/drop.sql")
if nil != err {
return err
}
sqlBytes, err := ioutil.ReadAll(f)
if nil != err {
return err
}
ctx, done := context.WithDeadline(context.Background(), time.Now().Add(1*time.Second))
defer done()
if _, err := DB.ExecContext(ctx, string(sqlBytes)); nil != err {
return err
}
return nil
}
// CanDropAllTables returns an error if the dbURL does not contain the words "test" or
// "demo" at a letter boundary
func CanDropAllTables(dbURL string) error {
var isDemo bool
nonalpha := regexp.MustCompile(`[^a-zA-Z]`)
haystack := nonalpha.Split(dbURL, -1)
sort.Strings(haystack)
for _, needle := range []string{"test", "demo"} {
// the index to insert x if x is not present (it could be len(a))
// (meaning that it is the index at which it exists, if it exists)
i := sort.SearchStrings(haystack, needle)
if i < len(haystack) && haystack[i] == needle {
isDemo = true
break
}
}
if isDemo {
return nil
}
return fmt.Errorf(
"test and demo database URLs must contain the word 'test' or 'demo' "+
"separated by a non-alphabet character, such as /test2/db_demo1\n%q\n",
dbURL,
)
}

66
internal/db/db_test.go Normal file
View File

@ -0,0 +1,66 @@
package db
import (
"errors"
"fmt"
"os"
"strings"
"testing"
)
func TestMain(m *testing.M) {
if err := testConnectAndInit(); nil != err {
fmt.Fprintf(os.Stderr, err.Error())
os.Exit(1)
return
}
os.Exit(m.Run())
}
func needsTestDB() (string, error) {
connStr := os.Getenv("TEST_DATABASE_URL")
if "" == connStr {
return "", errors.New(`no connection string defined
You must set TEST_DATABASE_URL to run db tests.
You may find this helpful:
psql 'postgres://postgres:postgres@localhost:5432/postgres'
DROP DATABASE IF EXISTS postgres_test;
CREATE DATABASE postgres_test;
\q
Then your test database URL will be
export TEST_DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres_test`)
}
return connStr, nil
}
func testConnectAndInit() error {
connStr, err := needsTestDB()
if nil != err {
return err
}
if strings.Contains(connStr, "@localhost/") || strings.Contains(connStr, "@localhost:") {
connStr += "?sslmode=disable"
} else {
connStr += "?sslmode=required"
}
if err := Init(connStr); nil != err {
return fmt.Errorf("db connection error: %w", err)
}
return nil
}
func TestDropAll(t *testing.T) {
connStr := os.Getenv("TEST_DATABASE_URL")
if err := DropAllTables(PleaseDoubleCheckTheDatabaseURLDontDropProd(connStr)); nil != err {
t.Fatal(err)
}
}

237
main.go
View File

@ -1,20 +1,27 @@
//go:generate go run git.rootprojects.org/root/go-gitver/v2
package main package main
import ( import (
"compress/flate" "compress/flate"
"flag" "flag"
"fmt" "fmt"
"io/ioutil"
"log" "log"
"math/rand"
"net/http" "net/http"
"os" "os"
"strings" "strings"
"time" "time"
"git.coolaj86.com/coolaj86/goserv/assets" "git.example.com/example/goserv/assets"
"git.coolaj86.com/coolaj86/goserv/internal/db" "git.example.com/example/goserv/internal/api"
"git.example.com/example/goserv/internal/db"
"git.rootprojects.org/root/keypairs"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"github.com/go-chi/chi/middleware" "github.com/go-chi/chi/middleware"
_ "github.com/joho/godotenv/autoload" _ "github.com/joho/godotenv/autoload"
) )
@ -45,6 +52,9 @@ type runOptions struct {
trustProxy bool trustProxy bool
compress bool compress bool
static string static string
pub string
oidcWL string
demo bool
} }
var runFlags *flag.FlagSet var runFlags *flag.FlagSet
@ -53,17 +63,32 @@ var initFlags *flag.FlagSet
var dbURL string var dbURL string
func init() { func init() {
// chosen by fair dice roll.
// guaranteed to be random.
rand.Seed(4)
// j/k
rand.Seed(time.Now().UnixNano())
initFlags = flag.NewFlagSet("init", flag.ExitOnError)
var conftodo bool
var confdomain string
initFlags.BoolVar(&conftodo, "todo", false, "TODO init should copy out nice templated config files")
initFlags.StringVar(&confdomain, "base-url", "https://example.com", "TODO the domain to use for templated scripts")
runOpts = runOptions{} runOpts = runOptions{}
runFlags = flag.NewFlagSet("run", flag.ExitOnError) runFlags = flag.NewFlagSet("run", flag.ExitOnError)
runFlags.StringVar( runFlags.StringVar(
&runOpts.listen, "listen", "", &runOpts.listen, "listen", "",
"the address and port on which to listen (default \""+defaultAddr+"\")") "the address and port on which to listen (default \""+defaultAddr+"\")")
runFlags.BoolVar(&runOpts.trustProxy, "trust-proxy", false, "trust X-Forwarded-For header") runFlags.BoolVar(&runOpts.trustProxy, "trust-proxy", false, "trust X-Forwarded-* headers")
runFlags.BoolVar(&runOpts.compress, "compress", true, "enable compression for text,html,js,css,etc") runFlags.BoolVar(&runOpts.compress, "compress", true, "enable compression for text,html,js,css,etc")
runFlags.StringVar(&runOpts.static, "serve-path", "", "path to serve, falls back to built-in web app") runFlags.StringVar(&runOpts.static, "serve-path", "", "path to serve, falls back to built-in web app")
runFlags.StringVar( runFlags.StringVar(&runOpts.pub, "public-key", "", "path to public key, or key string - RSA or ECDSA, JWK (JSON) or PEM")
&dbURL, "db-url", "postgres://postgres:postgres@localhost:5432/postgres", runFlags.StringVar(&runOpts.oidcWL, "oidc-whitelist", "", "list of trusted OIDC issuer URLs (ex: Auth0, Google, PocketID) for SSO")
"database (postgres) connection url") runFlags.BoolVar(&runOpts.demo, "demo", false, "demo mode enables unauthenticated 'DELETE /api/public/reset' to reset")
runFlags.StringVar(&dbURL, "database-url", "",
"database (postgres) connection url (default postgres://postgres:postgres@localhost:5432/postgres)",
)
} }
func main() { func main() {
@ -106,6 +131,19 @@ func main() {
runOpts.listen = defaultAddr runOpts.listen = defaultAddr
} }
} }
if "" == dbURL {
dbURL = os.Getenv("DATABASE_URL")
}
if "" == dbURL {
dbURL = "postgres://postgres:postgres@localhost:5432/postgres"
}
api.OIDCWhitelist = runOpts.oidcWL
if "" == api.OIDCWhitelist {
api.OIDCWhitelist = os.Getenv("OIDC_WHITELIST")
}
if "" == runOpts.pub {
runOpts.pub = os.Getenv("PUBLIC_KEY")
}
serve() serve()
default: default:
usage() usage()
@ -114,100 +152,6 @@ func main() {
} }
} }
var startedAt = time.Now()
var defaultMaxBytes int64 = 1 << 20
func serve() {
initDB(dbURL)
r := chi.NewRouter()
// A good base middleware stack
if runOpts.trustProxy {
r.Use(middleware.RealIP)
}
if runOpts.compress {
r.Use(middleware.Compress(flate.DefaultCompression))
}
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
r.Route("/api", func(r chi.Router) {
r.Use(limitResponseSize)
r.Use(jsonAllTheThings)
r.Route("/public", func(r chi.Router) {
r.Get("/status", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(fmt.Sprintf(
`{ "success": true, "uptime": %.0f }%s`,
time.Since(startedAt).Seconds(),
"\n",
)))
})
})
r.Route("/user", func(r chi.Router) {
r.Get("/inspect", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(fmt.Sprintf(
`{ "success": false, "error": "not implemented" }%s`, "\n",
)))
})
})
})
var staticHandler http.HandlerFunc
pub := http.FileServer(assets.Assets)
if len(runOpts.static) > 0 {
// try the user-provided directory first, then fallback to the built-in
devFS := http.Dir(runOpts.static)
dev := http.FileServer(devFS)
staticHandler = func(w http.ResponseWriter, r *http.Request) {
if _, err := devFS.Open(r.URL.Path); nil != err {
pub.ServeHTTP(w, r)
return
}
dev.ServeHTTP(w, r)
}
} else {
staticHandler = func(w http.ResponseWriter, r *http.Request) {
pub.ServeHTTP(w, r)
}
}
r.Get("/*", staticHandler)
fmt.Println("Listening for http (with reasonable timeouts) on", runOpts.listen)
srv := &http.Server{
Addr: runOpts.listen,
Handler: r,
ReadHeaderTimeout: 2 * time.Second,
ReadTimeout: 10 * time.Second,
WriteTimeout: 20 * time.Second,
MaxHeaderBytes: 1024 * 1024, // 1MiB
}
if err := srv.ListenAndServe(); nil != err {
fmt.Fprintf(os.Stderr, "%s", err)
os.Exit(1)
return
}
}
func jsonAllTheThings(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// just setting a default, other handlers can change this
w.Header().Set("Content-Type", "application/json")
next.ServeHTTP(w, r)
})
}
func limitResponseSize(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, defaultMaxBytes)
next.ServeHTTP(w, r)
})
}
func initDB(connStr string) { func initDB(connStr string) {
// TODO url.Parse // TODO url.Parse
if strings.Contains(connStr, "@localhost/") || strings.Contains(connStr, "@localhost:") { if strings.Contains(connStr, "@localhost/") || strings.Contains(connStr, "@localhost:") {
@ -225,3 +169,96 @@ func initDB(connStr string) {
return return
} }
func serve() {
initDB(dbURL)
r := chi.NewRouter()
// A good base middleware stack
if runOpts.trustProxy {
api.TrustProxy = true
r.Use(middleware.RealIP)
}
if runOpts.compress {
r.Use(middleware.Compress(flate.DefaultCompression))
}
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
var pub keypairs.PublicKey = nil
if "" != runOpts.pub {
var err error
pub, err = keypairs.ParsePublicKey([]byte(runOpts.pub))
if nil != err {
b, err := ioutil.ReadFile(runOpts.pub)
if nil != err {
// ignore
} else {
pub, err = keypairs.ParsePublicKey(b)
if nil != err {
log.Fatal("could not parse public key:", err)
}
}
}
}
_ = api.Init(pub, r)
var staticHandler http.HandlerFunc
pubdir := http.FileServer(assets.Assets)
if len(runOpts.static) > 0 {
// try the user-provided directory first, then fallback to the built-in
devFS := http.Dir(runOpts.static)
dev := http.FileServer(devFS)
staticHandler = func(w http.ResponseWriter, r *http.Request) {
if _, err := devFS.Open(r.URL.Path); nil != err {
pubdir.ServeHTTP(w, r)
return
}
dev.ServeHTTP(w, r)
}
} else {
staticHandler = func(w http.ResponseWriter, r *http.Request) {
pubdir.ServeHTTP(w, r)
}
}
if runOpts.demo {
if err := db.CanDropAllTables(dbURL); nil != err {
fmt.Fprintf(os.Stderr, err.Error())
os.Exit(1)
}
fmt.Println("DANGER: running in demo mode with DELETE /api/public/reset enabled")
fmt.Fprintf(os.Stderr, "DANGER: running in demo mode with DELETE /api/public/reset enabled\n")
r.Delete("/api/public/reset", func(w http.ResponseWriter, r *http.Request) {
if err := db.DropAllTables(db.PleaseDoubleCheckTheDatabaseURLDontDropProd(dbURL)); nil != err {
w.Write([]byte("error dropping tabels: " + err.Error()))
return
}
api.Reset()
initDB(dbURL)
w.Write([]byte("re-initialized\n"))
})
}
r.Get("/*", staticHandler)
fmt.Println("")
fmt.Println("Listening for http (with reasonable timeouts) on", runOpts.listen)
srv := &http.Server{
Addr: runOpts.listen,
Handler: r,
ReadHeaderTimeout: 2 * time.Second,
ReadTimeout: 10 * time.Second,
WriteTimeout: 20 * time.Second,
MaxHeaderBytes: 1024 * 1024, // 1MiB
}
if err := srv.ListenAndServe(); nil != err {
fmt.Fprintf(os.Stderr, "%s\n", err)
time.Sleep(100 * time.Millisecond)
os.Exit(1)
return
}
}

View File

@ -1,82 +1,76 @@
<!DOCTYPE htmtl> <!DOCTYPE htmtl>
<html> <html>
<head> <head>
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<title>Example</title> <title>Example</title>
<link <link
rel="stylesheet" rel="stylesheet"
href="./vendor/css/bootswatch.com/4/materia/bootstrap.min.css" href="./vendor/css/bootswatch.com/4/materia/bootstrap.min.css"
/> />
</head> </head>
<body> <body>
<nav class="navbar navbar-expand-lg navbar-dark bg-primary"> <nav class="navbar navbar-expand-lg navbar-dark bg-primary">
<a class="navbar-brand" href="#">Example</a> <a class="navbar-brand" href="#">Example</a>
<button <button
class="navbar-toggler" class="navbar-toggler"
type="button" type="button"
data-toggle="collapse" data-toggle="collapse"
data-target="#navbarColor01" data-target="#navbarColor01"
aria-controls="navbarColor01" aria-controls="navbarColor01"
aria-expanded="false" aria-expanded="false"
aria-label="Toggle navigation" aria-label="Toggle navigation"
> >
<span class="navbar-toggler-icon"></span> <span class="navbar-toggler-icon"></span>
</button> </button>
<div class="collapse navbar-collapse" id="navbarColor01"> <div class="collapse navbar-collapse" id="navbarColor01">
<ul class="navbar-nav mr-auto"> <ul class="navbar-nav mr-auto">
<li class="nav-item active"> <li class="nav-item active">
<a class="nav-link" href="#nav-foobar">Foobar</a> <a class="nav-link" href="#nav-foobar">Foobar</a>
</li> </li>
</ul> </ul>
<form class="js-signin form-inline my-2 my-lg-0"> <form class="js-signin form-inline my-2 my-lg-0">
<input <input
class="form-control mr-sm-2" class="form-control mr-sm-2"
type="email" type="email"
name="email" name="email"
placeholder="email" placeholder="email"
/> />
<button <button class="btn btn-secondary my-2 my-sm-0" type="submit">
class="btn btn-secondary my-2 my-sm-0" Sign in
type="submit" </button>
> </form>
Sign in </div>
</button> </nav>
<div class="container">
<div class="row">
<div class="pocket"></div>
</div>
<div class="row">
<div class="col-xs-12">
<div class="card border-primary mb-6">
<a id="nav-foobar"
><h3 class="card-header">
Server Health
<form class="js-healthcheck">
<button type="submit" class="float-right btn btn-primary">
Check
</button>
</form> </form>
</h3></a
>
<div class="js-pre card-body">
<h5 class="card-title">Check Server Status</h5>
<div class="card-text">
<pre><code>curl https://example.com/api/public/ping</code></pre>
<pre><code class="js-server-health">-</code></pre>
</div>
</div> </div>
</nav> </div>
<div class="container">
<div class="row">
<div class="pocket"></div>
</div>
<div class="row">
<div class="col-xs-12">
<div class="card border-primary mb-6">
<a id="nav-foobar"
><h3 class="card-header">
Server Health
<form class="js-healthcheck">
<button
type="submit"
class="float-right btn btn-primary"
>
Check
</button>
</form>
</h3></a
>
<div class="card-body">
<h5 class="card-title">Check Server Status</h5>
<div class="card-text">
<pre><code>curl https://example.com/api/public/status</code></pre>
<pre><code class="js-server-health">-</code></pre>
</div>
</div>
</div>
</div>
</div>
</div> </div>
<script src="https://mock.pocketid.app/pocket/consumer.js"></script> </div>
<script src="./js/app.js"></script> </div>
</body> <script src="https://mock.pocketid.app/pocket/consumer.js"></script>
<script src="./js/app.js"></script>
</body>
</html> </html>

View File

@ -1,49 +1,111 @@
(function () { (function () {
"use strict"; "use strict";
// AJQuery // AJQuery
function $(sel, el) { function $(sel, el) {
if (!el) { if (!el) {
el = document; el = document;
}
return el.querySelector(sel);
} }
function $$(sel, el) { return el.querySelector(sel);
if (!el) { }
el = document; function $$(sel, el) {
} if (!el) {
return el.querySelectorAll(sel); el = document;
} }
return el.querySelectorAll(sel);
}
function displayToken(token) { function displayToken(token) {
$$(".js-token").forEach(function (el) { $$(".js-token").forEach(function (el) {
el.innerText = token; el.innerText = token;
});
}
Pocket.onToken(function (token) {
// TODO Pocket v1.0 will make this obsolete
localStorage.setItem("pocket-token", token);
displayToken();
}); });
displayToken(localStorage.getItem("pocket-token")); }
// requires div with class 'pocket' Pocket.onToken(function (token) {
$("form.js-signin").addEventListener("submit", function (ev) { // TODO Pocket v1.0 will make this obsolete
ev.preventDefault(); localStorage.setItem("pocket-token", token);
ev.stopPropagation(); displayToken();
});
displayToken(localStorage.getItem("pocket-token"));
var email = $("[name=email]").value; // requires div with class 'pocket'
Pocket.openSignin(ev, { email: email }); $("form.js-signin").addEventListener("submit", function (ev) {
ev.preventDefault();
ev.stopPropagation();
var email = $("[name=email]").value;
Pocket.openSignin(ev, { email: email });
});
$("form.js-healthcheck").addEventListener("submit", function (ev) {
ev.preventDefault();
ev.stopPropagation();
window.fetch("/api/public/ping").then(async function (resp) {
var res = await resp.json();
$(".js-server-health").innerText = JSON.stringify(res, null, 2);
}); });
});
`
# Demo Mode Only
DELETE /public/reset Drop database and re-initialize
$("form.js-healthcheck").addEventListener("submit", function (ev) { # Public
ev.preventDefault(); GET /public/ping Health Check
ev.stopPropagation(); POST /public/setup <= (none) Bootstrap
window.fetch("/api/public/status").then(async function (resp) { # Admin-only
var res = await resp.json(); GET /admin/ping (authenticated) Health Check
$(".js-server-health").innerText = JSON.stringify(res, null, 2); GET /admin/users []Users => List ALL Users
});
# User
GET /user/ping (authenticated) Health Check
GET /user => User User profile
`
.trim()
.split(/\n/)
.forEach(function (line) {
line = line.trim();
if ("#" === line[0] || !line.trim()) {
return;
}
line = line.replace(/(<=)?\s*\(none\)\s*(=>)?/g, "");
var parts = line.split(/\s+/g);
var method = parts[0];
if ("GET" != method) {
method = "-X " + method + " ";
} else {
method = "";
}
var pathname = parts[1];
var auth = pathname.match(/(public|user|admin)/)[1];
if ("admin" == auth) {
auth = " \\\n -H 'Authorization: Bearer ADMIN_TOKEN'";
} else if ("user" == auth) {
auth = " \\\n -H 'Authorization: Bearer USER_TOKEN'";
} else {
auth = "";
}
document.body.querySelector(".js-pre").innerHTML += (
`
<div class="card-text">
<pre><code>curl -X POST https://example.com/api</code></pre>
<pre><code class="js-` +
pathname.replace(/\//g, "-") +
`">-</code></pre>
</div>
`
)
.replace(
/https:\/\/example\.com/g,
location.protocol + "//" + location.host
)
.replace(/-X POST /g, method)
.replace(/\/api/g, "/api" + pathname + auth);
}); });
/*
document.body.querySelector(".js-pre").innerHTML = document.body
.querySelector(".js-pre")
.innerHTML
*/
})(); })();

View File

@ -7,4 +7,7 @@ import (
// these are 'go generate' tooling dependencies, not including in the binary // these are 'go generate' tooling dependencies, not including in the binary
_ "github.com/shurcooL/vfsgen" _ "github.com/shurcooL/vfsgen"
_ "github.com/shurcooL/vfsgen/cmd/vfsgendev" _ "github.com/shurcooL/vfsgen/cmd/vfsgendev"
_ "golang.org/x/tools/cmd/stringer"
_ "git.rootprojects.org/root/keypairs/cmd/keypairs"
_ "git.rootprojects.org/root/go-gitver/v2"
) )

31
vendor/modules.txt vendored
View File

@ -1,3 +1,15 @@
# git.rootprojects.org/root/go-gitver v1.1.3
## explicit
git.rootprojects.org/root/go-gitver/gitver
# git.rootprojects.org/root/go-gitver/v2 v2.0.1
## explicit
git.rootprojects.org/root/go-gitver/v2
# git.rootprojects.org/root/keypairs v0.6.3
## explicit
git.rootprojects.org/root/keypairs
git.rootprojects.org/root/keypairs/cmd/keypairs
git.rootprojects.org/root/keypairs/keyfetch
git.rootprojects.org/root/keypairs/keyfetch/uncached
# github.com/go-chi/chi v4.1.2+incompatible # github.com/go-chi/chi v4.1.2+incompatible
## explicit ## explicit
github.com/go-chi/chi github.com/go-chi/chi
@ -22,7 +34,24 @@ github.com/shurcooL/httpfs/vfsutil
## explicit ## explicit
github.com/shurcooL/vfsgen github.com/shurcooL/vfsgen
github.com/shurcooL/vfsgen/cmd/vfsgendev github.com/shurcooL/vfsgen/cmd/vfsgendev
# golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346 # golang.org/x/mod v0.3.0
golang.org/x/mod/semver
# golang.org/x/tools v0.0.0-20201001230009-b5b87423c93b
## explicit ## explicit
golang.org/x/tools/cmd/stringer
golang.org/x/tools/go/gcexportdata
golang.org/x/tools/go/internal/gcimporter
golang.org/x/tools/go/internal/packagesdriver
golang.org/x/tools/go/packages
golang.org/x/tools/internal/event
golang.org/x/tools/internal/event/core
golang.org/x/tools/internal/event/keys
golang.org/x/tools/internal/event/label
golang.org/x/tools/internal/gocommand
golang.org/x/tools/internal/packagesinternal
golang.org/x/tools/internal/typesinternal
# golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
golang.org/x/xerrors
golang.org/x/xerrors/internal
# google.golang.org/appengine v1.6.6 # google.golang.org/appengine v1.6.6
## explicit ## explicit