From 7ee247afe6bd5ae45cead0ad7d8c626a6ebc1858 Mon Sep 17 00:00:00 2001 From: AJ ONeal Date: Tue, 25 Jul 2017 18:37:29 -0600 Subject: [PATCH] update README with config info --- README.md | 467 ++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 352 insertions(+), 115 deletions(-) diff --git a/README.md b/README.md index 7fedd61..2ad41b3 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,25 @@ - - -About Daplie: We're taking back the Internet! --------------- - -Down with Google, Apple, and Facebook! - -We're re-decentralizing the web and making it read-write again - one home cloud system at a time. - -Tired of serving the Empire? Come join the Rebel Alliance: - -jobs@daplie.com | [Invest in Daplie on Wefunder](https://daplie.com/invest/) | [Pre-order Cloud](https://daplie.com/preorder/), The World's First Home Server for Everyone - - - Goldilocks ========== -The node.js webserver that's just right. +The node.js netserver that's just right. +* **HTTPS Web Server** with Automatic TLS (SSL) via ACME ([Let's Encrypt](https://letsencrypt.org)) + * Static Web Server + * URL Redirects + * SSL on localhost (with bundled localhost.daplie.me certificates) +* **TLS** name-based (SNI) proxy +* **TCP** port-based proxy +* WS **Tunnel Server** (i.e. run on Digital Ocean and expose a home-firewalled Raspberry Pi to the Internet) +* WS **Tunnel Client** (i.e. run on a Raspberry Pi and connect to a Daplie Tunnel) +* Configurable via API +* mDNS Discoverable (configure in home or office with mobile and desktop apps) +* OAuth3 Authentication -A simple HTTPS static file server with valid TLS (SSL) certs. - -Comes bundled a valid certificate for localhost.daplie.me, -which is great for testing and development, and you can specify your own. - -Also great for testing ACME certs from letsencrypt.org. - -Install +Install Standalone ------- ```bash -# v2 in npm +# v1 in npm npm install -g goldilocks # master in git (via ssh) @@ -48,123 +37,371 @@ goldilocks Serving /Users/foo/ at https://localhost.daplie.me:8443 ``` -With service support for +Install as a System Service (daemon-mode) -* systemd -* launchd +We have service support for + +* systemd (Linux, Ubuntu) +* launchd (macOS) ```bash curl https://git.daplie.com/Daplie/goldilocks.js/raw/master/install.sh | bash ``` -Usage +Modules & Configuration ----- -Examples: +Goldilocks has several core systems, which all have their own configuration and some of which have modules: ``` -# Install -npm install -g git+https://git@git.daplie.com:Daplie/goldilocks.js - -# Use tunnel -goldilocks --sites jane.daplie.me --agree-tos --email jane@example.com --tunnel - -# BEFORE you access in a browser for the first time, use curl -# (because there's a concurrency bug in the greenlock setup) -curl https://jane.daplie.me +* http + - static + - redirect + - proxy (reverse proxy) +* tls + - acme + - proxy (reverse proxy) +* tcp + - forward +* tunnel_server +* tunnel_client +* mdns +* api ``` -Options: +### http -* `-p ` - i.e. `sudo goldilocks -p 443` (defaults to 80+443 or 8443) -* `-d ` - i.e. `goldilocks -d /tmp/` (defaults to `pwd`) - * you can use `:hostname` as a template for multiple directories - * Example A: `goldilocks -d /srv/www/:hostname --sites localhost.foo.daplie.me,localhost.bar.daplie.me` - * Example B: `goldilocks -d ./:hostname/public/ --sites localhost.foo.daplie.me,localhost.bar.daplie.me` -* `-c ` - i.e. `server-https -c 'Hello, World! '` (defaults to directory index) -* `--express-app ` - path to a file the exports an express-style app (`function (req, res, next) { ... }`) -* `--livereload` - inject livereload into all html pages (see also: [fswatch](http://stackoverflow.com/a/13807906/151312)), but be careful if `` has thousands of files it will spike your CPU usage to 100% - -* `--email ` - email to use for Let's Encrypt, Daplie DNS, Daplie Tunnel -* `--agree-tos` - agree to terms for Let's Encrypt, Daplie DNS -* `--sites ` comma-separated list of domains to respond to (default is `localhost.daplie.me`) - * optionally you may include the path to serve with `|` such as `example.com|/tmp,example.net/srv/www` -* `--tunnel` - make world-visible (must use `--sites`) - -Specifying a custom HTTPS certificate: - -* `--key /path/to/privkey.pem` specifies the server private key -* `--cert /path/to/fullchain.pem` specifies the bundle of server certificate and all intermediate certificates -* `--root /path/to/root.pem` specifies the certificate authority(ies) - -Note: `--root` may specify single cert or a bundle, and may be used multiple times like so: +The HTTP system handles plain http (TLS / SSL is handled by the tls system) ``` ---root /path/to/primary-root.pem --root /path/to/cross-root.pem +http: + trust_proxy: true # allow localhost, 192.x, 10.x, 172.x, etc to set headers + allow_insecure: false # allow non-https even without proxy https headers + primary_domain: example.com # attempts to access via IP address will redirect here + + # modules can be nested in domains + domains: + - names: + - example.com + modules: + - name: static + root: /srv/www/:hostname + + # The configuration above could also be represented as follows: + modules: + - name: static + domains: + - example.com + root: /srv/www/:hostname ``` -Other options: +### http.static - how to serve a web page -* `--serve-root true` alias for `-c` with the contents of root.pem -* `--sites example.com` changes the servername logged to the console -* `--letsencrypt-certs example.com` sets and key, fullchain, and root to standard letsencrypt locations +The static module is for serving static web pages and assets and has the following options: -Examples --------- - -```bash -goldilocks -p 1443 -c 'Hello from 1443' & -goldilocks -p 2443 -c 'Hello from 2443' & -goldilocks -p 3443 -d /tmp & - -curl https://localhost.daplie.me:1443 -> Hello from 1443 - -curl --insecure https://localhost:2443 -> Hello from 2443 - -curl https://localhost.daplie.me:3443 -> [html index listing of /tmp] +``` +root The path to serve as a string. + The template variable `:hostname` represents the HTTP Host header without port information + ex: `root: /srv/www/example.com` would load the example.com folder for any domain listed + ex: `root: /srv/www/:hostname` would load `/srv/www/example.com` if so indicated by the Host header ``` -And if you tested in a browser, -it would redirect to (on the same port). +Example config: -(in curl it would just show an error message) - -### Testing ACME Let's Encrypt certs - -In case you didn't know, you can get free https certificates from -[letsencrypt.org](https://letsencrypt.org) -(ACME letsencrypt) -and even a free subdomain from . - -If you want to quickly test the certificates you installed, -you can do so like this: - -```bash -goldilocks -p 8443 \ - --letsencrypt-certs test.mooo.com \ - --serve-root true +``` +http: + modules: + - name: static + domains: + - example.com + root: /srv/www/:hostname ``` -which is equilavent to +### http.proxy - how to reverse proxy (ruby, python, etc) -```bash -goldilocks -p 8443 \ - --sites test.mooo.com - --key /etc/letsencrypt/live/test.mooo.com/privkey.pem \ - --cert /etc/letsencrypt/live/test.mooo.com/fullchain.pem \ - --root /etc/letsencrypt/live/test.mooo.com/root.pem \ - -c "$(cat 'sudo /etc/letsencrypt/live/test.mooo.com/root.pem')" +The proxy module is for reverse proxying, typically to an application on the same machine. + +It has the following options: + +``` +host The DNS-resolvable hostname (or IP address) of the system to which the request will be proxied + ex: localhost + ex: 192.168.1.100 + +port The port on said system to which the request will be proxied + ex: 3000 + ex: 80 ``` -and can be tested like so +Example config: -```bash -curl --insecure https://test.mooo.com:8443 > ./root.pem -curl https://test.mooo.com:8843 --cacert ./root.pem +``` +http: + modules: + - name: proxy + domains: + - example.com + host: localhost + port: 3000 ``` -* [QuickStart Guide for Let's Encrypt](https://coolaj86.com/articles/lets-encrypt-on-raspberry-pi/) -* [QuickStart Guide for FreeDNS](https://coolaj86.com/articles/free-dns-hosting-with-freedns-afraid-org.html) +### http.redirect - how to redirect URLs + +The redirect module is for, you guessed it, redirecting URLs. + +It has the following options: + +``` +status The HTTP status code to issue (301 is usual permanent redirect, 302 is temporary) + ex: 301 + +from The URL path that was used in the request. + The `*` wildcard character can be used for matching a full segment of the path + ex: /photos/ + ex: /photos/*/*/ + +to The new URL path which should be used. + If wildcards matches were used they will be available as `:1`, `:2`, etc. + ex: /pics/ + ex: /pics/:1/:2/ +``` + +Example config: + +``` +http: + modules: + - name: proxy + domains: + - example.com + status: 301 + from: /archives/*/*/*/ + to: https://example.net/year/:1/month/:2/day/:3/ +``` + +### tls + +The tls system handles encrypted connections, including fetching certificates, +and uses ServerName Indication (SNI) to determine if the connection should be handled +by the http system, a tls system module, or rejected. + +It has the following options: + +``` +acme.email The default email address for ACME certificate issuance + ex: john.doe@example.com + +acme.server The default ACME server to use + ex: https://acme-v01.api.letsencrypt.org/directory + ex: https://acme-staging.api.letsencrypt.org/directory + +acme.challenge_type The default ACME challenge to request + ex: http-01, dns-01, tls-01 + +acme.approved_domains The domains for which to request certificates + ex: example.com +``` + +Example config: + +```yml +tls: + acme: + email: 'joe.shmoe@example.com' + # IMPORTANT: Switch to in production 'https://acme-v01.api.letsencrypt.org/directory' + server: 'https://acme-staging.api.letsencrypt.org/directory' + challenge_type: 'http-01' + approved_domains: + - example.com + - example.net + + modules: + - name: proxy + domains: + - example.com + - example.net + address: '127.0.0.1:6443' +``` + +Certificates are saved to `~/acme`, which may be `/var/www/acme` if Goldilocks is run as the www-data user. + +### tls.acme + +The acme module overrides the acme defaults of the tls system and uses the same options except that `approved_domains` +(in favor of the domains in the scope of the module). + +Example config: + +```yml +tls: + modules: + - name: acme + domains: + - example.com + - example.net + email: 'joe.shmoe@example.com' + server: 'https://acme-staging.api.letsencrypt.org/directory' + challenge_type: 'http-01' +``` + +### tls.proxy + +The proxy module routes the traffic based on the ServerName Indication (SNI) **without decrypting** it. + +It has the following options: + +``` +address The hostname (or IP) and port of the system or application that should receive the traffic +``` + +Example config: + +```yml +tls: + modules: + - name: forward + domains: + - example.com + address: '127.0.0.1:5443' +``` + +### tcp + +The tcp system handles all tcp network traffic **before decryption** and may use port numbers +or traffic sniffing to determine how the connection should be handled. + +It has the following options: + +```yml +bind An array of numeric ports on which to bind + ex: 80 +``` + +Example Config + +``` +tcp: + bind: + - 22 + - 80 + - 443 + modules: + - name: forward + ports: + - 22 + address: '127.0.0.1:2222' +``` + +### tcp.forward + +The forward module routes traffic based on port number **without decrypting** it. + +It has the following options: + +``` +ports A numeric array of source ports + ex: 22 + +address The destination hostname and port + ex: 127.0.0.1:2222 +``` + +Example Config + +``` +tcp: + bind: + - 22 + - 80 + - 443 + modules: + - name: forward + ports: + - 22 + address: '127.0.0.1:2222' +``` + + + +### tunnel\_server + +The tunnel server system is meant to be run on a publicly accessible IP address to server tunnel clients +which are behind firewalls, carrier-grade NAT, or otherwise Internet-connect but inaccessible devices. + +It has the following options: + +``` +secret A 128-bit or greater string to use for signing tokens (HMAC JWT) + ex: abc123 + +servernames An array of string servernames that should be captured as the tunnel server, ignoring the TLS forward module + ex: api.tunnel.example.com +``` + +Example config: + +``` +tunnel_server: + secret: abc123def456ghi789 + servernames: + - 'api.tunnel.example.com' +``` + +### tunnel\_client + +TODO + +### ddns + +TODO + +### mdns + +enabled by default + +Although it does not announce itself, Goldilocks is discoverable via mDNS with the special query `_cloud._tcp.local`. +This is so that it can be easily configured via Desktop and Mobile apps when run on devices such as a Raspberry Pi or +SOHO servers. + +``` +mdns: + disabled: false + port: 5353 + broadcast: '224.0.0.251' + ttl: 300 +``` + +You can discover goldilocks with `mdig`. + +``` +npm install -g git+https://git.daplie.com/Daplie/mdig.git + +mdig _cloud._tcp.local +``` + +### api + +The API system is intended for use with Desktop and Mobile clients. +It must be accessed using one of the following domains as the Host header: + +``` +admin.invalid +localhost.admin.daplie.me +``` + +@tigerbot: How are the APIs used (in terms of URL, Method, Headers, etc)? + +TODO +---- + +* http - nowww module +* http - Allow match styles of `www.*`, `*`, and `*.example.com` equally +* http - redirect based on domain name (not just path) +* tcp - bind should be able to specify localhost, uniquelocal, private, or ip +* tcp - if destination host is omitted default to localhost, if dst port is missing, default to src +* sys - handle SIGHUP +* sys - `curl https://daplie.me/goldilocks | bash -s example.com` +* oauth3 - `example.com/.well-known/domains@oauth3.org/directives.json` +* oauth3 - commandline questionnaire +* modules - use consistent conventions (i.e. address vs host + port) + * tls - tls.acme vs tls.modules.acme