Compare commits

..

No commits in common. "master" and "api-rewrite" have entirely different histories.

51 changed files with 3595 additions and 2463 deletions

View File

@ -13,5 +13,4 @@
, "latedef": true
, "curly": true
, "trailing": true
, "esversion": 6
}

60
API.md
View File

@ -10,48 +10,6 @@ It must be accessed using one of the following domains as the Host header:
All requests require an OAuth3 token in the request headers.
## Tokens
Some of the functionality of goldilocks requires the use of OAuth3 tokens to
perform tasks like setting DNS records. Management of these tokens can be done
using the following APIs.
### Get A Single Token
* **URL** `/api/goldilocks@daplie.com/tokens/:id`
* **Method** `GET`
* **Reponse**: The token matching the specified ID. Has the following properties.
* `id`: The hash used to identify the token. Based on several of the fields
inside the decoded token.
* `provider_uri`: The URI for the one who issued the token. Should be the same
as the `iss` field inside the decoded token.
* `client_uri`: The URI for the app authorized to use the token. Should be the
same as the `azp` field inside the decoded token.
* `scope`: The list of permissions granted by the token. Should be the same
as the `scp` field inside the decoded token.
* `access_token`: The encoded JWT.
* `token`: The decoded token.
### Get All Tokens
* **URL** `/api/goldilocks@daplie.com/tokens`
* **Method** `GET`
* **Reponse**: An array of the tokens stored. Each item looks the same as if it
had been requested individually.
### Save New Token
* **URL** `/api/goldilocks@daplie.com/tokens`
* **Method** `POST`
* **Body**: An object similar to an OAuth3 session used by the javascript
library. The only important fields are `refresh_token` or `access_token`, and
`refresh_token` will be used before `access_token`. (This is because the
`access_token` usually expires quickly, making it meaningless to store.)
* **Reponse**: The response looks the same as a single GET request.
### Delete Token
* **URL** `/api/goldilocks@daplie.com/tokens/:id`
* **Method** `DELETE`
* **Reponse**: Either `{"success":true}` or `{"success":false}`, depending on
whether the token was present before the request.
## Config
### Get All Settings
@ -151,6 +109,24 @@ using the following APIs.
* **Reponse**: The list of domain groups.
## Tunnel
### Check Status
* **URL** `/api/goldilocks@daplie.com/tunnel`
* **Method** `POST`
* **Reponse**: An object whose keys are the URLs for the tunnels, and whose
properties are arrays of the tunnel tokens.
This route with return only the sessions started by the same user who is
checking the status.
### Start Tunnel
* **URL** `/api/goldilocks@daplie.com/tunnel`
* **Method** `POST`
This route will use the stored token for the user matching the request
header to request a tunnel token from the audience of the stored token.
## Socks5 Proxy
### Check Status

View File

@ -1,12 +0,0 @@
v1.1.5 - Implemented dns-01 ACME challenges
v1.1.4 - Improved responsiveness to config updates
* changed which TCP/UDP ports are bound to on config update
* update tunnel server settings on config update
* update socks5 setting on config update
v1.1.3 - Better late than never... here's some stuff we've got
* fixed (probably) network settings not being readable
* supports timeouts in loopback check
* loopback check less likely to fail / throw errors, will try again
* supports ddns using audience of token

41
LICENSE
View File

@ -1,41 +0,0 @@
Copyright 2017 Daplie, Inc
This is open source software; you can redistribute it and/or modify it under the
terms of either:
a) the "MIT License"
b) the "Apache-2.0 License"
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Apache-2.0 License Summary
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

3
LICENSE.txt Normal file
View File

@ -0,0 +1,3 @@
Hello all. We make our source code available to view, but we retain copyright.
It's not because we're trying to be mean or anything, we just want to maintain our distribution channel.

260
README.md
View File

@ -20,51 +20,17 @@ The node.js netserver that's just right.
Install Standalone
-------
### curl | bash
```bash
curl -fsSL https://git.coolaj86.com/coolaj86/goldilocks.js/raw/v1.1/installer/get.sh | bash
```
### git
```bash
git clone https://git.coolaj86.com/coolaj86/goldilocks.js
pushd goldilocks.js
git checkout v1.1
bash installer/install.sh
```
### npm
```bash
# v1 in git (unauthenticated)
npm install -g git+https://git@git.coolaj86.com:coolaj86/goldilocks.js#v1
# v1 in npm
npm install -g goldilocks
# v1 in git (via ssh)
npm install -g git+ssh://git@git.coolaj86.com:coolaj86/goldilocks.js#v1
npm install -g git+ssh://git@git.daplie.com:Daplie/goldilocks.js#v1
# v1 in npm
npm install -g goldilocks@v1
# v1 in git (unauthenticated)
npm install -g git+https://git@git.daplie.com:Daplie/goldilocks.js#v1
```
### Uninstall
Remove goldilocks and services:
```
rm -rf /opt/goldilocks/ /srv/goldilocks/ /var/goldilocks/ /var/log/goldilocks/ /etc/tmpfiles.d/goldilocks.conf /etc/systemd/system/goldilocks.service
```
Remove config as well
```
rm -rf /etc/goldilocks/ /etc/ssl/goldilocks
```
Usage
-----
```bash
goldilocks
```
@ -81,7 +47,7 @@ We have service support for
* launchd (macOS)
```bash
curl https://git.coolaj86.com/coolaj86/goldilocks.js/raw/master/install.sh | bash
curl https://git.daplie.com/Daplie/goldilocks.js/raw/master/install.sh | bash
```
Modules & Configuration
@ -98,15 +64,13 @@ some of which have modules:
- [proxy (reverse proxy)](#tlsproxy)
- [acme](#tlsacme)
* [tcp](#tcp)
- [proxy](#tcpproxy)
- [forward](#tcpforward)
* [udp](#udp)
- [forward](#udpforward)
* [domains](#domains)
* [tunnel_server](#tunnel_server)
* [DDNS](#ddns)
* [tunnel_client](#tunnel)
* [mDNS](#mdns)
* [mdns](#mdns)
* [socks5](#socks5)
* api
@ -183,18 +147,6 @@ root The path to serve as a string.
The template variable `:hostname` represents the HTTP Host header without port information
ex: `root: /srv/www/example.com` would load the example.com folder for any domain listed
ex: `root: /srv/www/:hostname` would load `/srv/www/example.com` if so indicated by the Host header
index Set to `false` to disable the default behavior of loading `index.html` in directories
ex: `false`
dotfiles Set to `allow` to load dotfiles rather than ignoring them
ex: `"allow"`
redirect Set to `false` to disable the default behavior of ensuring that directory paths end in '/'
ex: `false`
indexes An array of directories which should be have indexes served rather than blocked
ex: `[ '/' ]` will allow all directories indexes to be served
```
Example config:
@ -225,7 +177,6 @@ to The new URL path which should be used.
If wildcards matches were used they will be available as `:1`, `:2`, etc.
ex: /pics/
ex: /pics/:1/:2/
ex: https://mydomain.com/photos/:1/:2/
```
Example config:
@ -305,16 +256,9 @@ tls:
challenge_type: 'http-01'
```
**NOTE:** If you specify `dns-01` as the challenge type there must also be a
[DDNS module](#ddns) defined for all of the relevant domains (though not all
domains handled by a single TLS module need to be handled by the same DDNS
module). The DDNS module provides all of the information needed to actually
set the DNS records needed to verify ownership.
### tcp
The tcp system handles both *raw* and *tls-terminated* tcp network traffic
(see the _Note_ section below the example). It may use port numbers
The tcp system handles all tcp network traffic **before decryption** and may use port numbers
or traffic sniffing to determine how the connection should be handled.
It has the following options:
@ -337,83 +281,6 @@ tcp:
address: '127.0.0.1:2222'
```
_Note_: When tcp traffic comes into goldilocks it will be tested against the tcp modules.
The connection may be handed to the TLS module if it appears to be a TLS/SSL/HTTPS connection
and if the tls module terminates the traffic, the connection will be sent back to the TLS module.
Due to the complexity of node.js' networking stack it is not currently possible to tell which
port tls-terminated traffic came from, so only the SNI header (serername / domain name) may be used for
modules matching terminated TLS.
### tcp.proxy
The proxy module routes traffic **after tls-termination** based on the servername (domain name)
contained in a SNI header. As such this only works to route TCP connections wrapped in a TLS stream.
It has the same options as the [HTTP proxy module](#httpproxy-how-to-reverse-proxy-ruby-python-etc).
This is particularly useful for routing ssh and vpn traffic over tcp port 443 as wrapped TLS
connections in order to access one of your servers even when connecting from a harsh or potentially
misconfigured network environment (i.e. hotspots in public libraries and shopping malls).
Example config:
```yml
tcp:
modules:
- type: proxy
domains:
- ssh.example.com # Note: this domain would also listed in tls.acme.domains
host: localhost
port: 22
- type: proxy
domains:
- vpn.example.com # Note: this domain would also listed in tls.acme.domains
host: localhost
port: 1194
```
_Note_: In same cases network administrators purposefully block ssh and vpn connections using
Application Firewalls with DPI (deep packet inspection) enabled. You should read the ToS of the
network you are connected to to ensure that you aren't subverting policies that are purposefully
in place on such networks.
#### Using with ssh
In order to use this to route SSH connections you will need to use `ssh`'s
`ProxyCommand` option. For example to use the TLS certificate for `ssh.example.com`
to wrap an ssh connection you could use the following command:
```bash
ssh user@example.com -o ProxyCommand='openssl s_client -quiet -connect example.com:443 -servername ssh.example.com'
```
Alternatively you could add the following lines to your ssh config file.
```
Host example.com
ProxyCommand openssl s_client -quiet -connect example.com:443 -servername ssh.example.com
```
#### Using with OpenVPN
There are two strategies that will work well for you:
1) [Use ssh](https://redfern.me/tunneling-openvpn-through-ssh/) with the config above to reverse proxy tcp port 1194 to you.
```bash
ssh -L 1194:localhost:1194 example.com
```
2) [Use stunnel]https://serverfault.com/questions/675553/stunnel-vpn-traffic-and-ensure-it-looks-like-ssl-traffic-on-port-443/681497)
```
[openvpn-over-goldilocks]
client = yes
accept = 127.0.0.1:1194
sni = vpn.example.com
connect = example.com:443
```
3) [Use stunnel.js](https://git.coolaj86.com/coolaj86/tunnel-client.js) as described in the "tunnel_server" section below.
### tcp.forward
The forward module routes traffic based on port number **without decrypting** it.
@ -486,45 +353,27 @@ udp:
To reduce repetition defining multiple modules that operate on the same domain
name the `domains` field can define multiple modules of multiple types for a
single list of names. The modules defined this way do not need to have their
own `domains` field. Note that the [tcp.forward](#tcpforward) module is not
allowed in a domains group since its routing is not based on domains.
own `domains` field.
Example Config
```yml
domains:
- names:
- example.com
- www.example.com
- api.example.com
modules:
tls:
- type: acme
email: joe.schmoe@example.com
challenge_type: 'http-01'
http:
- type: redirect
from: /deprecated/path
to: /new/path
- type: proxy
port: 3000
dns:
- type: 'dns@oauth3.org'
token_id: user_token_id
- names:
- ssh.example.com
modules:
tls:
- type: acme
email: john.smith@example.com
challenge_type: 'http-01'
tcp:
- type: proxy
port: 22
dns:
- type: 'dns@oauth3.org'
token_id: user_token_id
names:
- example.com
- www.example.com
- api.example.com
modules:
tls:
- type: acme
email: joe.schmoe@example.com
challenge_type: 'http-01'
http:
- type: redirect
from: /deprecated/path
to: /new/path
- type: proxy
port: 3000
```
@ -554,50 +403,31 @@ tunnel_server:
- 'api.tunnel.example.com'
```
### DDNS
### tunnel
The DDNS module watches the network environment of the unit and makes sure the
device is always accessible on the internet using the domains listed in the
config. If the device has a public address or if it can automatically set up
port forwarding the device will periodically check its public address to ensure
the DNS records always point to it. Otherwise it will to connect to a tunnel
server and set the DNS records to point to that server.
The tunnel client is meant to be run from behind a firewalls, carrier-grade NAT,
or otherwise inaccessible devices to allow them to be accessed publicly on the
internet.
The `loopback` setting specifies how the unit will check its public IP address
and whether connections can reach it. Currently only `tunnel@oauth3.org` is
supported. If the loopback setting is not defined it will default to using
`oauth3.org`.
It has no options per se, but is rather a list of tokens that can be used to
connect to tunnel servers. If the token does not have an `aud` field it must be
provided in an object with the token provided in the `jwt` field and the tunnel
server url provided in the `tunnelUrl` field.
The `tunnel` setting can be used to specify how to connect to the tunnel.
Currently only `tunnel@oauth3.org` is supported. The token specified in the
`tunnel` setting will be used to acquire the tokens that are used directly with
the tunnel server. If the tunnel setting is not defined it will default to try
using the tokens in the modules for the relevant domains.
Example config:
If a particular DDNS module has been disabled the device will still try to set
up port forwarding (and connect to a tunnel if that doesn't work), but the DNS
records will not be updated to point to the device. This is to allow a setup to
be tested before transitioning services between devices.
```yaml
ddns:
disabled: false
loopback:
type: 'tunnel@oauth3.org'
domain: oauth3.org
tunnel:
type: 'tunnel@oauth3.org'
token_id: user_token_id
modules:
- type: 'dns@oauth3.org'
token_id: user_token_id
domains:
- www.example.com
- api.example.com
- test.example.com
```yml
tunnel:
- 'some.jwt_encoded.token'
- jwt: 'other.jwt_encoded.token'
tunnelUrl: 'wss://api.tunnel.example.com/'
```
### mDNS
### ddns
TODO
### mdns
enabled by default
@ -616,7 +446,7 @@ mdns:
You can discover goldilocks with `mdig`.
```
npm install -g git+https://git.coolaj86.com/coolaj86/mdig.js.git
npm install -g git+https://git.daplie.com/Daplie/mdig.git
mdig _cloud._tcp.local
```
@ -645,7 +475,7 @@ TODO
* [ ] http - redirect based on domain name (not just path)
* [ ] tcp - bind should be able to specify localhost, uniquelocal, private, or ip
* [ ] tcp - if destination host is omitted default to localhost, if dst port is missing, default to src
* [ ] sys - `curl https://coolaj86.com/goldilocks | bash -s example.com`
* [ ] sys - `curl https://daplie.me/goldilocks | bash -s example.com`
* [ ] oauth3 - `example.com/.well-known/domains@oauth3.org/directives.json`
* [ ] oauth3 - commandline questionnaire
* [x] modules - use consistent conventions (i.e. address vs host + port)

View File

@ -311,6 +311,7 @@ function fillConfig(config, args) {
config.debug = config.debug || args.debug;
config.socks5 = config.socks5 || { enabled: false };
config.ddns = config.ddns || { enabled: false };
// Use Object.assign to copy any real config values over the default values so we can
// easily make sure all the fields we need exist .
@ -337,10 +338,20 @@ function fillConfig(config, args) {
fillComponent('tcp', true);
fillComponent('http', false);
fillComponent('tls', false);
fillComponent('ddns', false);
if (!config.tls.acme && (args.email || args.agreeTos)) {
config.tls.acme = {};
}
if (typeof args.agreeTos === 'string') {
config.tls.acme.approvedDomains = args.agreeTos.split(',');
}
if (args.email) {
config.tls.acme.email = args.email;
}
config.device = { hostname: require('os').hostname() };
config.tunnel = args.tunnel || config.tunnel;
if (Array.isArray(config.tcp.bind) && config.tcp.bind.length) {
return PromiseA.resolve(config);
}
@ -440,7 +451,9 @@ function readEnv(args) {
} catch (err) {}
var env = {
cwd: process.env.GOLDILOCKS_HOME || process.cwd()
tunnel: process.env.GOLDILOCKS_TUNNEL_TOKEN || process.env.GOLDILOCKS_TUNNEL && true
, email: process.env.GOLDILOCKS_EMAIL
, cwd: process.env.GOLDILOCKS_HOME || process.cwd()
, debug: process.env.GOLDILOCKS_DEBUG && true
};
@ -451,7 +464,10 @@ var program = require('commander');
program
.version(require('../package.json').version)
.option('--agree-tos [url1,url2]', "agree to all Terms of Service for Daplie, Let's Encrypt, etc (or specific URLs only)")
.option('-c --config <file>', 'Path to config file (Goldilocks.json or Goldilocks.yml) example: --config /etc/goldilocks/Goldilocks.json')
.option('--tunnel [token]', 'Turn tunnel on. This will enter interactive mode for login if no token is specified.')
.option('--email <email>', "(Re)set default email to use for Daplie, Let's Encrypt, ACME, etc.")
.option('--debug', "Enable debug output")
.parse(process.argv);

View File

@ -1,5 +0,0 @@
# /etc/tmpfiles.d/goldilocks.conf
# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
# Type Path Mode UID GID Age Argument
d /run/goldilocks 0755 MY_USER MY_GROUP - -

View File

@ -44,7 +44,7 @@ http:
- localhost.beta.daplie.me
status: 301
from: /old/path/*/other/*
to: https://example.com/path/new/:2/something/:1
to: /path/new/:2/something/:1
- type: proxy
domains:
- localhost.daplie.me
@ -85,22 +85,12 @@ mdns:
broadcast: '224.0.0.251'
ttl: 300
# tunnel: jwt
# tunnel:
# - jwt1
# - jwt2
tunnel_server:
secret: abc123
servernames:
- 'tunnel.localhost.com'
ddns:
loopback:
type: 'tunnel@oauth3.org'
domain: oauth3.org
tunnel:
type: 'tunnel@oauth3.org'
token: user_token_id
modules:
- type: 'dns@oauth3.org'
token: user_token_id
domains:
- www.example.com
- api.example.com
- test.example.com

View File

@ -19,14 +19,14 @@ StartLimitBurst=3
# User and group the process will run as
# (www-data is the de facto standard on most systems)
User=MY_USER
Group=MY_GROUP
User=www-data
Group=www-data
# If we need to pass environment variables in the future
Environment=GOLDILOCKS_PATH=/srv/www NODE_PATH=/opt/goldilocks/lib/node_modules NPM_CONFIG_PREFIX=/opt/goldilocks
# Set a sane working directory, sane flags, and specify how to reload the config file
WorkingDirectory=/opt/goldilocks
WorkingDirectory=/srv/www
ExecStart=/opt/goldilocks/bin/node /opt/goldilocks/bin/goldilocks --config /etc/goldilocks/goldilocks.yml
ExecReload=/bin/kill -USR1 $MAINPID
@ -46,7 +46,7 @@ ProtectSystem=full
# … except TLS/SSL, ACME, and Let's Encrypt certificates
# and /var/log/goldilocks, because we want a place where logs can go.
# This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks /opt/goldilocks
ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks
# you may also want to add other directories such as /opt/goldilocks /etc/acme /etc/letsencrypt
# Note: in v231 and above ReadWritePaths has been renamed to ReadWriteDirectories

View File

@ -0,0 +1,10 @@
# /etc/tmpfiles.d/goldilocks.conf
# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
# Type Path Mode UID GID Age Argument
d /etc/goldilocks 0755 www-data www-data - -
d /opt/goldilocks 0775 www-data www-data - -
d /srv/www 0775 www-data www-data - -
d /etc/ssl/goldilocks 0750 www-data www-data - -
d /var/log/goldilocks 0750 www-data www-data - -
#d /run/goldilocks 0755 www-data www-data - -

224
install.sh Normal file
View File

@ -0,0 +1,224 @@
#!/bin/bash
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
MY_ROOT=""
if [ -z "${PREFIX-}" ]; then
MY_ROOT=""
else
MY_ROOT="$PREFIX"
fi
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
###############################
# #
# http_get #
# boilerplate for curl / wget #
# #
###############################
# See https://git.daplie.com/Daplie/daplie-snippets/blob/master/bash/http-get.sh
http_get=""
http_opts=""
http_out=""
detect_http_get()
{
if type -p curl >/dev/null 2>&1; then
http_get="curl"
http_opts="-fsSL"
http_out="-o"
elif type -p wget >/dev/null 2>&1; then
http_get="wget"
http_opts="--quiet"
http_out="-O"
else
echo "Aborted, could not find curl or wget"
return 7
fi
}
dap_dl()
{
$http_get $http_opts $http_out "$2" "$1"
touch "$2"
}
dap_dl_bash()
{
dap_url=$1
#dap_args=$2
rm -rf dap-tmp-runner.sh
$http_get $http_opts $http_out dap-tmp-runner.sh "$dap_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
}
detect_http_get
## END HTTP_GET ##
###################
# #
# Install service #
# #
###################
my_app_name=goldilocks
my_app_pkg_name=com.daplie.goldilocks.web
my_app_dir=$(mktemp -d)
installer_base="https://git.daplie.com/Daplie/goldilocks.js/raw/master"
my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
my_app_etc_example_config="etc/${my_app_name}/${my_app_name}.example.yml"
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
install_for_systemd()
{
echo ""
echo "Installing as systemd service"
echo ""
mkdir -p $(dirname "$my_app_dir/$my_app_systemd_service")
dap_dl "$installer_base/$my_app_systemd_service" "$my_app_dir/$my_app_systemd_service"
$sudo_cmd mv "$my_app_dir/$my_app_systemd_service" "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_service"
mkdir -p $(dirname "$my_app_dir/$my_app_systemd_tmpfiles")
dap_dl "$installer_base/$my_app_systemd_tmpfiles" "$my_app_dir/$my_app_systemd_tmpfiles"
$sudo_cmd mv "$my_app_dir/$my_app_systemd_tmpfiles" "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null
$sudo_cmd systemctl daemon-reload
$sudo_cmd systemctl start "${my_app_name}.service"
$sudo_cmd systemctl enable "${my_app_name}.service"
echo "$my_app_name started with systemctl, check its status like so"
echo " $sudo_cmd systemctl status $my_app_name"
echo " $sudo_cmd journalctl -xe -u goldilocks"
}
install_for_launchd()
{
echo ""
echo "Installing as launchd service"
echo ""
# See http://www.launchd.info/
mkdir -p $(dirname "$my_app_dir/$my_app_launchd_service")
dap_dl "$installer_base/$my_app_launchd_service" "$my_app_dir/$my_app_launchd_service"
$sudo_cmd mv "$my_app_dir/$my_app_launchd_service" "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd chown root:wheel "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd chmod 0644 "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd launchctl unload -w "$MY_ROOT/$my_app_launchd_service" >/dev/null 2>/dev/null
$sudo_cmd launchctl load -w "$MY_ROOT/$my_app_launchd_service"
echo "$my_app_name started with launchd"
}
install_etc_config()
{
$sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_example_config")
mkdir -p $(dirname "$my_app_dir/$my_app_etc_example_config")
dap_dl "$installer_base/$my_app_etc_example_config" "$my_app_dir/$my_app_etc_example_config"
$sudo_cmd mv "$my_app_dir/$my_app_etc_example_config" "$MY_ROOT/$my_app_etc_example_config"
if [ ! -e "$MY_ROOT/$my_app_etc_config" ]; then
$sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_config")
mkdir -p $(dirname "$my_app_dir/$my_app_etc_config")
dap_dl "$installer_base/$my_app_etc_config" "$my_app_dir/$my_app_etc_config"
$sudo_cmd mv "$my_app_dir/$my_app_etc_config" "$MY_ROOT/$my_app_etc_config"
fi
# OS X
$sudo_cmd chown -R _www:_www $(dirname "$MY_ROOT/$my_app_etc_config") || true
# Linux
$sudo_cmd chown -R www-data:www-data $(dirname "$MY_ROOT/$my_app_etc_config") || true
$sudo_cmd chmod 775 $(dirname "$MY_ROOT/$my_app_etc_config")
$sudo_cmd chmod 664 "$MY_ROOT/$my_app_etc_config"
}
install_service()
{
install_etc_config
installable=""
if [ -d "$MY_ROOT/etc/systemd/system" ]; then
install_for_systemd
installable="true"
fi
if [ -d "/Library/LaunchDaemons" ]; then
install_for_launchd
installable="true"
fi
if [ -z "$installable" ]; then
echo ""
echo "Unknown system service init type. You must install as a system service manually."
echo '(please file a bug with the output of "uname -a")'
echo ""
fi
}
## END SERVICE_INSTALL ##
set -e
set -u
# Install
# TODO install to tmp location, then move to /opt
export NODE_PATH=/opt/goldilocks/lib/node_modules
export NPM_CONFIG_PREFIX=/opt/goldilocks
$sudo_cmd mkdir -p /etc/goldilocks
$sudo_cmd mkdir -p /var/log/goldilocks
$sudo_cmd mkdir -p /srv/www
$sudo_cmd mkdir -p /var/www
$sudo_cmd mkdir -p /opt/goldilocks/{lib,bin,etc}
# Dependencies
dap_dl_bash "https://git.daplie.com/Daplie/node-install-script/raw/master/setup-min.sh"
# Change to user perms
# OS X or Linux
$sudo_cmd chown -R $(whoami) /opt/goldilocks/ || true
my_npm="$NPM_CONFIG_PREFIX/bin/npm"
$my_npm install -g npm@4
$my_npm install -g 'git+https://git@git.daplie.com/Daplie/goldilocks.js.git'
# Finish up with submodule
pushd /opt/goldilocks/lib/node_modules/goldilocks
bash ./update-packages.sh
popd
# Change to admin perms
# OS X
$sudo_cmd chown -R _www:_www /var/www /srv/www /opt/goldilocks || true
# Linux
$sudo_cmd chown -R www-data:www-data /var/www /srv/www /opt/goldilocks || true
# make sure the files are all read/write for the owner and group, and then set
# the setuid and setgid bits so that any files/directories created inside these
# directories have the same owner and group.
$sudo_cmd chmod -R ug+rwX /opt/goldilocks
find /opt/goldilocks -type d -exec $sudo_cmd chmod ug+s {} \;
# Uninstall
dap_dl "https://git.daplie.com/Daplie/goldilocks.js/raw/master/uninstall.sh" "./goldilocks-uninstall"
$sudo_cmd chmod 755 "./goldilocks-uninstall"
# OS X
$sudo_cmd chown root:wheel "./goldilocks-uninstall" || true
# Linux
$sudo_cmd chown root:root "./goldilocks-uninstall" || true
$sudo_cmd mv "./goldilocks-uninstall" "/usr/local/bin/uninstall-goldilocks"
# Install Service
install_service

View File

@ -1,20 +0,0 @@
set -e
set -u
my_name=goldilocks
# TODO provide an option to supply my_ver and my_tmp
my_ver=master
my_tmp=$(mktemp -d)
mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
git clone https://git.coolaj86.com/coolaj86/goldilocks.js.git $my_tmp/opt/$my_name/lib/node_modules/$my_name
echo "Installing to $my_tmp (will be moved after install)"
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
git checkout $my_ver
source ./installer/install.sh
popd
echo "Installation successful, now cleaning up $my_tmp ..."
rm -rf $my_tmp
echo "Done"

View File

@ -1,48 +0,0 @@
###############################
# #
# http_get #
# boilerplate for curl / wget #
# #
###############################
# See https://git.coolaj86.com/coolaj86/snippets/blob/master/bash/http-get.sh
_h_http_get=""
_h_http_opts=""
_h_http_out=""
detect_http_get()
{
set +e
if type -p curl >/dev/null 2>&1; then
_h_http_get="curl"
_h_http_opts="-fsSL"
_h_http_out="-o"
elif type -p wget >/dev/null 2>&1; then
_h_http_get="wget"
_h_http_opts="--quiet"
_h_http_out="-O"
else
echo "Aborted, could not find curl or wget"
return 7
fi
set -e
}
http_get()
{
$_h_http_get $_h_http_opts $_h_http_out "$2" "$1"
touch "$2"
}
http_bash()
{
_http_url=$1
#dap_args=$2
rm -rf dap-tmp-runner.sh
$_h_http_get $_h_http_opts $_h_http_out dap-tmp-runner.sh "$_http_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
}
detect_http_get
## END HTTP_GET ##

View File

@ -1,17 +0,0 @@
set -u
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
echo ""
echo "Installing as launchd service"
echo ""
# See http://www.launchd.info/
safe_copy_config "$my_app_dist/$my_app_launchd_service" "$my_root/$my_app_launchd_service"
$sudo_cmd chown root:wheel "$my_root/$my_app_launchd_service"
$sudo_cmd launchctl unload -w "$my_root/$my_app_launchd_service" >/dev/null 2>/dev/null
$sudo_cmd launchctl load -w "$my_root/$my_app_launchd_service"
echo "$my_app_name started with launchd"

View File

@ -1,37 +0,0 @@
set -u
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
echo ""
echo "Installing as systemd service"
echo ""
sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_service" > "$my_app_dist/$my_app_systemd_service.2"
sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_service.2" > "$my_app_dist/$my_app_systemd_service"
rm "$my_app_dist/$my_app_systemd_service.2"
safe_copy_config "$my_app_dist/$my_app_systemd_service" "$my_root/$my_app_systemd_service"
$sudo_cmd chown root:root "$my_root/$my_app_systemd_service"
sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_tmpfiles" > "$my_app_dist/$my_app_systemd_tmpfiles.2"
sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_tmpfiles.2" > "$my_app_dist/$my_app_systemd_tmpfiles"
rm "$my_app_dist/$my_app_systemd_tmpfiles.2"
safe_copy_config "$my_app_dist/$my_app_systemd_tmpfiles" "$my_root/$my_app_systemd_tmpfiles"
$sudo_cmd chown root:root "$my_root/$my_app_systemd_tmpfiles"
$sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null || true
$sudo_cmd systemctl daemon-reload
$sudo_cmd systemctl start "${my_app_name}.service"
$sudo_cmd systemctl enable "${my_app_name}.service"
echo ""
echo ""
echo "Fun systemd commands to remember:"
echo " $sudo_cmd systemctl daemon-reload"
echo " $sudo_cmd systemctl restart $my_app_name.service"
echo ""
echo "$my_app_name started with systemctl, check its status like so:"
echo " $sudo_cmd systemctl status $my_app_name"
echo " $sudo_cmd journalctl -xefu $my_app_name"
echo ""
echo ""

View File

@ -1,37 +0,0 @@
safe_copy_config()
{
src=$1
dst=$2
$sudo_cmd mkdir -p $(dirname "$dst")
if [ -f "$dst" ]; then
$sudo_cmd rsync -a "$src" "$dst.latest"
# TODO edit config file with $my_user and $my_group
if [ "$(cat $dst)" == "$(cat $dst.latest)" ]; then
$sudo_cmd rm $dst.latest
else
echo "MANUAL INTERVENTION REQUIRED: check the systemd script update and manually decide what you want to do"
echo "diff $dst $dst.latest"
$sudo_cmd chown -R root:root "$dst.latest"
fi
else
$sudo_cmd rsync -a --ignore-existing "$src" "$dst"
fi
$sudo_cmd chown -R root:root "$dst"
$sudo_cmd chmod 644 "$dst"
}
installable=""
if [ -d "$my_root/etc/systemd/system" ]; then
source ./installer/install-for-systemd.sh
installable="true"
fi
if [ -d "/Library/LaunchDaemons" ]; then
source ./installer/install-for-launchd.sh
installable="true"
fi
if [ -z "$installable" ]; then
echo ""
echo "Unknown system service init type. You must install as a system service manually."
echo '(please file a bug with the output of "uname -a")'
echo ""
fi

View File

@ -1,150 +0,0 @@
#!/bin/bash
set -e
set -u
### IMPORTANT ###
### VERSION ###
my_name=goldilocks
my_app_pkg_name=com.coolaj86.goldilocks.web
my_app_ver="v1.1"
my_azp_oauth3_ver="v1.2.3"
export NODE_VERSION="v8.9.3"
if [ -z "${my_tmp-}" ]; then
my_tmp="$(mktemp -d)"
mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
echo "Installing to $my_tmp (will be moved after install)"
git clone ./ $my_tmp/opt/$my_name/lib/node_modules/$my_name
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
fi
#################
export NODE_PATH=$my_tmp/opt/$my_name/lib/node_modules
export PATH=$my_tmp/opt/$my_name/bin/:$PATH
export NPM_CONFIG_PREFIX=$my_tmp/opt/$my_name
my_npm="$NPM_CONFIG_PREFIX/bin/npm"
#################
my_app_dist=$my_tmp/opt/$my_name/lib/node_modules/$my_name/dist
installer_base="https://git.coolaj86.com/coolaj86/goldilocks.js/raw/$my_app_ver"
# Backwards compat
# some scripts still use the old names
my_app_dir=$my_tmp
my_app_name=$my_name
git checkout $my_app_ver
mkdir -p "$my_tmp/opt/$my_name"/{lib,bin,etc}
ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name
ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name.js
mkdir -p "$my_tmp/etc/$my_name"
chmod 775 "$my_tmp/etc/$my_name"
cat "$my_app_dist/etc/$my_name/$my_name.example.yml" > "$my_tmp/etc/$my_name/$my_name.example.yml"
chmod 664 "$my_tmp/etc/$my_name/$my_name.example.yml"
mkdir -p $my_tmp/srv/www
mkdir -p $my_tmp/var/www
mkdir -p $my_tmp/var/log/$my_name
#
# Helpers
#
source ./installer/sudo-cmd.sh
source ./installer/http-get.sh
#
# Dependencies
#
echo $NODE_VERSION > /tmp/NODEJS_VER
http_bash "https://git.coolaj86.com/coolaj86/node-installer.sh/raw/v1.1/install.sh"
$my_npm install -g npm@4
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
$my_npm install
popd
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name/packages/assets
OAUTH3_GIT_URL="https://git.oauth3.org/OAuth3/oauth3.js.git"
git clone ${OAUTH3_GIT_URL} oauth3.org || true
ln -s oauth3.org org.oauth3
pushd oauth3.org
git remote set-url origin ${OAUTH3_GIT_URL}
git checkout $my_azp_oauth3_ver
#git pull
popd
mkdir -p jquery.com
ln -s jquery.com com.jquery
pushd jquery.com
http_get 'https://code.jquery.com/jquery-3.1.1.js' jquery-3.1.1.js
popd
mkdir -p google.com
ln -s google.com com.google
pushd google.com
http_get 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js' angular.1.6.2.min.js
popd
mkdir -p well-known
ln -s well-known .well-known
pushd well-known
ln -snf ../oauth3.org/well-known/oauth3 ./oauth3
popd
echo "installed dependencies"
popd
#
# System Service
#
source ./installer/my-root.sh
echo "Pre-installation to $my_tmp complete, now installing to $my_root/ ..."
set +e
if type -p tree >/dev/null 2>/dev/null; then
#tree -I "node_modules|include|share" $my_tmp
tree -L 6 -I "include|share|npm" $my_tmp
else
ls $my_tmp
fi
set -e
source ./installer/my-user-my-group.sh
echo "User $my_user Group $my_group"
source ./installer/install-system-service.sh
$sudo_cmd chown -R $my_user:$my_group $my_tmp/*
$sudo_cmd chown root:root $my_tmp/*
$sudo_cmd chown root:root $my_tmp
$sudo_cmd chmod 0755 $my_tmp
# don't change permissions on /, /etc, etc
$sudo_cmd rsync -a --ignore-existing $my_tmp/ $my_root/
$sudo_cmd rsync -a --ignore-existing $my_app_dist/etc/$my_name/$my_name.yml $my_root/etc/$my_name/$my_name.yml
# Change to admin perms
$sudo_cmd chown -R $my_user:$my_group $my_root/opt/$my_name
$sudo_cmd chown -R $my_user:$my_group $my_root/var/www $my_root/srv/www
# make sure the files are all read/write for the owner and group, and then set
# the setuid and setgid bits so that any files/directories created inside these
# directories have the same owner and group.
$sudo_cmd chmod -R ug+rwX $my_root/opt/$my_name
find $my_root/opt/$my_name -type d -exec $sudo_cmd chmod ug+s {} \;
echo ""
echo "$my_name installation complete!"
echo ""
echo ""
echo "Update the config at: /etc/$my_name/$my_name.yml"
echo ""
echo "Unistall: rm -rf /srv/$my_name/ /var/$my_name/ /etc/$my_name/ /opt/$my_name/ /var/log/$my_name/ /etc/tmpfiles.d/$my_name.conf /etc/systemd/system/$my_name.service /etc/ssl/$my_name"

View File

@ -1,8 +0,0 @@
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
my_root=""
if [ -z "${PREFIX-}" ]; then
my_root=""
else
my_root="$PREFIX"
fi

View File

@ -1,19 +0,0 @@
if type -p adduser >/dev/null 2>/dev/null; then
if [ -z "$(cat $my_root/etc/passwd | grep $my_app_name)" ]; then
$sudo_cmd adduser --home $my_root/opt/$my_app_name --gecos '' --disabled-password $my_app_name
fi
my_user=$my_app_name
my_group=$my_app_name
elif [ -n "$(cat /etc/passwd | grep www-data:)" ]; then
# Linux (Ubuntu)
my_user=www-data
my_group=www-data
elif [ -n "$(cat /etc/passwd | grep _www:)" ]; then
# Mac
my_user=_www
my_group=_www
else
# Unsure
my_user=$(whoami)
my_group=$(id -g -n)
fi

View File

@ -1,7 +0,0 @@
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
set +e
if type -p sudo >/dev/null 2>/dev/null; then
((EUID)) && [[ -z "${ANDROID_ROOT-}" ]] && sudo_cmd="sudo"
fi
set -e

View File

@ -21,7 +21,6 @@ module.exports.create = function (deps, conf) {
res.setHeader('Access-Control-Allow-Origin', req.headers.origin || '*');
res.setHeader('Access-Control-Allow-Methods', methods.join(', '));
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization');
res.setHeader('Access-Control-Allow-Credentials', 'true');
if (req.method.toUpperCase() === 'OPTIONS') {
res.setHeader('Allow', methods.join(', '));
@ -47,19 +46,6 @@ module.exports.create = function (deps, conf) {
};
}
function handlePromise(req, res, prom) {
prom.then(function (result) {
res.send(deps.recase.snakeCopy(result));
}).catch(function (err) {
if (conf.debug) {
console.log(err);
}
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
}
function isAuthorized(req, res, fn) {
var auth = jwt.decode((req.headers.authorization||'').replace(/^bearer\s+/i, ''));
if (!auth) {
@ -259,6 +245,33 @@ module.exports.create = function (deps, conf) {
});
}
, tunnel: function (req, res) {
if (handleCors(req, res)) {
return;
}
isAuthorized(req, res, function () {
if ('POST' !== req.method) {
res.setHeader('Content-Type', 'application/json');
return deps.tunnelClients.get(req.userId).then(function (result) {
res.end(JSON.stringify(result));
}, function (err) {
res.statusCode = 500;
res.end(JSON.stringify({ error: { message: err.message, code: err.code, uri: err.uri } }));
});
}
return deps.storage.owners.get(req.userId).then(function (session) {
return deps.tunnelClients.start(session).then(function () {
res.setHeader('Content-Type', 'application/json;');
res.end(JSON.stringify({ success: true }));
}, function (err) {
res.setHeader('Content-Type', 'application/json;');
res.statusCode = 500;
res.end(JSON.stringify({ error: { message: err.message, code: err.code, uri: err.uri } }));
});
});
});
}
, request: function (req, res) {
if (handleCors(req, res, '*')) {
return;
@ -290,6 +303,29 @@ module.exports.create = function (deps, conf) {
});
});
}
, loopback: function (req, res) {
if (handleCors(req, res, 'GET')) {
return;
}
isAuthorized(req, res, function () {
var prom;
var query = require('querystring').parse(require('url').parse(req.url).query);
if (query.provider) {
prom = deps.loopback(query.provider);
} else {
prom = deps.storage.owners.get(req.userId).then(function (session) {
return deps.loopback(session.token.aud);
});
}
res.setHeader('Content-Type', 'application/json');
prom.then(function (result) {
res.end(JSON.stringify(result));
}, function (err) {
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
});
}
, paywall_check: function (req, res) {
if (handleCors(req, res, 'GET')) {
return;
@ -383,7 +419,7 @@ module.exports.create = function (deps, conf) {
return;
}
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var update;
if (req.params.group) {
update = {};
@ -395,13 +431,16 @@ module.exports.create = function (deps, conf) {
var changer = new (require('./config').ConfigChanger)(conf);
changer.update(update);
return config.save(changer);
}).then(function (newConf) {
}).then(function (config) {
if (req.params.group) {
return newConf[req.params.group];
config = config[req.params.group];
}
return newConf;
res.send(deps.recase.snakeCopy(config));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.extractModList = function (changer, params) {
@ -435,7 +474,7 @@ module.exports.create = function (deps, conf) {
return;
}
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var changer = new (require('./config').ConfigChanger)(conf);
var modList = config.extractModList(changer, req.params);
@ -447,9 +486,12 @@ module.exports.create = function (deps, conf) {
return config.save(changer);
}).then(function (newConf) {
return config.extractModList(newConf, req.params);
res.send(deps.recase.snakeCopy(config.extractModList(newConf, req.params)));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.restful.updateModule = function (req, res, next) {
if (req.params.group === 'domains') {
@ -457,17 +499,18 @@ module.exports.create = function (deps, conf) {
return;
}
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var changer = new (require('./config').ConfigChanger)(conf);
var modList = config.extractModList(changer, req.params);
modList.update(req.params.modId, req.body);
return config.save(changer);
}).then(function (newConf) {
return config.extractModule(newConf, req.params).find(function (mod) {
return mod.id === req.params.modId;
});
res.send(deps.recase.snakeCopy(config.extractModList(newConf, req.params)));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.restful.removeModule = function (req, res, next) {
if (req.params.group === 'domains') {
@ -475,19 +518,22 @@ module.exports.create = function (deps, conf) {
return;
}
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var changer = new (require('./config').ConfigChanger)(conf);
var modList = config.extractModList(changer, req.params);
modList.remove(req.params.modId);
return config.save(changer);
}).then(function (newConf) {
return config.extractModList(newConf, req.params);
res.send(deps.recase.snakeCopy(config.extractModList(newConf, req.params)));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.restful.createDomain = function (req, res) {
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var changer = new (require('./config').ConfigChanger)(conf);
var update = req.body;
@ -496,13 +542,16 @@ module.exports.create = function (deps, conf) {
}
update.forEach(changer.domains.add, changer.domains);
return config.save(changer);
}).then(function (newConf) {
return newConf.domains;
}).then(function (config) {
res.send(deps.recase.snakeCopy(config.domains));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.restful.updateDomain = function (req, res) {
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
if (req.body.modules) {
throw Object.assign(new Error('do not add modules with this route'), {statusCode: 400});
}
@ -510,51 +559,38 @@ module.exports.create = function (deps, conf) {
var changer = new (require('./config').ConfigChanger)(conf);
changer.domains.update(req.params.domId, req.body);
return config.save(changer);
}).then(function (newConf) {
return newConf.domains.find(function (dom) {
return dom.id === req.params.domId;
});
}).then(function (config) {
res.send(deps.recase.snakeCopy(config.domains));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
config.restful.removeDomain = function (req, res) {
var promise = deps.PromiseA.resolve().then(function () {
deps.PromiseA.resolve().then(function () {
var changer = new (require('./config').ConfigChanger)(conf);
changer.domains.remove(req.params.domId);
return config.save(changer);
}).then(function (newConf) {
return newConf.domains;
}).then(function (config) {
res.send(deps.recase.snakeCopy(config.domains));
}, function (err) {
res.statusCode = err.statusCode || 500;
err.message = err.message || err.toString();
res.end(JSON.stringify({error: {message: err.message, code: err.code}}));
});
handlePromise(req, res, promise);
};
var tokens = { restful: {} };
tokens.restful.getAll = function (req, res) {
handlePromise(req, res, deps.storage.tokens.all());
};
tokens.restful.getOne = function (req, res) {
handlePromise(req, res, deps.storage.tokens.get(req.params.id));
};
tokens.restful.save = function (req, res) {
handlePromise(req, res, deps.storage.tokens.save(req.body));
};
tokens.restful.revoke = function (req, res) {
var promise = deps.storage.tokens.remove(req.params.id).then(function (success) {
return {success: success};
});
handlePromise(req, res, promise);
};
var app = require('express')();
// Handle all of the API endpoints using the old definition style, and then we can
// add middleware without worrying too much about the consequences to older code.
app.use('/:name', handleOldApis);
// Not all routes support all of these methods, but not worth making this more specific
app.use('/', makeCorsHandler(['GET', 'POST', 'PUT', 'DELETE']), isAuthorized, jsonParser);
app.use('/', isAuthorized, jsonParser);
// Not all config routes support PUT or DELETE, but not worth making this more specific
app.use( '/config', makeCorsHandler(['GET', 'POST', 'PUT', 'DELETE']));
app.get( '/config', config.restful.readConfig);
app.get( '/config/:group', config.restful.readConfig);
app.get( '/config/:group/:mod(modules)/:modId?', config.restful.readConfig);
@ -576,10 +612,5 @@ module.exports.create = function (deps, conf) {
app.put( '/config/domains/:domId', config.restful.updateDomain);
app.delete('/config/domains/:domId', config.restful.removeDomain);
app.get( '/tokens', tokens.restful.getAll);
app.get( '/tokens/:id', tokens.restful.getOne);
app.post( '/tokens', tokens.restful.save);
app.delete('/tokens/:id', tokens.restful.revoke);
return app;
};

View File

@ -48,15 +48,6 @@ var moduleSchemas = {
, challenge_type: { type: 'string' }
}
}
// the dns control modules for DDNS
, 'dns@oauth3.org': {
type: 'object'
, required: [ 'token_id' ]
, properties: {
token_id: { type: 'string' }
}
}
};
// forward is basically the same as proxy, but specifies the relevant incoming port(s).
// only allows for the raw transport layers (TCP/UDP)
@ -73,14 +64,6 @@ Object.keys(moduleSchemas).forEach(function (name) {
validator.addSchema(schema, schema.id);
});
function addDomainRequirement(itemSchema) {
var result = Object.assign({}, itemSchema);
result.required = (result.required || []).concat('domains');
result.properties = Object.assign({}, result.properties);
result.properties.domains = { type: 'array', items: { type: 'string' }, minLength: 1};
return result;
}
function toSchemaRef(name) {
return { '$ref': '/modules/'+name };
}
@ -89,14 +72,14 @@ var moduleRefs = {
, tls: [ 'proxy', 'acme' ].map(toSchemaRef)
, tcp: [ 'forward' ].map(toSchemaRef)
, udp: [ 'forward' ].map(toSchemaRef)
, ddns: [ 'dns@oauth3.org' ].map(toSchemaRef)
};
// TCP is a bit special in that it has a module that doesn't operate based on domain name
// (ie forward), and a modules that does (ie proxy). It therefore has different module
// when part of the `domains` config, and when not part of the `domains` config the proxy
// modules must have the `domains` property while forward should not have it.
moduleRefs.tcp.push(addDomainRequirement(toSchemaRef('proxy')));
function addDomainRequirement(itemSchema) {
itemSchema.required = (itemSchema.required || []).concat('domains');
itemSchema.properties = itemSchema.properties || {};
itemSchema.domains = { type: 'array', items: { type: 'string' }, minLength: 1};
return itemSchema;
}
var domainSchema = {
type: 'array'
@ -110,8 +93,6 @@ var domainSchema = {
, properties: {
tls: { type: 'array', items: { oneOf: moduleRefs.tls }}
, http: { type: 'array', items: { oneOf: moduleRefs.http }}
, ddns: { type: 'array', items: { oneOf: moduleRefs.ddns }}
, tcp: { type: 'array', items: { oneOf: ['proxy'].map(toSchemaRef)}}
}
, additionalProperties: false
}
@ -174,34 +155,10 @@ var mdnsSchema = {
}
};
var tunnelSvrSchema = {
type: 'object'
, properties: {
servernames: { type: 'array', items: { type: 'string' }}
, secret: { type: 'string' }
}
};
var ddnsSchema = {
type: 'object'
, properties: {
loopback: {
type: 'object'
, required: [ 'type', 'domain' ]
, properties: {
type: { type: 'string', const: 'tunnel@oauth3.org' }
, domain: { type: 'string'}
}
}
, tunnel: {
type: 'object'
, required: [ 'type', 'token_id' ]
, properties: {
type: { type: 'string', const: 'tunnel@oauth3.org' }
, token_id: { type: 'string'}
}
}
, modules: { type: 'array', items: addDomainRequirement({ oneOf: moduleRefs.ddns })}
enabled: { type: 'boolean' }
}
};
var socks5Schema = {
@ -231,7 +188,6 @@ var mainSchema = {
, ddns: ddnsSchema
, socks5: socks5Schema
, device: deviceSchema
, tunnel_server: tunnelSvrSchema
}
, additionalProperties: false
};
@ -309,8 +265,6 @@ class DomainList extends IdList {
dom.modules = {
http: new ModuleList((dom.modules || {}).http)
, tls: new ModuleList((dom.modules || {}).tls)
, ddns: new ModuleList((dom.modules || {}).ddns)
, tcp: new ModuleList((dom.modules || {}).tcp)
};
});
}
@ -326,17 +280,14 @@ class DomainList extends IdList {
var modLists = {
http: new ModuleList()
, tls: new ModuleList()
, ddns: new ModuleList()
, tcp: new ModuleList()
};
// We add these after instead of in the constructor to run the validation and manipulation
// in the ModList add function since these are all new modules.
if (dom.modules) {
Object.keys(modLists).forEach(function (key) {
if (Array.isArray(dom.modules[key])) {
dom.modules[key].forEach(modLists[key].add, modLists[key]);
}
});
if (dom.modules && Array.isArray(dom.modules.http)) {
dom.modules.http.forEach(modLists.http.add, modLists.http);
}
if (dom.modules && Array.isArray(dom.modules.tls)) {
dom.modules.tls.forEach(modLists.tls.add, modLists.tls);
}
dom.id = require('crypto').randomBytes(4).toString('hex');
@ -349,14 +300,12 @@ class ConfigChanger {
constructor(start) {
Object.assign(this, JSON.parse(JSON.stringify(start)));
delete this.device;
delete this.debug;
this.domains = new DomainList(this.domains);
this.http.modules = new ModuleList(this.http.modules);
this.tls.modules = new ModuleList(this.tls.modules);
this.tcp.modules = new ModuleList(this.tcp.modules);
this.udp.modules = new ModuleList(this.udp.modules);
this.ddns.modules = new ModuleList(this.ddns.modules);
}
update(update) {
@ -365,7 +314,7 @@ class ConfigChanger {
if (update.domains) {
update.domains.forEach(self.domains.add, self.domains);
}
[ 'http', 'tls', 'tcp', 'udp', 'ddns' ].forEach(function (name) {
[ 'http', 'tls', 'tcp', 'udp' ].forEach(function (name) {
if (update[name] && update[name].modules) {
update[name].modules.forEach(self[name].modules.add, self[name].modules);
delete update[name].modules;

149
lib/ddns.js Normal file
View File

@ -0,0 +1,149 @@
'use strict';
module.exports.create = function (deps, conf) {
var PromiseA = deps.PromiseA;
var request = PromiseA.promisify(require('request'));
var OAUTH3 = require('../packages/assets/org.oauth3');
require('../packages/assets/org.oauth3/oauth3.dns.js');
OAUTH3._hooks = require('../packages/assets/org.oauth3/oauth3.node.storage.js');
function dnsType(addr) {
if (/^\d+\.\d+\.\d+\.\d+$/.test(addr)) {
return 'A';
}
if (-1 !== addr.indexOf(':') && /^[a-f:\.\d]+$/i.test(addr)) {
return 'AAAA';
}
}
async function getSession() {
var sessions = await deps.storage.owners.all();
var session = sessions.filter(function (sess) {
return sess.token.scp.indexOf('dns') >= 0;
})[0];
if (!session) {
throw new Error('no sessions with DNS grants');
}
// The OAUTH3 library stores some things on the root session object that we usually
// just leave inside the token, but we need to pull those out before we use it here
session.provider_uri = session.provider_uri || session.token.provider_uri || session.token.iss;
session.client_uri = session.client_uri || session.token.azp;
session.scope = session.scope || session.token.scp;
return session;
}
async function setDeviceAddress(addr) {
var session = await getSession();
var directives = await OAUTH3.discover(session.token.aud);
// Set the address of the device to our public address.
await request({
url: directives.api+'/api/com.daplie.domains/acl/devices/' + conf.device.hostname
, method: 'POST'
, headers: {
'Authorization': 'Bearer ' + session.refresh_token
, 'Accept': 'application/json; charset=utf-8'
}
, json: {
addresses: [
{ value: addr, type: dnsType(addr) }
]
}
});
// Then update all of the records attached to our hostname, first removing the old records
// to remove the reference to the old address, then creating new records for the same domains
// using our new address.
var allDns = OAUTH3.api(directives.api, {session: session, api: 'dns.list'});
var ourDomains = allDns.filter(function (record) {
return record.device === conf.device.hostname;
}).map(function (record) {
var zoneSplit = record.zone.split('.');
return {
tld: zoneSplit.slice(1).join('.')
, sld: zoneSplit[0]
, sub: record.host.slice(0, -(record.zone.length + 1))
};
});
var common = {
api: 'devices.detach'
, session: session
, device: conf.device.hostname
};
await PromiseA.all(ourDomains.map(function (record) {
return OAUTH3.api(directives.api, Object.assign({}, common, record));
}));
common = {
api: 'devices.attach'
, session: session
, device: conf.device.hostname
, ip: addr
, ttl: 300
};
await PromiseA.all(ourDomains.map(function (record) {
return OAUTH3.api(directives.api, Object.assign({}, common, record));
}));
}
async function getDeviceAddresses() {
var session = await getSession();
var directives = await OAUTH3.discover(session.token.aud);
var result = await request({
url: directives.api+'/api/org.oauth3.dns/acl/devices'
, method: 'GET'
, headers: {
'Authorization': 'Bearer ' + session.refresh_token
, 'Accept': 'application/json; charset=utf-8'
}
, json: true
});
if (!result.body) {
throw new Error('No response body in request for device addresses');
}
if (result.body.error) {
throw Object.assign(new Error('error getting device list'), result.body.error);
}
var dev = result.body.devices.filter(function (dev) {
return dev.name === conf.device.hostname;
})[0];
return (dev || {}).addresses || [];
}
var publicAddress;
async function recheckPubAddr() {
if (!conf.ddns.enabled) {
return;
}
var session = await getSession();
var directives = await OAUTH3.discover(session.token.aud);
var addr = await deps.loopback.checkPublicAddr(directives.api);
if (publicAddress === addr) {
return;
}
if (conf.debug) {
console.log('previous public address',publicAddress, 'does not match current public address', addr);
}
await setDeviceAddress(addr);
publicAddress = addr;
}
recheckPubAddr();
setInterval(recheckPubAddr, 5*60*1000);
return {
setDeviceAddress: setDeviceAddress
, getDeviceAddresses: getDeviceAddresses
, recheckPubAddr: recheckPubAddr
};
};

View File

@ -1,122 +0,0 @@
'use strict';
// Much of this file was based on the `le-challenge-ddns` library (which we are not using
// here because it's method of setting records requires things we don't really want).
module.exports.create = function (deps, conf, utils) {
function getReleventSessionId(domain) {
var sessId;
utils.iterateAllModules(function (mod, domainList) {
// We return a truthy value in these cases because of the way the iterate function
// handles modules grouped by domain. By returning true we are saying these domains
// are "handled" and so if there are multiple modules we won't be given the rest.
if (sessId) { return true; }
if (domainList.indexOf(domain) < 0) { return true; }
// But if the domains are relevant but we don't know how to handle the module we
// return false to allow us to look at any other modules that might exist here.
if (mod.type !== 'dns@oauth3.org') { return false; }
sessId = mod.tokenId || mod.token_id;
return true;
});
return sessId;
}
function get(args, domain, challenge, done) {
done(new Error("Challenge.get() does not need an implementation for dns-01. (did you mean Challenge.loopback?)"));
}
// same as get, but external
function loopback(args, domain, challenge, done) {
var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
require('dns').resolveTxt(challengeDomain, done);
}
var activeChallenges = {};
async function removeAsync(args, domain) {
var data = activeChallenges[domain];
if (!data) {
console.warn(new Error('cannot remove DNS challenge for ' + domain + ': already removed'));
return;
}
var session = await utils.getSession(data.sessId);
var directives = await deps.OAUTH3.discover(session.token.aud);
var apiOpts = {
api: 'dns.unset'
, session: session
, type: 'TXT'
, value: data.keyAuthDigest
};
await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, data.splitDomain));
delete activeChallenges[domain];
}
async function setAsync(args, domain, challenge, keyAuth) {
if (activeChallenges[domain]) {
await removeAsync(args, domain, challenge);
}
var sessId = getReleventSessionId(domain);
if (!sessId) {
throw new Error('no DDNS module handles the domain ' + domain);
}
var session = await utils.getSession(sessId);
var directives = await deps.OAUTH3.discover(session.token.aud);
// I'm not sure what role challenge is supposed to play since even in the library
// this code is based on it was never used, but check for it anyway because ...
if (!challenge || keyAuth) {
console.warn(new Error('DDNS challenge missing challenge or keyAuth'));
}
var keyAuthDigest = require('crypto').createHash('sha256').update(keyAuth || '').digest('base64')
.replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
var splitDomain = (await utils.splitDomains(directives.api, [challengeDomain]))[0];
var apiOpts = {
api: 'dns.set'
, session: session
, type: 'TXT'
, value: keyAuthDigest
, ttl: args.ttl || 0
};
await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, splitDomain));
activeChallenges[domain] = {
sessId
, keyAuthDigest
, splitDomain
};
return new Promise(res => setTimeout(res, 1000));
}
// It might be slightly easier to use arguments and apply, but the library that will use
// this function counts the arguments we expect.
function set(a, b, c, d, done) {
setAsync(a, b, c, d).then(result => done(null, result), done);
}
function remove(a, b, c, done) {
removeAsync(a, b, c).then(result => done(null, result), done);
}
function getOptions() {
return {
oauth3: 'oauth3.org'
, debug: conf.debug
, acmeChallengeDns: '_acme-challenge.'
};
}
return {
getOptions
, set
, get
, remove
, loopback
};
};

View File

@ -1,132 +0,0 @@
'use strict';
module.exports.create = function (deps, conf, utils) {
function dnsType(addr) {
if (/^\d+\.\d+\.\d+\.\d+$/.test(addr)) {
return 'A';
}
if (-1 !== addr.indexOf(':') && /^[a-f:\.\d]+$/i.test(addr)) {
return 'AAAA';
}
}
async function setDeviceAddress(session, addr, domains) {
var directives = await deps.OAUTH3.discover(session.token.aud);
// Set the address of the device to our public address.
await deps.request({
url: deps.OAUTH3.url.normalize(directives.api)+'/api/com.daplie.domains/acl/devices/' + conf.device.hostname
, method: 'POST'
, headers: {
'Authorization': 'Bearer ' + session.refresh_token
, 'Accept': 'application/json; charset=utf-8'
}
, json: {
addresses: [
{ value: addr, type: dnsType(addr) }
]
}
});
// Then update all of the records attached to our hostname, first removing the old records
// to remove the reference to the old address, then creating new records for the same domains
// using our new address.
var allDns = await deps.OAUTH3.api(directives.api, {session: session, api: 'dns.list'});
var ourDns = allDns.filter(function (record) {
if (record.device !== conf.device.hostname) {
return false;
}
if ([ 'A', 'AAAA' ].indexOf(record.type) < 0) {
return false;
}
return domains.indexOf(record.host) !== -1;
});
// Of all the DNS records referring to our device and the current list of domains determine
// which domains have records with outdated address, and which ones we can just leave be
// without updating them.
var badAddrDomains = ourDns.filter(function (record) {
return record.value !== addr;
}).map(record => record.host);
var goodAddrDomains = ourDns.filter(function (record) {
return record.value === addr && badAddrDomains.indexOf(record.host) < 0;
}).map(record => record.host);
var requiredUpdates = domains.filter(function (domain) {
return goodAddrDomains.indexOf(domain) < 0;
});
var oldDns = await utils.splitDomains(directives.api, badAddrDomains);
var common = {
api: 'devices.detach'
, session: session
, device: conf.device.hostname
};
await deps.PromiseA.all(oldDns.map(function (record) {
return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
}));
if (conf.debug && badAddrDomains.length) {
console.log('removed bad DNS records for ' + badAddrDomains.join(', '));
}
var newDns = await utils.splitDomains(directives.api, requiredUpdates);
common = {
api: 'devices.attach'
, session: session
, device: conf.device.hostname
, ip: addr
, ttl: 300
};
await deps.PromiseA.all(newDns.map(function (record) {
return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
}));
if (conf.debug && requiredUpdates.length) {
console.log('set new DNS records for ' + requiredUpdates.join(', '));
}
}
async function getDeviceAddresses(session) {
var directives = await deps.OAUTH3.discover(session.token.aud);
var result = await deps.request({
url: deps.OAUTH3.url.normalize(directives.api)+'/api/org.oauth3.dns/acl/devices'
, method: 'GET'
, headers: {
'Authorization': 'Bearer ' + session.refresh_token
, 'Accept': 'application/json; charset=utf-8'
}
, json: true
});
if (!result.body) {
throw new Error('No response body in request for device addresses');
}
if (result.body.error) {
throw Object.assign(new Error('error getting device list'), result.body.error);
}
var dev = result.body.devices.filter(function (dev) {
return dev.name === conf.device.hostname;
})[0];
return (dev || {}).addresses || [];
}
async function removeDomains(session, domains) {
var directives = await deps.OAUTH3.discover(session.token.aud);
var oldDns = await utils.splitDomains(directives.api, domains);
var common = {
api: 'devices.detach'
, session: session
, device: conf.device.hostname
};
await deps.PromiseA.all(oldDns.map(function (record) {
return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
}));
}
return {
getDeviceAddresses
, setDeviceAddress
, removeDomains
};
};

View File

@ -1,326 +0,0 @@
'use strict';
module.exports.create = function (deps, conf) {
var dns = deps.PromiseA.promisifyAll(require('dns'));
var network = deps.PromiseA.promisifyAll(deps.recase.camelCopy(require('network')));
var equal = require('deep-equal');
var utils = require('./utils').create(deps, conf);
var loopback = require('./loopback').create(deps, conf, utils);
var dnsCtrl = require('./dns-ctrl').create(deps, conf, utils);
var challenge = require('./challenge-responder').create(deps, conf, utils);
var tunnelClients = require('./tunnel-client-manager').create(deps, conf, utils);
var loopbackDomain;
var tunnelActive = false;
async function startTunnel(tunnelSession, mod, domainList) {
try {
var dnsSession = await utils.getSession(mod.tokenId);
var tunnelDomain = await tunnelClients.start(tunnelSession || dnsSession, domainList);
var addrList;
try {
addrList = await dns.resolve4Async(tunnelDomain);
} catch (e) {}
if (!addrList || !addrList.length) {
try {
addrList = await dns.resolve6Async(tunnelDomain);
} catch (e) {}
}
if (!addrList || !addrList.length || !addrList[0]) {
throw new Error('failed to lookup IP for tunnel domain "' + tunnelDomain + '"');
}
if (!mod.disabled) {
await dnsCtrl.setDeviceAddress(dnsSession, addrList[0], domainList);
}
} catch (err) {
console.log('error starting tunnel for', domainList.join(', '));
console.log(err);
}
}
async function connectAllTunnels() {
var tunnelSession;
if (conf.ddns.tunnel) {
// In the case of a non-existant token, I'm not sure if we want to throw here and prevent
// any tunnel connections, or if we want to ignore it and fall back to the DNS tokens
tunnelSession = await deps.storage.tokens.get(conf.ddns.tunnel.tokenId);
}
await utils.iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return null; }
return startTunnel(tunnelSession, mod, domainList);
});
tunnelActive = true;
}
async function disconnectTunnels() {
tunnelClients.disconnect();
tunnelActive = false;
await Promise.resolve();
}
async function checkTunnelTokens() {
var oldTokens = tunnelClients.current();
var newTokens = await utils.iterateAllModules(function checkTokens(mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return null; }
var domainStr = domainList.slice().sort().join(',');
// If there is already a token handling exactly the domains this modules
// needs handled remove it from the list of tokens to be removed. Otherwise
// return the module and domain list so we can get new tokens.
if (oldTokens[domainStr]) {
delete oldTokens[domainStr];
} else {
return Promise.resolve({ mod, domainList });
}
});
await Promise.all(Object.values(oldTokens).map(tunnelClients.remove));
if (!newTokens.length) { return; }
var tunnelSession;
if (conf.ddns.tunnel) {
// In the case of a non-existant token, I'm not sure if we want to throw here and prevent
// any tunnel connections, or if we want to ignore it and fall back to the DNS tokens
tunnelSession = await deps.storage.tokens.get(conf.ddns.tunnel.tokenId);
}
await Promise.all(newTokens.map(function ({mod, domainList}) {
return startTunnel(tunnelSession, mod, domainList);
}));
}
var localAddr, gateway;
async function checkNetworkEnv() {
// Since we can't detect the OS level events when a user plugs in an ethernet cable to recheck
// what network environment we are in we check our local network address and the gateway to
// determine if we need to run the loopback check and router configuration again.
var addr = await network.getPrivateIpAsync();
// Until the author of the `network` package publishes the pull request we gave him
// checking the gateway on our units fails because we have the busybox versions of
// the linux commands. Gateway is realistically less important than address, so if
// we fail in getting it go ahead and use the null value.
var gw;
try {
gw = await network.getGatewayIpAsync();
} catch (err) {
gw = null;
}
if (localAddr === addr && gateway === gw) {
return;
}
var loopResult = await loopback(loopbackDomain);
var notLooped = Object.keys(loopResult.ports).filter(function (port) {
return !loopResult.ports[port];
});
// if (notLooped.length) {
// // TODO: try to automatically configure router to forward ports to us.
// }
// If we are on a public address or all ports we are listening on are forwarded to us then
// we don't need the tunnel and we can set the DNS records for all our domains to our public
// address. Otherwise we need to use the tunnel to accept traffic. Also since the tunnel will
// only be listening on ports 80 and 443 if those are forwarded to us we don't want the tunnel.
if (!notLooped.length || (loopResult.ports['80'] && loopResult.ports['443'])) {
if (tunnelActive) {
await disconnectTunnels();
}
} else {
if (!tunnelActive) {
await connectAllTunnels();
}
}
// Don't assign these until the end of the function. This means that if something failed
// in the loopback or tunnel connection that we will try to go through the whole process
// again next time and hopefully the error is temporary (but if not I'm not sure what the
// correct course of action would be anyway).
localAddr = addr;
gateway = gw;
}
var publicAddress;
async function recheckPubAddr() {
await checkNetworkEnv();
if (tunnelActive) {
return;
}
var addr = await loopback.checkPublicAddr(loopbackDomain);
if (publicAddress === addr) {
return;
}
if (conf.debug) {
console.log('previous public address',publicAddress, 'does not match current public address', addr);
}
publicAddress = addr;
await utils.iterateAllModules(function setModuleDNS(mod, domainList) {
if (mod.type !== 'dns@oauth3.org' || mod.disabled) { return null; }
return utils.getSession(mod.tokenId).then(function (session) {
return dnsCtrl.setDeviceAddress(session, addr, domainList);
}).catch(function (err) {
console.log('error setting DNS records for', domainList.join(', '));
console.log(err);
});
});
}
function getModuleDiffs(prevConf) {
var prevMods = {};
var curMods = {};
// this returns a Promise, but since the functions we use are synchronous
// and change our enclosed variables we don't need to wait for the return.
utils.iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return; }
prevMods[mod.id] = { mod, domainList };
return true;
}, prevConf);
utils.iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return; }
curMods[mod.id] = { mod, domainList };
return true;
});
// Filter out all of the modules that are exactly the same including domainList
// since there is no required action to transition.
Object.keys(prevMods).map(function (id) {
if (equal(prevMods[id], curMods[id])) {
delete prevMods[id];
delete curMods[id];
}
});
return {prevMods, curMods};
}
async function cleanOldDns(prevConf) {
var {prevMods, curMods} = getModuleDiffs(prevConf);
// Then remove DNS records for the domains that we are no longer responsible for.
await Promise.all(Object.values(prevMods).map(function ({mod, domainList}) {
// If the module was disabled before there should be any records that we need to clean up
if (mod.disabled) { return; }
var oldDomains;
if (!curMods[mod.id] || curMods[mod.id].disabled || mod.tokenId !== curMods[mod.id].mod.tokenId) {
oldDomains = domainList.slice();
} else {
oldDomains = domainList.filter(function (domain) {
return curMods[mod.id].domainList.indexOf(domain) < 0;
});
}
if (conf.debug) {
console.log('removing old domains for module', mod.id, oldDomains.join(', '));
}
if (!oldDomains.length) {
return;
}
return utils.getSession(mod.tokenId).then(function (session) {
return dnsCtrl.removeDomains(session, oldDomains);
});
}).filter(Boolean));
}
async function setNewDns(prevConf) {
var {prevMods, curMods} = getModuleDiffs(prevConf);
// And add DNS records for any newly added domains.
await Promise.all(Object.values(curMods).map(function ({mod, domainList}) {
// Don't set any new records if the module has been disabled.
if (mod.disabled) { return; }
var newDomains;
if (!prevMods[mod.id] || mod.tokenId !== prevMods[mod.id].mod.tokenId) {
newDomains = domainList.slice();
} else {
newDomains = domainList.filter(function (domain) {
return prevMods[mod.id].domainList.indexOf(domain) < 0;
});
}
if (conf.debug) {
console.log('adding new domains for module', mod.id, newDomains.join(', '));
}
if (!newDomains.length) {
return;
}
return utils.getSession(mod.tokenId).then(function (session) {
return dnsCtrl.setDeviceAddress(session, publicAddress, newDomains);
});
}).filter(Boolean));
}
function check() {
recheckPubAddr().catch(function (err) {
console.error('failed to handle all actions needed for DDNS');
console.error(err);
});
}
check();
setInterval(check, 5*60*1000);
var curConf;
function updateConf() {
if (curConf && equal(curConf.ddns, conf.ddns) && equal(curConf.domains, conf.domains)) {
// We could update curConf, but since everything we care about is the same...
return;
}
if (!curConf || !equal(curConf.ddns.loopback, conf.ddns.loopback)) {
loopbackDomain = 'oauth3.org';
if (conf.ddns && conf.ddns.loopback) {
if (conf.ddns.loopback.type === 'tunnel@oauth3.org' && conf.ddns.loopback.domain) {
loopbackDomain = conf.ddns.loopback.domain;
} else {
console.error('invalid loopback configuration: bad type or missing domain');
}
}
}
if (!curConf) {
// We need to make a deep copy of the config so we can use it next time to
// compare and see what setup/cleanup is needed to adapt to the changes.
curConf = JSON.parse(JSON.stringify(conf));
return;
}
cleanOldDns(curConf).then(function () {
if (!tunnelActive) {
return setNewDns(curConf);
}
if (equal(curConf.ddns.tunnel, conf.ddns.tunnel)) {
return checkTunnelTokens();
} else {
return disconnectTunnels().then(connectAllTunnels);
}
}).catch(function (err) {
console.error('error transitioning DNS between configurations');
console.error(err);
}).then(function () {
// We need to make a deep copy of the config so we can use it next time to
// compare and see what setup/cleanup is needed to adapt to the changes.
curConf = JSON.parse(JSON.stringify(conf));
});
}
updateConf();
return {
loopbackServer: loopback.server
, setDeviceAddress: dnsCtrl.setDeviceAddress
, getDeviceAddresses: dnsCtrl.getDeviceAddresses
, recheckPubAddr: recheckPubAddr
, updateConf: updateConf
, challenge
};
};

View File

@ -1,191 +0,0 @@
'use strict';
module.exports.create = function (deps, config) {
var stunnel = require('stunnel');
var jwt = require('jsonwebtoken');
var activeTunnels = {};
var activeDomains = {};
var customNet = {
createConnection: function (opts, cb) {
console.log('[gl.tunnel] creating connection');
// here "reader" means the socket that looks like the connection being accepted
// here "writer" means the remote-looking part of the socket that driving the connection
var writer;
function usePair(err, reader) {
if (err) {
process.nextTick(function () {
writer.emit('error', err);
});
return;
}
var wrapOpts = Object.assign({localAddress: '127.0.0.2', localPort: 'tunnel-0'}, opts);
wrapOpts.firstChunk = opts.data;
wrapOpts.hyperPeek = !!opts.data;
// Also override the remote and local address info. We use `defineProperty` because
// otherwise we run into problems of setting properties with only getters defined.
Object.defineProperty(reader, 'remoteAddress', { value: wrapOpts.remoteAddress });
Object.defineProperty(reader, 'remotePort', { value: wrapOpts.remotePort });
Object.defineProperty(reader, 'remoteFamiliy', { value: wrapOpts.remoteFamiliy });
Object.defineProperty(reader, 'localAddress', { value: wrapOpts.localAddress });
Object.defineProperty(reader, 'localPort', { value: wrapOpts.localPort });
Object.defineProperty(reader, 'localFamiliy', { value: wrapOpts.localFamiliy });
deps.tcp.handler(reader, wrapOpts);
process.nextTick(function () {
// this cb will cause the stream to emit its (actually) first data event
// (even though it already gave a peek into that first data chunk)
console.log('[tunnel] callback, data should begin to flow');
cb();
});
}
// We used to use `stream-pair` for non-tls connections, but there are places
// that require properties/functions to be present on the socket that aren't
// present on a JSStream so it caused problems.
writer = require('socket-pair').create(usePair);
return writer;
}
};
function fillData(data) {
if (typeof data === 'string') {
data = { jwt: data };
}
if (!data.jwt) {
throw new Error("missing 'jwt' from tunnel data");
}
var decoded = jwt.decode(data.jwt);
if (!decoded) {
throw new Error('invalid JWT');
}
if (!data.tunnelUrl) {
if (!decoded.aud) {
throw new Error('missing tunnelUrl and audience');
}
data.tunnelUrl = 'wss://' + decoded.aud + '/';
}
data.domains = (decoded.domains || []).slice().sort().join(',');
if (!data.domains) {
throw new Error('JWT contains no domains to be forwarded');
}
return data;
}
async function removeToken(data) {
data = fillData(data);
// Not sure if we might want to throw an error indicating the token didn't
// even belong to a server that existed, but since it never existed we can
// consider it as "removed".
if (!activeTunnels[data.tunnelUrl]) {
return;
}
console.log('removing token from tunnel at', data.tunnelUrl);
return activeTunnels[data.tunnelUrl].clear(data.jwt).then(function () {
delete activeDomains[data.domains];
});
}
async function addToken(data) {
data = fillData(data);
if (activeDomains[data.domains]) {
// If already have a token with the exact same domains and to the same tunnel
// server there isn't really a need to add a new one
if (activeDomains[data.domains].tunnelUrl === data.tunnelUrl) {
return;
}
// Otherwise we want to detach from the other tunnel server in favor of the new one
console.warn('added token with the exact same domains as another');
await removeToken(activeDomains[data.domains]);
}
if (!activeTunnels[data.tunnelUrl]) {
console.log('creating new tunnel client for', data.tunnelUrl);
// We create the tunnel without an initial token so we can append the token and
// get the promise that should tell us more about if it worked or not.
activeTunnels[data.tunnelUrl] = stunnel.connect({
stunneld: data.tunnelUrl
, net: customNet
// NOTE: the ports here aren't that important since we are providing a custom
// `net.createConnection` that doesn't actually use the port. What is important
// is that any services we are interested in are listed in this object and have
// a '*' sub-property.
, services: {
https: { '*': 443 }
, http: { '*': 80 }
, smtp: { '*': 25 }
, smtps: { '*': 587 /*also 465/starttls*/ }
, ssh: { '*': 22 }
}
});
}
console.log('appending token to tunnel at', data.tunnelUrl, 'for domains', data.domains);
await activeTunnels[data.tunnelUrl].append(data.jwt);
// Now that we know the tunnel server accepted our token we can save it
// to keep record of what domains we are handling and what tunnel server
// those domains should go to.
activeDomains[data.domains] = data;
// This is mostly for the start, but return the host for the tunnel server
// we've connected to (after stripping the protocol and path away).
return data.tunnelUrl.replace(/^[a-z]*:\/\//i, '').replace(/\/.*/, '');
}
async function acquireToken(session, domains) {
var OAUTH3 = deps.OAUTH3;
// The OAUTH3 library stores some things on the root session object that we usually
// just leave inside the token, but we need to pull those out before we use it here
session.provider_uri = session.provider_uri || session.token.provider_uri || session.token.iss;
session.client_uri = session.client_uri || session.token.azp;
session.scope = session.scope || session.token.scp;
console.log('asking for tunnel token from', session.token.aud);
var opts = {
api: 'tunnel.token'
, session: session
, data: {
domains: domains
, device: {
hostname: config.device.hostname
, id: config.device.uid || config.device.id
}
}
};
var directives = await OAUTH3.discover(session.token.aud);
var tokenData = await OAUTH3.api(directives.api, opts);
return addToken(tokenData);
}
function disconnectAll() {
Object.keys(activeTunnels).forEach(function (key) {
activeTunnels[key].end();
});
}
function currentTokens() {
return JSON.parse(JSON.stringify(activeDomains));
}
return {
start: acquireToken
, startDirect: addToken
, remove: removeToken
, disconnect: disconnectAll
, current: currentTokens
};
};

View File

@ -1,102 +0,0 @@
'use strict';
module.exports.create = function (deps, conf) {
async function getSession(id) {
var session = await deps.storage.tokens.get(id);
if (!session) {
throw new Error('no user token with ID "' + id + '"');
}
return session;
}
function iterateAllModules(action, curConf) {
curConf = curConf || conf;
var promises = [];
curConf.domains.forEach(function (dom) {
if (!dom.modules || !Array.isArray(dom.modules.ddns) || !dom.modules.ddns.length) {
return null;
}
// For the time being all of our things should only be tried once (regardless if it succeeded)
// TODO: revisit this behavior when we support multiple ways of setting records, and/or
// if we want to allow later modules to run if early modules fail.
promises.push(dom.modules.ddns.reduce(function (prom, mod) {
if (prom) { return prom; }
return action(mod, dom.names);
}, null));
});
curConf.ddns.modules.forEach(function (mod) {
promises.push(action(mod, mod.domains));
});
return Promise.all(promises.filter(Boolean));
}
var tldCache = {};
async function updateTldCache(provider) {
var reqObj = {
url: deps.OAUTH3.url.normalize(provider) + '/api/com.daplie.domains/prices'
, method: 'GET'
, json: true
};
var resp = await deps.OAUTH3.request(reqObj);
var tldObj = {};
resp.data.forEach(function (tldInfo) {
if (tldInfo.enabled) {
tldObj[tldInfo.tld] = true;
}
});
tldCache[provider] = {
time: Date.now()
, tlds: tldObj
};
return tldObj;
}
async function getTlds(provider) {
// If we've never cached the results we need to return the promise that will fetch the result,
// otherwise we can return the cached value. If the cached value has "expired", we can still
// return the cached value we just want to update the cache in parellel (making sure we only
// update once).
if (!tldCache[provider]) {
tldCache[provider] = {
updating: true
, tlds: updateTldCache(provider)
};
}
if (!tldCache[provider].updating && Date.now() - tldCache[provider].time > 24 * 60 * 60 * 1000) {
tldCache[provider].updating = true;
updateTldCache(provider);
}
return tldCache[provider].tlds;
}
async function splitDomains(provider, domains) {
var tlds = await getTlds(provider);
return domains.map(function (domain) {
var split = domain.split('.');
var tldSegCnt = tlds[split.slice(-2).join('.')] ? 2 : 1;
// Currently assuming that the sld can't contain dots, and that the tld can have at
// most one dot. Not 100% sure this is a valid assumption, but exceptions should be
// rare even if the assumption isn't valid.
return {
tld: split.slice(-tldSegCnt).join('.')
, sld: split.slice(-tldSegCnt - 1, -tldSegCnt).join('.')
, sub: split.slice(0, -tldSegCnt - 1).join('.')
};
});
}
return {
getSession
, iterateAllModules
, getTlds
, splitDomains
};
};

241
lib/goldilocks.js Normal file
View File

@ -0,0 +1,241 @@
'use strict';
module.exports.create = function (deps, config) {
console.log('config', config);
//var PromiseA = global.Promise;
var PromiseA = require('bluebird');
var listeners = require('./servers').listeners;
var modules;
function loadModules() {
modules = {};
modules.tls = require('./modules/tls').create(deps, config, netHandler);
modules.http = require('./modules/http.js').create(deps, config, modules.tls.middleware);
}
// opts = { servername, encrypted, peek, data, remoteAddress, remotePort }
function peek(conn, firstChunk, opts) {
if (!modules) {
loadModules();
}
opts.firstChunk = firstChunk;
conn.__opts = opts;
// TODO port/service-based routing can do here
// TLS byte 1 is handshake and byte 6 is client hello
if (0x16 === firstChunk[0]/* && 0x01 === firstChunk[5]*/) {
modules.tls.emit('connection', conn);
return;
}
// This doesn't work with TLS, but now that we know this isn't a TLS connection we can
// unshift the first chunk back onto the connection for future use. The unshift should
// happen after any listeners are attached to it but before any new data comes in.
if (!opts.hyperPeek) {
process.nextTick(function () {
conn.unshift(firstChunk);
});
}
// Connection is not TLS, check for HTTP next.
if (firstChunk[0] > 32 && firstChunk[0] < 127) {
var firstStr = firstChunk.toString();
if (/HTTP\//i.test(firstStr)) {
modules.http.emit('connection', conn);
return;
}
}
console.warn('failed to identify protocol from first chunk', firstChunk);
conn.destroy();
}
function netHandler(conn, opts) {
function getProp(name) {
return opts[name] || opts['_'+name] || conn[name] || conn['_'+name];
}
opts = opts || {};
var logName = getProp('remoteAddress') + ':' + getProp('remotePort') + ' -> ' +
getProp('localAddress') + ':' + getProp('localPort');
console.log('[netHandler]', logName, 'encrypted: '+opts.encrypted);
var start = Date.now();
conn.on('timeout', function () {
console.log('[netHandler]', logName, 'connection timed out', (Date.now()-start)/1000);
});
conn.on('end', function () {
console.log('[netHandler]', logName, 'connection ended', (Date.now()-start)/1000);
});
conn.on('close', function () {
console.log('[netHandler]', logName, 'connection closed', (Date.now()-start)/1000);
});
// XXX PEEK COMMENT XXX
// TODO we can have our cake and eat it too
// we can skip the need to wrap the TLS connection twice
// because we've already peeked at the data,
// but this needs to be handled better before we enable that
// (because it creates new edge cases)
if (opts.hyperPeek) {
console.log('hyperpeek');
peek(conn, opts.firstChunk, opts);
return;
}
function onError(err) {
console.error('[error] socket errored peeking -', err);
conn.destroy();
}
conn.once('error', onError);
conn.once('data', function (chunk) {
conn.removeListener('error', onError);
peek(conn, chunk, opts);
});
}
function dnsListener(port, msg) {
if (!Array.isArray(config.udp.modules)) {
return;
}
var socket = require('dgram').createSocket('udp4');
config.udp.modules.forEach(function (mod) {
if (mod.type !== 'forward') {
console.warn('found bad DNS module', mod);
return;
}
if (mod.ports.indexOf(port) < 0) {
return;
}
var dest = require('./domain-utils').separatePort(mod.address || '');
dest.port = dest.port || mod.port;
dest.host = dest.host || mod.host || 'localhost';
socket.send(msg, dest.port, dest.host);
});
}
function createTcpForwarder(mod) {
var dest = require('./domain-utils').separatePort(mod.address || '');
dest.port = dest.port || mod.port;
dest.host = dest.host || mod.host || 'localhost';
return function (conn) {
var newConnOpts = {};
['remote', 'local'].forEach(function (end) {
['Family', 'Address', 'Port'].forEach(function (name) {
newConnOpts['_'+end+name] = conn[end+name];
});
});
deps.proxy(conn, Object.assign(newConnOpts, dest));
};
}
deps.tunnel = deps.tunnel || {};
deps.tunnel.net = {
createConnection: function (opts, cb) {
console.log('[gl.tunnel] creating connection');
// here "reader" means the socket that looks like the connection being accepted
// here "writer" means the remote-looking part of the socket that driving the connection
var writer;
var wrapOpts = {};
function usePair(err, reader) {
if (err) {
process.nextTick(function () {
writer.emit('error', err);
});
return;
}
// this has the normal net/tcp stuff plus our custom stuff
// opts = { address, port,
// hostname, servername, tls, encrypted, data, localAddress, localPort, remoteAddress, remotePort, remoteFamily }
Object.keys(opts).forEach(function (key) {
wrapOpts[key] = opts[key];
try {
reader[key] = opts[key];
} catch(e) {
// can't set real socket getters, like remoteAddr
}
});
// A few more extra specialty options
wrapOpts.localAddress = wrapOpts.localAddress || '127.0.0.2'; // TODO use the tunnel's external address
wrapOpts.localPort = wrapOpts.localPort || 'tunnel-0';
try {
reader._remoteAddress = wrapOpts.remoteAddress;
reader._remotePort = wrapOpts.remotePort;
reader._remoteFamily = wrapOpts.remoteFamily;
reader._localAddress = wrapOpts.localAddress;
reader._localPort = wrapOpts.localPort;
reader._localFamily = wrapOpts.localFamily;
} catch(e) {
}
netHandler(reader, wrapOpts);
process.nextTick(function () {
// this cb will cause the stream to emit its (actually) first data event
// (even though it already gave a peek into that first data chunk)
console.log('[tunnel] callback, data should begin to flow');
cb();
});
}
wrapOpts.firstChunk = opts.data;
wrapOpts.hyperPeek = !!opts.data;
// We used to use `stream-pair` for non-tls connections, but there are places
// that require properties/functions to be present on the socket that aren't
// present on a JSStream so it caused problems.
writer = require('socket-pair').create(usePair);
return writer;
}
};
deps.tunnelClients = require('./tunnel-client-manager').create(deps, config);
deps.tunnelServer = require('./tunnel-server-manager').create(deps, config);
var listenPromises = [];
var tcpPortMap = {};
config.tcp.bind.filter(Number).forEach(function (port) {
tcpPortMap[port] = true;
});
(config.tcp.modules || []).forEach(function (mod) {
if (mod.type === 'forward') {
var forwarder = createTcpForwarder(mod);
mod.ports.forEach(function (port) {
if (!tcpPortMap[port]) {
console.log("forwarding port", port, "that wasn't specified in bind");
} else {
delete tcpPortMap[port];
}
listenPromises.push(listeners.tcp.add(port, forwarder));
});
}
else {
console.warn('unknown TCP module specified', mod);
}
});
var portList = Object.keys(tcpPortMap).map(Number).sort();
portList.forEach(function (port) {
listenPromises.push(listeners.tcp.add(port, netHandler));
});
if (config.udp.bind) {
config.udp.bind.forEach(function (port) {
listenPromises.push(listeners.udp.add(port, dnsListener.bind(port)));
});
}
if (!config.mdns.disabled) {
require('./mdns').start(deps, config, portList[0]);
}
return PromiseA.all(listenPromises);
};

View File

@ -1,12 +1,14 @@
'use strict';
module.exports.create = function (deps, conf) {
var PromiseA = require('bluebird');
var request = PromiseA.promisify(require('request'));
var pending = {};
async function _checkPublicAddr(host) {
var result = await deps.request({
async function checkPublicAddr(host) {
var result = await request({
method: 'GET'
, url: deps.OAUTH3.url.normalize(host)+'/api/org.oauth3.tunnel/checkip'
, url: host+'/api/org.oauth3.tunnel/checkip'
, json: true
});
@ -17,15 +19,8 @@ module.exports.create = function (deps, conf) {
// Note that the error on the body will probably have a message that overwrites the default
throw Object.assign(new Error('error in check IP response'), result.body.error);
}
if (!result.body.address) {
throw new Error("public address resonse doesn't contain address: "+JSON.stringify(result.body));
}
return result.body.address;
}
async function checkPublicAddr(provider) {
var directives = await deps.OAUTH3.discover(provider);
return _checkPublicAddr(directives.api);
}
async function checkSinglePort(host, address, port) {
var crypto = require('crypto');
@ -35,35 +30,27 @@ module.exports.create = function (deps, conf) {
var reqObj = {
method: 'POST'
, url: deps.OAUTH3.url.normalize(host)+'/api/org.oauth3.tunnel/loopback'
, timeout: 20*1000
, url: host+'/api/org.oauth3.tunnel/loopback'
, json: {
address: address
, port: port
, token: token
, keyAuthorization: keyAuth
, iat: Date.now()
, timeout: 18*1000
}
};
var result;
try {
result = await deps.request(reqObj);
result = await request(reqObj);
} catch (err) {
delete pending[token];
if (conf.debug) {
console.log('error making loopback request for port ' + port + ' loopback', err.message);
}
return false;
throw err;
}
delete pending[token];
if (!result.body) {
if (conf.debug) {
console.log('No response body in loopback request for port '+port);
}
return false;
throw new Error('No response body in loopback request for port '+port);
}
// If the loopback requests don't go to us then there are all kinds of ways it could
// error, but none of them really provide much extra information so we don't do
@ -76,27 +63,23 @@ module.exports.create = function (deps, conf) {
async function loopback(provider) {
var directives = await deps.OAUTH3.discover(provider);
var address = await _checkPublicAddr(directives.api);
if (conf.debug) {
console.log('checking to see if', address, 'gets back to us');
}
var address = await checkPublicAddr(directives.api);
console.log('checking to see if', address, 'gets back to us');
var ports = require('../servers').listeners.tcp.list();
var values = await deps.PromiseA.all(ports.map(function (port) {
var ports = require('./servers').listeners.tcp.list();
var values = await PromiseA.all(ports.map(function (port) {
return checkSinglePort(directives.api, address, port);
}));
if (conf.debug && Object.keys(pending).length) {
if (conf.debug) {
console.log('remaining loopback tokens', pending);
}
return {
address: address
, ports: ports.reduce(function (obj, port, ind) {
obj[port] = values[ind];
return obj;
}, {})
};
var result = {error: null, address: address};
ports.forEach(function (port, ind) {
result[port] = values[ind];
});
return result;
}
loopback.checkPublicAddr = checkPublicAddr;

View File

@ -2,7 +2,6 @@
var PromiseA = require('bluebird');
var queryName = '_cloud._tcp.local';
var dnsSuite = require('dns-suite');
function createResponse(name, ownerIds, packet, ttl, mainPort) {
var rpacket = {
@ -86,19 +85,20 @@ function createResponse(name, ownerIds, packet, ttl, mainPort) {
});
});
return dnsSuite.DNSPacket.write(rpacket);
return require('dns-suite').DNSPacket.write(rpacket);
}
module.exports.create = function (deps, config) {
var socket;
module.exports.start = function (deps, config, mainPort) {
var socket = require('dgram').createSocket({ type: 'udp4', reuseAddr: true });
var dns = require('dns-suite');
var nextBroadcast = -1;
function handlePacket(message, rinfo) {
socket.on('message', function (message, rinfo) {
// console.log('Received %d bytes from %s:%d', message.length, rinfo.address, rinfo.port);
var packet;
try {
packet = dnsSuite.DNSPacket.parse(message);
packet = dns.DNSPacket.parse(message);
}
catch (er) {
// `dns-suite` actually errors on a lot of the packets floating around in our network,
@ -108,12 +108,16 @@ module.exports.create = function (deps, config) {
}
// Only respond to queries.
if (packet.header.qr !== 0) { return; }
if (packet.header.qr !== 0) {
return;
}
// Only respond if they were asking for cloud devices.
if (packet.question.length !== 1) { return; }
if (packet.question[0].name !== queryName) { return; }
if (packet.question[0].typeName !== 'PTR') { return; }
if (packet.question[0].className !== 'IN' ) { return; }
if (packet.question.length !== 1 || packet.question[0].name !== queryName) {
return;
}
if (packet.question[0].typeName !== 'PTR' || packet.question[0].className !== 'IN' ) {
return;
}
var proms = [
deps.storage.mdnsId.get()
@ -127,7 +131,7 @@ module.exports.create = function (deps, config) {
];
PromiseA.all(proms).then(function (results) {
var resp = createResponse(results[0], results[1], packet, config.mdns.ttl, deps.tcp.mainPort);
var resp = createResponse(results[0], results[1], packet, config.mdns.ttl, mainPort);
var now = Date.now();
if (now > nextBroadcast) {
socket.send(resp, config.mdns.port, config.mdns.broadcast);
@ -136,68 +140,18 @@ module.exports.create = function (deps, config) {
socket.send(resp, rinfo.port, rinfo.address);
}
});
}
});
function start() {
socket = require('dgram').createSocket({ type: 'udp4', reuseAddr: true });
socket.on('message', handlePacket);
socket.bind(config.mdns.port, function () {
var addr = this.address();
console.log('bound on UDP %s:%d for mDNS', addr.address, addr.port);
return new Promise(function (resolve, reject) {
socket.once('error', reject);
socket.bind(config.mdns.port, function () {
var addr = this.address();
console.log('bound on UDP %s:%d for mDNS', addr.address, addr.port);
socket.setBroadcast(true);
socket.addMembership(config.mdns.broadcast);
// This is supposed to be a local device discovery mechanism, so we shouldn't
// need to hop through any gateways. This helps with security by making it
// much more difficult for someone to use us as part of a DDoS attack by
// spoofing the UDP address a request came from.
socket.setTTL(1);
socket.removeListener('error', reject);
resolve();
});
});
}
function stop() {
return new Promise(function (resolve, reject) {
socket.once('error', reject);
socket.close(function () {
socket.removeListener('error', reject);
socket = null;
resolve();
});
});
}
function updateConf() {
var promise;
if (config.mdns.disabled) {
if (socket) {
promise = stop();
}
} else {
if (!socket) {
promise = start();
} else if (socket.address().port !== config.mdns.port) {
promise = stop().then(start);
} else {
// Can't check membership, so just add the current broadcast address to make sure
// it's set. If it's already set it will throw an exception (at least on linux).
try {
socket.addMembership(config.mdns.broadcast);
} catch (e) {}
promise = Promise.resolve();
}
}
}
updateConf();
return {
updateConf
};
socket.setBroadcast(true);
socket.addMembership(config.mdns.broadcast);
// This is supposed to be a local device discovery mechanism, so we shouldn't
// need to hop through any gateways. This helps with security by making it
// much more difficult for someone to use us as part of a DDoS attack by
// spoofing the UDP address a request came from.
socket.setTTL(1);
});
};

View File

@ -1,6 +1,6 @@
'use strict';
module.exports.create = function (deps, conf, tcpMods) {
module.exports.create = function (deps, conf, greenlockMiddleware) {
var PromiseA = require('bluebird');
var statAsync = PromiseA.promisify(require('fs').stat);
var domainMatches = require('../domain-utils').match;
@ -162,8 +162,8 @@ module.exports.create = function (deps, conf, tcpMods) {
return false;
}
if (deps.stunneld.isClientDomain(separatePort(headers.host).host)) {
deps.stunneld.handleClientConn(conn);
if (deps.tunnelServer.isClientDomain(separatePort(headers.host).host)) {
deps.tunnelServer.handleClientConn(conn);
process.nextTick(function () {
conn.unshift(opts.firstChunk);
conn.resume();
@ -172,7 +172,7 @@ module.exports.create = function (deps, conf, tcpMods) {
}
if (!acmeServer) {
acmeServer = require('http').createServer(tcpMods.tls.middleware);
acmeServer = require('http').createServer(greenlockMiddleware);
}
return emitConnection(acmeServer, conn, opts);
}
@ -181,7 +181,7 @@ module.exports.create = function (deps, conf, tcpMods) {
if (headers.url.indexOf('/.well-known/cloud-challenge/') !== 0) {
return false;
}
return emitConnection(deps.ddns.loopbackServer, conn, opts);
return emitConnection(deps.loopback.server, conn, opts);
}
var httpsRedirectServer;
@ -214,8 +214,8 @@ module.exports.create = function (deps, conf, tcpMods) {
return emitConnection(adminServer, conn, opts);
}
if (deps.stunneld.isAdminDomain(host)) {
deps.stunneld.handleAdminConn(conn);
if (deps.tunnelServer.isAdminDomain(host)) {
deps.tunnelServer.handleAdminConn(conn);
process.nextTick(function () {
conn.unshift(opts.firstChunk);
conn.resume();
@ -241,7 +241,7 @@ module.exports.create = function (deps, conf, tcpMods) {
res.statusCode = 502;
res.setHeader('Connection', 'close');
res.setHeader('Content-Type', 'text/html');
res.end(tcpMods.proxy.getRespBody(err, conf.debug));
res.end(require('../proxy-conn').getRespBody(err, conf.debug));
});
proxyServer = http.createServer(function (req, res) {
@ -292,7 +292,7 @@ module.exports.create = function (deps, conf, tcpMods) {
newConnOpts.remoteAddress = opts.address || conn.remoteAddress;
newConnOpts.remotePort = opts.port || conn.remotePort;
tcpMods.proxy(conn, newConnOpts, opts.firstChunk);
deps.proxy(conn, newConnOpts, opts.firstChunk);
}
function checkProxy(mod, conn, opts, headers) {
@ -357,78 +357,30 @@ module.exports.create = function (deps, conf, tcpMods) {
var staticServer;
var staticHandlers = {};
var indexHandlers = {};
function serveStatic(req, res) {
var rootDir = req.connection.rootDir;
var modOpts = req.connection.modOpts;
if (!staticHandlers[rootDir]) {
staticHandlers[rootDir] = require('express').static(rootDir, {
dotfiles: modOpts.dotfiles
, fallthrough: false
, redirect: modOpts.redirect
, index: modOpts.index
});
staticHandlers[rootDir] = require('express').static(rootDir, { fallthrough: false });
}
staticHandlers[rootDir](req, res, function (err) {
function doFinal() {
if (err) {
res.statusCode = err.statusCode;
} else {
res.statusCode = 404;
}
res.setHeader('Content-Type', 'text/html');
if (res.statusCode === 404) {
res.end('File Not Found');
} else {
res.end(require('http').STATUS_CODES[res.statusCode]);
}
if (err) {
res.statusCode = err.statusCode;
} else {
res.statusCode = 404;
}
res.setHeader('Content-Type', 'text/html');
var handlerHandle = rootDir
+ (modOpts.hidden||'')
+ (modOpts.icons||'')
+ (modOpts.stylesheet||'')
+ (modOpts.template||'')
+ (modOpts.view||'')
;
function pathMatchesUrl(pathname) {
if (req.url === pathname) {
return true;
}
if (0 === req.url.replace(/\/?$/, '/').indexOf(pathname.replace(/\/?$/, '/'))) {
return true;
}
if (res.statusCode === 404) {
res.end('File Not Found');
} else {
res.end(require('http').STATUS_CODES[res.statusCode]);
}
if (!modOpts.indexes || ('*' !== modOpts.indexes[0] && !modOpts.indexes.some(pathMatchesUrl))) {
doFinal();
return;
}
if (!indexHandlers[handlerHandle]) {
// https://www.npmjs.com/package/serve-index
indexHandlers[handlerHandle] = require('serve-index')(rootDir, {
hidden: modOpts.hidden
, icons: modOpts.icons
, stylesheet: modOpts.stylesheet
, template: modOpts.template
, view: modOpts.view
});
}
indexHandlers[handlerHandle](req, res, function (_err) {
err = _err || err;
doFinal();
});
});
}
function checkStatic(modOpts, conn, opts, headers) {
var rootDir = modOpts.root.replace(':hostname', separatePort(headers.host).host);
function checkStatic(mod, conn, opts, headers) {
var rootDir = mod.root.replace(':hostname', separatePort(headers.host).host);
return statAsync(rootDir)
.then(function (stats) {
if (!stats || !stats.isDirectory()) {
@ -439,7 +391,6 @@ module.exports.create = function (deps, conf, tcpMods) {
staticServer = require('http').createServer(serveStatic);
}
conn.rootDir = rootDir;
conn.modOpts = modOpts;
return emitConnection(staticServer, conn, opts);
})
.catch(function (err) {
@ -451,8 +402,6 @@ module.exports.create = function (deps, conf, tcpMods) {
;
}
// The function signature is as follows
// function module(moduleOptions, tcpConnection, connectionOptions, headers) { ... }
var moduleChecks = {
proxy: checkProxy
, redirect: checkRedirect

View File

@ -1,6 +1,6 @@
'use strict';
module.exports.create = function (deps, config, tcpMods) {
module.exports.create = function (deps, config, netHandler) {
var path = require('path');
var tls = require('tls');
var parseSni = require('sni');
@ -50,7 +50,10 @@ module.exports.create = function (deps, config, tcpMods) {
return;
}
writer.write(opts.firstChunk);
process.nextTick(function () {
socket.unshift(opts.firstChunk);
});
socket.pipe(writer);
writer.pipe(socket);
@ -86,7 +89,8 @@ module.exports.create = function (deps, config, tcpMods) {
, challenges: {
'http-01': require('le-challenge-fs').create({ debug: config.debug })
, 'tls-sni-01': require('le-challenge-sni').create({ debug: config.debug })
, 'dns-01': deps.ddns.challenge
// TODO dns-01
//, 'dns-01': require('le-challenge-ddns').create({ debug: config.debug })
}
, challengeType: 'http-01'
@ -207,7 +211,7 @@ module.exports.create = function (deps, config, tcpMods) {
var terminateServer = tls.createServer(terminatorOpts, function (socket) {
console.log('(post-terminated) tls connection, addr:', extractSocketProp(socket, 'remoteAddress'));
tcpMods.tcpHandler(socket, {
netHandler(socket, {
servername: socket.servername
, encrypted: true
// remoteAddress... ugh... https://github.com/nodejs/node/issues/8854
@ -231,7 +235,7 @@ module.exports.create = function (deps, config, tcpMods) {
newConnOpts.remoteAddress = opts.address || extractSocketProp(socket, 'remoteAddress');
newConnOpts.remotePort = opts.port || extractSocketProp(socket, 'remotePort');
tcpMods.proxy(socket, newConnOpts, opts.firstChunk, function () {
deps.proxy(socket, newConnOpts, opts.firstChunk, function () {
// This function is called in the event of a connection error and should decrypt
// the socket so the proxy module can send a 502 HTTP response.
var tlsOpts = localhostCerts.mergeTlsOptions('localhost.daplie.me', {isServer: true});
@ -290,8 +294,8 @@ module.exports.create = function (deps, config, tcpMods) {
return;
}
if (deps.stunneld.isClientDomain(opts.servername)) {
deps.stunneld.handleClientConn(socket);
if (deps.tunnelServer.isClientDomain(opts.servername)) {
deps.tunnelServer.handleClientConn(socket);
if (!opts.hyperPeek) {
process.nextTick(function () {
socket.unshift(opts.firstChunk);

View File

@ -32,7 +32,7 @@ module.exports.getRespBody = getRespBody;
module.exports.sendBadGateway = sendBadGateway;
module.exports.create = function (deps, config) {
function proxy(conn, newConnOpts, firstChunk, decrypt) {
return function proxy(conn, newConnOpts, firstChunk, decrypt) {
var connected = false;
newConnOpts.allowHalfOpen = true;
var newConn = deps.net.createConnection(newConnOpts, function () {
@ -73,9 +73,5 @@ module.exports.create = function (deps, config) {
newConn.on('close', function () {
conn.destroy();
});
}
proxy.getRespBody = getRespBody;
proxy.sendBadGateway = sendBadGateway;
return proxy;
};
};

View File

@ -10,16 +10,20 @@ module.exports.addTcpListener = function (port, handler) {
if (stat) {
if (stat._closing) {
stat.server.destroy();
} else {
// We're already listening on the port, so we only have 2 options. We can either
// replace the handler or reject with an error. (Though neither is really needed
// if the handlers are the same). Until there is reason to do otherwise we are
// opting for the replacement.
module.exports.destroyTcpListener(port);
}
else if (handler !== stat.handler) {
// we'll replace the current listener
stat.handler = handler;
resolve();
return;
}
else {
// this exact listener is already open
resolve();
return;
}
}
var enableDestroy = require('server-destroy');
@ -30,7 +34,7 @@ module.exports.addTcpListener = function (port, handler) {
stat = serversMap[port] = {
server: server
, handler: handler
, _closing: false
, _closing: null
};
// Add .destroy so we can close all open connections. Better if added before listen
@ -62,24 +66,14 @@ module.exports.addTcpListener = function (port, handler) {
});
});
};
module.exports.closeTcpListener = function (port, timeout) {
module.exports.closeTcpListener = function (port) {
return new PromiseA(function (resolve) {
var stat = serversMap[port];
if (!stat) {
resolve();
return;
}
stat._closing = true;
var timeoutId;
if (timeout) {
timeoutId = setTimeout(() => stat.server.destroy(), timeout);
}
stat.server.once('close', function () {
clearTimeout(timeoutId);
resolve();
});
stat.server.once('close', resolve);
stat.server.close();
});
};
@ -90,9 +84,7 @@ module.exports.destroyTcpListener = function (port) {
}
};
module.exports.listTcpListeners = function () {
return Object.keys(serversMap).map(Number).filter(function (port) {
return port && !serversMap[port]._closing;
});
return Object.keys(serversMap).map(Number).filter(Boolean);
};

View File

@ -63,29 +63,15 @@ module.exports.create = function (deps, config) {
});
}
var configEnabled = false;
function updateConf() {
var wanted = config.socks5 && config.socks5.enabled;
if (configEnabled && !wanted) {
stop().catch(function (err) {
console.error('failed to stop socks5 proxy on config change', err);
});
configEnabled = false;
}
if (wanted && !configEnabled) {
start(config.socks5.port).catch(function (err) {
console.error('failed to start Socks5 proxy', err);
});
configEnabled = true;
}
if (config.socks5 && config.socks5.enabled) {
start(config.socks5.port).catch(function (err) {
console.error('failed to start Socks5 proxy', err);
});
}
process.nextTick(updateConf);
return {
curState
, start
, stop
, updateConf
curState: curState
, start: start
, stop: stop
};
};

View File

@ -3,8 +3,6 @@
var PromiseA = require('bluebird');
var path = require('path');
var fs = PromiseA.promisifyAll(require('fs'));
var jwt = require('jsonwebtoken');
var crypto = require('crypto');
module.exports.create = function (deps, conf) {
var hrIds = require('human-readable-ids').humanReadableIds;
@ -95,104 +93,6 @@ module.exports.create = function (deps, conf) {
}
}
var userTokens = {
_filename: 'user-tokens.json'
, _cache: {}
, _convertToken: function convertToken(id, token) {
// convert the token into something that looks more like what OAuth3 uses internally
// as sessions so we can use it with OAuth3. We don't use OAuth3's internal session
// storage because it effectively only supports storing tokens based on provider URI.
// We also use the token as the `access_token` instead of `refresh_token` because the
// refresh functionality is closely tied to the storage.
var decoded = jwt.decode(token);
if (!decoded) {
return null;
}
return {
id: id
, access_token: token
, token: decoded
, provider_uri: decoded.iss || decoded.issuer || decoded.provider_uri
, client_uri: decoded.azp
, scope: decoded.scp || decoded.scope || decoded.grants
};
}
, all: function allUserTokens() {
var self = this;
if (self._cacheComplete) {
return deps.PromiseA.resolve(Object.values(self._cache));
}
return read(self._filename).then(function (tokens) {
// We will read every single token into our cache, so it will be complete once we finish
// creating the result (it's set out of order so we can directly return the result).
self._cacheComplete = true;
return Object.keys(tokens).map(function (id) {
self._cache[id] = self._convertToken(id, tokens[id]);
return self._cache[id];
});
});
}
, get: function getUserToken(id) {
var self = this;
if (self._cache.hasOwnProperty(id) || self._cacheComplete) {
return deps.PromiseA.resolve(self._cache[id] || null);
}
return read(self._filename).then(function (tokens) {
self._cache[id] = self._convertToken(id, tokens[id]);
return self._cache[id];
});
}
, save: function saveUserToken(newToken) {
var self = this;
return read(self._filename).then(function (tokens) {
var rawToken;
if (typeof newToken === 'string') {
rawToken = newToken;
} else {
rawToken = newToken.refresh_token || newToken.access_token;
}
if (typeof rawToken !== 'string') {
throw new Error('cannot save invalid session: missing refresh_token and access_token');
}
var decoded = jwt.decode(rawToken);
var idHash = crypto.createHash('sha256');
idHash.update(decoded.sub || decoded.ppid || decoded.appScopedId || '');
idHash.update(decoded.iss || decoded.issuer || '');
idHash.update(decoded.aud || decoded.audience || '');
var scope = decoded.scope || decoded.scp || decoded.grants || '';
idHash.update(scope.split(/[,\s]+/mg).sort().join(','));
var id = idHash.digest('hex');
tokens[id] = rawToken;
return write(self._filename, tokens).then(function () {
// Delete the current cache so that if this is an update it will refresh
// the cache once we read the ID.
delete self._cache[id];
return self.get(id);
});
});
}
, remove: function removeUserToken(id) {
var self = this;
return read(self._filename).then(function (tokens) {
var present = delete tokens[id];
if (!present) {
return present;
}
return write(self._filename, tokens).then(function () {
delete self._cache[id];
return true;
});
});
}
};
var mdnsId = {
_filename: 'mdns-id'
, get: function () {
@ -219,7 +119,6 @@ module.exports.create = function (deps, conf) {
owners: owners
, config: config
, updateConf: updateConf
, tokens: userTokens
, mdnsId: mdnsId
};
};

View File

@ -1,242 +0,0 @@
'use strict';
module.exports.create = function (deps, config) {
console.log('config', config);
var listeners = require('../servers').listeners.tcp;
var domainUtils = require('../domain-utils');
var modules;
var addrProperties = [
'remoteAddress'
, 'remotePort'
, 'remoteFamily'
, 'localAddress'
, 'localPort'
, 'localFamily'
];
function nameMatchesDomains(name, domainList) {
return domainList.some(function (pattern) {
return domainUtils.match(pattern, name);
});
}
function proxy(mod, conn, opts) {
// First thing we need to add to the connection options is where to proxy the connection to
var newConnOpts = domainUtils.separatePort(mod.address || '');
newConnOpts.port = newConnOpts.port || mod.port;
newConnOpts.host = newConnOpts.host || mod.host || 'localhost';
// Then we add all of the connection address information. We need to prefix all of the
// properties with '_' so we can provide the information to any connection `createConnection`
// implementation but not have the default implementation try to bind the same local port.
addrProperties.forEach(function (name) {
newConnOpts['_' + name] = opts[name] || opts['_'+name] || conn[name] || conn['_'+name];
});
modules.proxy(conn, newConnOpts);
return true;
}
function checkTcpProxy(conn, opts) {
var proxied = false;
// TCP Proxying (ie routing based on domain name [vs local port]) only works for
// TLS wrapped connections, so if the opts don't give us a servername or don't tell us
// this is the decrypted side of a TLS connection we can't handle it here.
if (!opts.servername || !opts.encrypted) { return proxied; }
proxied = config.domains.some(function (dom) {
if (!dom.modules || !Array.isArray(dom.modules.tcp)) { return false; }
if (!nameMatchesDomains(opts.servername, dom.names)) { return false; }
return dom.modules.tcp.some(function (mod) {
if (mod.type !== 'proxy') { return false; }
return proxy(mod, conn, opts);
});
});
proxied = proxied || config.tcp.modules.some(function (mod) {
if (mod.type !== 'proxy') { return false; }
if (!nameMatchesDomains(opts.servername, mod.domains)) { return false; }
return proxy(mod, conn, opts);
});
return proxied;
}
function checkTcpForward(conn, opts) {
// TCP forwarding (ie routing connections based on local port) requires the local port
if (!conn.localPort) { return false; }
return config.tcp.modules.some(function (mod) {
if (mod.type !== 'forward') { return false; }
if (mod.ports.indexOf(conn.localPort) < 0) { return false; }
return proxy(mod, conn, opts);
});
}
// opts = { servername, encrypted, peek, data, remoteAddress, remotePort }
function peek(conn, firstChunk, opts) {
opts.firstChunk = firstChunk;
conn.__opts = opts;
// TODO port/service-based routing can do here
// TLS byte 1 is handshake and byte 6 is client hello
if (0x16 === firstChunk[0]/* && 0x01 === firstChunk[5]*/) {
modules.tls.emit('connection', conn);
return;
}
// This doesn't work with TLS, but now that we know this isn't a TLS connection we can
// unshift the first chunk back onto the connection for future use. The unshift should
// happen after any listeners are attached to it but before any new data comes in.
if (!opts.hyperPeek) {
process.nextTick(function () {
conn.unshift(firstChunk);
});
}
// Connection is not TLS, check for HTTP next.
if (firstChunk[0] > 32 && firstChunk[0] < 127) {
var firstStr = firstChunk.toString();
if (/HTTP\//i.test(firstStr)) {
modules.http.emit('connection', conn);
return;
}
}
console.warn('failed to identify protocol from first chunk', firstChunk);
conn.destroy();
}
function tcpHandler(conn, opts) {
function getProp(name) {
return opts[name] || opts['_'+name] || conn[name] || conn['_'+name];
}
opts = opts || {};
var logName = getProp('remoteAddress') + ':' + getProp('remotePort') + ' -> ' +
getProp('localAddress') + ':' + getProp('localPort');
console.log('[tcpHandler]', logName, 'connection started - encrypted: ' + (opts.encrypted || false));
var start = Date.now();
conn.on('timeout', function () {
console.log('[tcpHandler]', logName, 'connection timed out', (Date.now()-start)/1000);
});
conn.on('end', function () {
console.log('[tcpHandler]', logName, 'connection ended', (Date.now()-start)/1000);
});
conn.on('close', function () {
console.log('[tcpHandler]', logName, 'connection closed', (Date.now()-start)/1000);
});
if (checkTcpForward(conn, opts)) { return; }
if (checkTcpProxy(conn, opts)) { return; }
// XXX PEEK COMMENT XXX
// TODO we can have our cake and eat it too
// we can skip the need to wrap the TLS connection twice
// because we've already peeked at the data,
// but this needs to be handled better before we enable that
// (because it creates new edge cases)
if (opts.hyperPeek) {
console.log('hyperpeek');
peek(conn, opts.firstChunk, opts);
return;
}
function onError(err) {
console.error('[error] socket errored peeking -', err);
conn.destroy();
}
conn.once('error', onError);
conn.once('data', function (chunk) {
conn.removeListener('error', onError);
peek(conn, chunk, opts);
});
}
process.nextTick(function () {
modules = {};
modules.tcpHandler = tcpHandler;
modules.proxy = require('./proxy-conn').create(deps, config);
modules.tls = require('./tls').create(deps, config, modules);
modules.http = require('./http').create(deps, config, modules);
});
function updateListeners() {
var current = listeners.list();
var wanted = config.tcp.bind;
if (!Array.isArray(wanted)) { wanted = []; }
wanted = wanted.map(Number).filter((port) => port > 0 && port < 65356);
var closeProms = current.filter(function (port) {
return wanted.indexOf(port) < 0;
}).map(function (port) {
return listeners.close(port, 1000);
});
// We don't really need to filter here since listening on the same port with the
// same handler function twice is basically a no-op.
var openProms = wanted.map(function (port) {
return listeners.add(port, tcpHandler);
});
return Promise.all(closeProms.concat(openProms));
}
var mainPort;
function updateConf() {
updateListeners().catch(function (err) {
console.error('Error updating TCP listeners to match bind configuration');
console.error(err);
});
var unforwarded = {};
config.tcp.bind.forEach(function (port) {
unforwarded[port] = true;
});
config.tcp.modules.forEach(function (mod) {
if (['forward', 'proxy'].indexOf(mod.type) < 0) {
console.warn('unknown TCP module type specified', JSON.stringify(mod));
}
if (mod.type !== 'forward') { return; }
mod.ports.forEach(function (port) {
if (!unforwarded[port]) {
console.warn('trying to forward TCP port ' + port + ' multiple times or it is unbound');
} else {
delete unforwarded[port];
}
});
});
// Not really sure what we can reasonably do to prevent this. At least not without making
// our configuration validation more complicated.
if (!Object.keys(unforwarded).length) {
console.warn('no bound TCP ports are not being forwarded, admin interface will be inaccessible');
}
// If we are listening on port 443 make that the main port we respond to mDNS queries with
// otherwise choose the lowest number port we are bound to but not forwarding.
if (unforwarded['443']) {
mainPort = 443;
} else {
mainPort = Object.keys(unforwarded).map(Number).sort((a, b) => a - b)[0];
}
}
updateConf();
var result = {
updateConf
, handler: tcpHandler
};
Object.defineProperty(result, 'mainPort', {enumerable: true, get: () => mainPort});
return result;
};

View File

@ -0,0 +1,237 @@
'use strict';
module.exports.create = function (deps, config) {
var PromiseA = require('bluebird');
var fs = PromiseA.promisifyAll(require('fs'));
var stunnel = require('stunnel');
var activeTunnels = {};
var path = require('path');
var tokensPath = path.join(__dirname, '..', 'var', 'tokens.json');
var storage = {
_read: function () {
var tokens;
try {
tokens = require(tokensPath);
} catch (err) {
tokens = {};
}
return tokens;
}
, _write: function (tokens) {
return fs.mkdirAsync(path.dirname(tokensPath)).catch(function (err) {
if (err.code !== 'EEXIST') {
console.error('failed to mkdir', path.dirname(tokensPath), err.toString());
}
}).then(function () {
return fs.writeFileAsync(tokensPath, JSON.stringify(tokens), 'utf8');
});
}
, _makeKey: function (token) {
// We use a stripped down version of the token contents so that if the token is
// re-issued the nonce and the iat and any other less important things are different
// we don't save essentially duplicate tokens multiple times.
var parsed = JSON.parse((new Buffer(token.split('.')[1], 'base64')).toString());
var stripped = {};
['aud', 'iss', 'domains'].forEach(function (key) {
if (parsed[key]) {
stripped[key] = parsed[key];
}
});
stripped.domains.sort();
var hash = require('crypto').createHash('sha256');
return hash.update(JSON.stringify(stripped)).digest('hex');
}
, all: function () {
var tokens = storage._read();
return PromiseA.resolve(Object.keys(tokens).map(function (key) {
return tokens[key];
}));
}
, save: function (token) {
return PromiseA.resolve().then(function () {
var curTokens = storage._read();
curTokens[storage._makeKey(token.jwt)] = token;
return storage._write(curTokens);
});
}
, del: function (token) {
return PromiseA.resolve().then(function () {
var curTokens = storage._read();
delete curTokens[storage._makeKey(token.jwt)];
return storage._write(curTokens);
});
}
};
function acquireToken(session) {
var OAUTH3 = deps.OAUTH3;
// session seems to be changed by the API call for some reason, so save the
// owner before that happens.
var owner = session.id;
// The OAUTH3 library stores some things on the root session object that we usually
// just leave inside the token, but we need to pull those out before we use it here
session.provider_uri = session.provider_uri || session.token.provider_uri || session.token.iss;
session.client_uri = session.client_uri || session.token.azp;
session.scope = session.scope || session.token.scp;
console.log('asking for tunnel token from', session.token.aud);
return OAUTH3.discover(session.token.aud).then(function (directives) {
var opts = {
api: 'tunnel.token'
, session: session
, data: {
// filter to all domains that are on this device
//domains: Object.keys(domainsMap)
device: {
hostname: config.device.hostname
, id: config.device.uid || config.device.id
}
}
};
return OAUTH3.api(directives.api, opts).then(function (result) {
console.log('got a token from the tunnel server?');
result.owner = owner;
return result;
});
});
}
function addToken(data) {
if (!data.jwt) {
return PromiseA.reject(new Error("missing 'jwt' from tunnel data"));
}
if (!data.tunnelUrl) {
var decoded;
try {
decoded = JSON.parse(new Buffer(data.jwt.split('.')[1], 'base64').toString('ascii'));
} catch (err) {
console.warn('invalid web token given to tunnel manager', err);
return PromiseA.reject(err);
}
if (!decoded.aud) {
console.warn('tunnel manager given token with no tunnelUrl or audience');
var err = new Error('missing tunnelUrl and audience');
return PromiseA.reject(err);
}
data.tunnelUrl = 'wss://' + decoded.aud + '/';
}
if (!activeTunnels[data.tunnelUrl]) {
console.log('creating new tunnel client for', data.tunnelUrl);
// We create the tunnel without an initial token so we can append the token and
// get the promise that should tell us more about if it worked or not.
activeTunnels[data.tunnelUrl] = stunnel.connect({
stunneld: data.tunnelUrl
, net: deps.tunnel.net
// NOTE: the ports here aren't that important since we are providing a custom
// `net.createConnection` that doesn't actually use the port. What is important
// is that any services we are interested in are listed in this object and have
// a '*' sub-property.
, services: {
https: { '*': 443 }
, http: { '*': 80 }
, smtp: { '*': 25 }
, smtps: { '*': 587 /*also 465/starttls*/ }
, ssh: { '*': 22 }
}
});
}
console.log('appending token to tunnel at', data.tunnelUrl);
return activeTunnels[data.tunnelUrl].append(data.jwt);
}
function removeToken(data) {
if (!data.tunnelUrl) {
var decoded;
try {
decoded = JSON.parse(new Buffer(data.jwt.split('.')[1], 'base64').toString('ascii'));
} catch (err) {
console.warn('invalid web token given to tunnel manager', err);
return PromiseA.reject(err);
}
if (!decoded.aud) {
console.warn('tunnel manager given token with no tunnelUrl or audience');
var err = new Error('missing tunnelUrl and audience');
return PromiseA.reject(err);
}
data.tunnelUrl = 'wss://' + decoded.aud + '/';
}
// Not sure if we actually want to return an error that the token didn't even belong to a
// server that existed, but since it never existed we can consider it as "removed".
if (!activeTunnels[data.tunnelUrl]) {
return PromiseA.resolve();
}
console.log('removing token from tunnel at', data.tunnelUrl);
return activeTunnels[data.tunnelUrl].clear(data.jwt);
}
if (config.tunnel) {
var confTokens = config.tunnel;
if (typeof confTokens === 'string') {
confTokens = confTokens.split(',');
}
confTokens.forEach(function (jwt) {
if (typeof jwt === 'object') {
jwt.owner = 'config';
addToken(jwt);
} else {
addToken({ jwt: jwt, owner: 'config' });
}
});
}
storage.all().then(function (stored) {
stored.forEach(function (result) {
addToken(result);
});
});
return {
start: function (session) {
return acquireToken(session).then(function (token) {
return addToken(token).then(function () {
return storage.save(token);
});
});
}
, add: function (data) {
return addToken(data).then(function () {
return storage.save(data);
});
}
, remove: function (data) {
return storage.del(data.jwt).then(function () {
return removeToken(data);
});
}
, get: function (owner) {
return storage.all().then(function (tokens) {
var result = {};
tokens.forEach(function (data) {
if (!result[data.owner]) {
result[data.owner] = {};
}
if (!result[data.owner][data.tunnelUrl]) {
result[data.owner][data.tunnelUrl] = [];
}
data.decoded = JSON.parse(new Buffer(data.jwt.split('.')[0], 'base64'));
result[data.owner][data.tunnelUrl].push(data);
});
if (owner) {
return result[owner] || {};
}
return result;
});
}
};
};

View File

@ -1,131 +1,61 @@
'use strict';
function httpsTunnel(servername, conn) {
console.error('tunnel server received encrypted connection to', servername);
conn.end();
}
function handleHttp(servername, conn) {
console.error('tunnel server received un-encrypted connection to', servername);
conn.end([
'HTTP/1.1 404 Not Found'
, 'Date: ' + (new Date()).toUTCString()
, 'Connection: close'
, 'Content-Type: text/html'
, 'Content-Length: 9'
, ''
, 'Not Found'
].join('\r\n'));
}
function rejectNonWebsocket(req, res) {
// status code 426 = Upgrade Required
res.statusCode = 426;
res.setHeader('Content-Type', 'application/json');
res.send({error: { message: 'Only websockets accepted for tunnel server' }});
}
module.exports.create = function (deps, config) {
if (!config.tunnelServer || !Array.isArray(config.tunnelServer.servernames) || !config.tunnelServer.secret) {
return {
isAdminDomain: function () { return false; }
, isClientDomain: function () { return false; }
};
}
var defaultConfig = {
servernames: []
, secret: null
};
var tunnelFuncs = {
// These functions should not be called because connections to the admin domains
var tunnelOpts = Object.assign({}, config.tunnelServer);
// This function should not be called because connections to the admin domains
// should already be decrypted, and connections to non-client domains should never
// be given to us in the first place.
httpsTunnel: httpsTunnel
, httpsInvalid: httpsTunnel
// These function should not be called because ACME challenges should be handled
tunnelOpts.httpsTunnel = function (servername, conn) {
console.error('tunnel server received encrypted connection to', servername);
conn.end();
};
tunnelOpts.httpsInvalid = tunnelOpts.httpsTunnel;
// This function should not be called because ACME challenges should be handled
// before admin domain connections are given to us, and the only non-encrypted
// client connections that should be given to us are ACME challenges.
, handleHttp: handleHttp
, handleInsecureHttp: handleHttp
};
tunnelOpts.handleHttp = function (servername, conn) {
console.error('tunnel server received un-encrypted connection to', servername);
conn.end([
'HTTP/1.1 404 Not Found'
, 'Date: ' + (new Date()).toUTCString()
, 'Connection: close'
, 'Content-Type: text/html'
, 'Content-Length: 9'
, ''
, 'Not Found'
].join('\r\n'));
};
tunnelOpts.handleInsecureHttp = tunnelOpts.handleHttp;
module.exports.create = function (deps, config) {
var equal = require('deep-equal');
var enableDestroy = require('server-destroy');
var currentOpts = Object.assign({}, defaultConfig);
var tunnelServer = require('stunneld').create(tunnelOpts);
var httpServer, wsServer, stunneld;
function start() {
if (httpServer || wsServer || stunneld) {
throw new Error('trying to start already started tunnel server');
}
httpServer = require('http').createServer(rejectNonWebsocket);
enableDestroy(httpServer);
wsServer = new (require('ws').Server)({ server: httpServer });
var tunnelOpts = Object.assign({}, tunnelFuncs, currentOpts);
stunneld = require('stunneld').create(tunnelOpts);
wsServer.on('connection', stunneld.ws);
}
function stop() {
if (!httpServer || !wsServer || !stunneld) {
throw new Error('trying to stop unstarted tunnel server (or it got into semi-initialized state');
}
wsServer.close();
wsServer = null;
httpServer.destroy();
httpServer = null;
// Nothing to close here, just need to set it to null to allow it to be garbage-collected.
stunneld = null;
}
function updateConf() {
var newOpts = Object.assign({}, defaultConfig, config.tunnelServer);
if (!Array.isArray(newOpts.servernames)) {
newOpts.servernames = [];
}
var trimmedOpts = {
servernames: newOpts.servernames.slice().sort()
, secret: newOpts.secret
};
if (equal(trimmedOpts, currentOpts)) {
return;
}
currentOpts = trimmedOpts;
// Stop what's currently running, then if we are still supposed to be running then we
// can start it again with the updated options. It might be possible to make use of
// the existing http and ws servers when the config changes, but I'm not sure what
// state the actions needed to close all existing connections would put them in.
if (httpServer || wsServer || stunneld) {
stop();
}
if (currentOpts.servernames.length && currentOpts.secret) {
start();
}
}
process.nextTick(updateConf);
var httpServer = require('http').createServer(function (req, res) {
// status code 426 = Upgrade Required
res.statusCode = 426;
res.setHeader('Content-Type', 'application/json');
res.end(JSON.stringify({error: {
message: 'Only websockets accepted for tunnel server'
}}));
});
var wsServer = new (require('ws').Server)({ server: httpServer });
wsServer.on('connection', tunnelServer.ws);
return {
isAdminDomain: function (domain) {
return currentOpts.servernames.indexOf(domain) !== -1;
return config.tunnelServer.servernames.indexOf(domain) !== -1;
}
, handleAdminConn: function (conn) {
if (!httpServer) {
console.error(new Error('handleAdminConn called with no active tunnel server'));
conn.end();
} else {
return httpServer.emit('connection', conn);
}
httpServer.emit('connection', conn);
}
, isClientDomain: function (domain) {
if (!stunneld) { return false; }
return stunneld.isClientDomain(domain);
}
, handleClientConn: function (conn) {
if (!stunneld) {
console.error(new Error('handleClientConn called with no active tunnel server'));
conn.end();
} else {
return stunneld.tcp(conn);
}
}
, updateConf
, isClientDomain: tunnelServer.isClientDomain
, handleClientConn: tunnelServer.tcp
};
};

View File

@ -1,57 +0,0 @@
'use strict';
module.exports.create = function (deps, config) {
var listeners = require('./servers').listeners.udp;
function packetHandler(port, msg) {
if (!Array.isArray(config.udp.modules)) {
return;
}
var socket = require('dgram').createSocket('udp4');
config.udp.modules.forEach(function (mod) {
if (mod.type !== 'forward') {
// To avoid logging bad modules every time we get a UDP packet we assign a warned
// property to the module (non-enumerable so it won't be saved to the config or
// show up in the API).
if (!mod.warned) {
console.warn('found bad DNS module', mod);
Object.defineProperty(mod, 'warned', {value: true, enumerable: false});
}
return;
}
if (mod.ports.indexOf(port) < 0) {
return;
}
var dest = require('./domain-utils').separatePort(mod.address || '');
dest.port = dest.port || mod.port;
dest.host = dest.host || mod.host || 'localhost';
socket.send(msg, dest.port, dest.host);
});
}
function updateListeners() {
var current = listeners.list();
var wanted = config.udp.bind;
if (!Array.isArray(wanted)) { wanted = []; }
wanted = wanted.map(Number).filter((port) => port > 0 && port < 65356);
current.forEach(function (port) {
if (wanted.indexOf(port) < 0) {
listeners.close(port);
}
});
wanted.forEach(function (port) {
if (current.indexOf(port) < 0) {
listeners.add(port, packetHandler.bind(port));
}
});
}
updateListeners();
return {
updateConf: updateListeners
};
};

View File

@ -48,15 +48,14 @@ function create(conf) {
modules = {
storage: require('./storage').create(deps, conf)
, proxy: require('./proxy-conn').create(deps, conf)
, socks5: require('./socks5-server').create(deps, conf)
, loopback: require('./loopback').create(deps, conf)
, ddns: require('./ddns').create(deps, conf)
, mdns: require('./mdns').create(deps, conf)
, udp: require('./udp').create(deps, conf)
, tcp: require('./tcp').create(deps, conf)
, stunneld: require('./tunnel-server-manager').create(deps, config)
};
Object.assign(deps, modules);
require('./goldilocks.js').create(deps, conf);
process.removeListener('message', create);
process.on('message', update);
}

2260
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,14 +1,14 @@
{
"name": "goldilocks",
"version": "1.1.6",
"version": "1.0.0-placeholder",
"description": "The node.js webserver that's just right, Greenlock (HTTPS/TLS/SSL via ACME/Let's Encrypt) and tunneling (RVPN) included.",
"main": "bin/goldilocks.js",
"repository": {
"type": "git",
"url": "git.coolaj86.com:coolaj86/goldilocks.js.git"
"url": "git@git.daplie.com:Daplie/goldilocks.js.git"
},
"author": "AJ ONeal <coolaj86@gmail.com> (https://coolaj86.com/)",
"license": "(MIT OR Apache-2.0)",
"author": "AJ ONeal <aj@daplie.com> (https://daplie.com/)",
"license": "SEE LICENSE IN LICENSE.txt",
"scripts": {
"test": "node bin/goldilocks.js -p 8443 -d /tmp/"
},
@ -34,41 +34,40 @@
"server"
],
"bugs": {
"url": "https://git.coolaj86.com/coolaj86/goldilocks.js/issues"
"url": "https://git.daplie.com/Daplie/server-https/issues"
},
"homepage": "https://git.coolaj86.com/coolaj86/goldilocks.js",
"homepage": "https://git.daplie.com/Daplie/goldilocks.js#readme",
"dependencies": {
"bluebird": "^3.4.6",
"body-parser": "1",
"body-parser": "git+https://github.com/expressjs/body-parser.git#1.16.1",
"commander": "^2.9.0",
"deep-equal": "^1.0.1",
"dns-suite": "1",
"express": "4",
"dns-suite": "git+https://git@git.daplie.com/Daplie/dns-suite#v1",
"express": "git+https://github.com/expressjs/express.git#4.x",
"finalhandler": "^0.4.0",
"greenlock": "2.1",
"greenlock": "git+https://git.daplie.com/Daplie/node-greenlock.git#master",
"http-proxy": "^1.16.2",
"human-readable-ids": "1",
"ipaddr.js": "v1.3",
"human-readable-ids": "git+https://git.daplie.com/Daplie/human-readable-ids-js#master",
"ipaddr.js": "git+https://github.com/whitequark/ipaddr.js.git#v1.3.0",
"js-yaml": "^3.8.3",
"jsonschema": "^1.2.0",
"jsonwebtoken": "^7.4.0",
"le-challenge-fs": "2",
"le-challenge-ddns": "git+https://git.daplie.com/Daplie/le-challenge-ddns.git#master",
"le-challenge-fs": "git+https://git.daplie.com/Daplie/le-challenge-webroot.git#master",
"le-challenge-sni": "^2.0.1",
"le-store-certbot": "2",
"le-store-certbot": "git+https://git.daplie.com/Daplie/le-store-certbot.git#master",
"localhost.daplie.me-certificates": "^1.3.5",
"network": "^0.4.0",
"recase": "v1.0.4",
"recase": "git+https://git.daplie.com/coolaj86/recase-js.git#v1.0.4",
"redirect-https": "^1.1.0",
"request": "^2.81.0",
"scmp": "1",
"scmp": "git+https://github.com/freewil/scmp.git#1.x",
"serve-index": "^1.7.0",
"serve-static": "^1.10.0",
"server-destroy": "^1.0.1",
"sni": "^1.0.0",
"socket-pair": "^1.0.3",
"socket-pair": "^1.0.1",
"socksv5": "0.0.6",
"stunnel": "1.0",
"stunneld": "0.9",
"stunnel": "git+https://git.daplie.com/Daplie/node-tunnel-client.git#v1",
"stunneld": "git+https://git.daplie.com/Daplie/node-tunnel-server.git#v1",
"tunnel-packer": "^1.3.0",
"ws": "^2.3.1"
}

3
terms.sh Normal file
View File

@ -0,0 +1,3 @@
# adding TOS to TXT DNS Record
daplie dns:set -n _terms._cloud.localhost.foo.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600
daplie dns:set -n _terms._cloud.localhost.alpha.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600

17
test-chain.sh Executable file
View File

@ -0,0 +1,17 @@
#!/bin/bash
node serve.js \
--port 8443 \
--key node_modules/localhost.daplie.me-certificates/privkey.pem \
--cert node_modules/localhost.daplie.me-certificates/fullchain.pem \
--root node_modules/localhost.daplie.me-certificates/root.pem \
-c "$(cat node_modules/localhost.daplie.me-certificates/root.pem)" &
PID=$!
sleep 1
curl -s --insecure http://localhost.daplie.me:8443 > ./root.pem
curl -s https://localhost.daplie.me:8443 --cacert ./root.pem
rm ./root.pem
kill $PID 2>/dev/null

48
uninstall.sh Normal file
View File

@ -0,0 +1,48 @@
#!/bin/bash
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
MY_ROOT=""
if [ -z "${PREFIX-}" ]; then
MY_ROOT=""
else
MY_ROOT="$PREFIX"
fi
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
# you don't want any oopsies when an rm -rf is involved...
set -e
set -u
my_app_name=goldilocks
my_app_pkg_name=com.daplie.goldilocks.web
my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
my_app_upstart_service="etc/init.d/${my_app_name}.conf"
$sudo_cmd rm -f /usr/local/bin/$my_app_name
$sudo_cmd rm -f /usr/local/bin/uninstall-$my_app_name
$sudo_cmd rm -rf /usr/local/lib/node_modules/$my_app_name
$sudo_cmd rm -f "$MY_ROOT/$my_app_etc_config"
$sudo_cmd rmdir -p $(dirname "$MY_ROOT/$my_app_etc_config") 2>/dev/null || true
$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd rm -f "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd rm -f "$MY_ROOT/$my_app_upstart_service"
$sudo_cmd rm -rf /opt/$my_app_name
$sudo_cmd rm -rf /var/log/$my_app_name
# TODO flag for --purge
#rm -rf /etc/goldilocks
# TODO trap uninstall function
echo "uninstall complete: $my_app_name"

31
update-packages.sh Executable file
View File

@ -0,0 +1,31 @@
#!/bin/bash
set -e
set -u
pushd $(dirname ${0})/packages/assets
OAUTH3_GIT_URL="https://git.daplie.com/Oauth3/oauth3.js.git"
git clone ${OAUTH3_GIT_URL} org.oauth3 || true
pushd org.oauth3
git remote set-url origin ${OAUTH3_GIT_URL}
git checkout master
git pull
popd
mkdir -p com.jquery
pushd com.jquery
curl -o jquery-3.1.1.js 'https://code.jquery.com/jquery-3.1.1.js'
popd
mkdir -p com.google
pushd com.google
curl -o angular.1.6.2.min.js 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js'
popd
mkdir -p well-known
pushd well-known
ln -snf ../org.oauth3/well-known/oauth3 ./oauth3
popd
popd