Compare commits

..

No commits in common. "master" and "reorganize-modules" have entirely different histories.

33 changed files with 2822 additions and 807 deletions

View File

@ -1,12 +0,0 @@
v1.1.5 - Implemented dns-01 ACME challenges
v1.1.4 - Improved responsiveness to config updates
* changed which TCP/UDP ports are bound to on config update
* update tunnel server settings on config update
* update socks5 setting on config update
v1.1.3 - Better late than never... here's some stuff we've got
* fixed (probably) network settings not being readable
* supports timeouts in loopback check
* loopback check less likely to fail / throw errors, will try again
* supports ddns using audience of token

41
LICENSE
View File

@ -1,41 +0,0 @@
Copyright 2017 Daplie, Inc
This is open source software; you can redistribute it and/or modify it under the
terms of either:
a) the "MIT License"
b) the "Apache-2.0 License"
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Apache-2.0 License Summary
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

3
LICENSE.txt Normal file
View File

@ -0,0 +1,3 @@
Hello all. We make our source code available to view, but we retain copyright.
It's not because we're trying to be mean or anything, we just want to maintain our distribution channel.

121
README.md
View File

@ -20,51 +20,17 @@ The node.js netserver that's just right.
Install Standalone Install Standalone
------- -------
### curl | bash
```bash ```bash
curl -fsSL https://git.coolaj86.com/coolaj86/goldilocks.js/raw/v1.1/installer/get.sh | bash # v1 in npm
``` npm install -g goldilocks
### git
```bash
git clone https://git.coolaj86.com/coolaj86/goldilocks.js
pushd goldilocks.js
git checkout v1.1
bash installer/install.sh
```
### npm
```bash
# v1 in git (unauthenticated)
npm install -g git+https://git@git.coolaj86.com:coolaj86/goldilocks.js#v1
# v1 in git (via ssh) # v1 in git (via ssh)
npm install -g git+ssh://git@git.coolaj86.com:coolaj86/goldilocks.js#v1 npm install -g git+ssh://git@git.daplie.com:Daplie/goldilocks.js#v1
# v1 in npm # v1 in git (unauthenticated)
npm install -g goldilocks@v1 npm install -g git+https://git@git.daplie.com:Daplie/goldilocks.js#v1
``` ```
### Uninstall
Remove goldilocks and services:
```
rm -rf /opt/goldilocks/ /srv/goldilocks/ /var/goldilocks/ /var/log/goldilocks/ /etc/tmpfiles.d/goldilocks.conf /etc/systemd/system/goldilocks.service
```
Remove config as well
```
rm -rf /etc/goldilocks/ /etc/ssl/goldilocks
```
Usage
-----
```bash ```bash
goldilocks goldilocks
``` ```
@ -81,7 +47,7 @@ We have service support for
* launchd (macOS) * launchd (macOS)
```bash ```bash
curl https://git.coolaj86.com/coolaj86/goldilocks.js/raw/master/install.sh | bash curl https://git.daplie.com/Daplie/goldilocks.js/raw/master/install.sh | bash
``` ```
Modules & Configuration Modules & Configuration
@ -305,16 +271,9 @@ tls:
challenge_type: 'http-01' challenge_type: 'http-01'
``` ```
**NOTE:** If you specify `dns-01` as the challenge type there must also be a
[DDNS module](#ddns) defined for all of the relevant domains (though not all
domains handled by a single TLS module need to be handled by the same DDNS
module). The DDNS module provides all of the information needed to actually
set the DNS records needed to verify ownership.
### tcp ### tcp
The tcp system handles both *raw* and *tls-terminated* tcp network traffic The tcp system handles all tcp network traffic **before decryption** and may use port numbers
(see the _Note_ section below the example). It may use port numbers
or traffic sniffing to determine how the connection should be handled. or traffic sniffing to determine how the connection should be handled.
It has the following options: It has the following options:
@ -337,83 +296,37 @@ tcp:
address: '127.0.0.1:2222' address: '127.0.0.1:2222'
``` ```
_Note_: When tcp traffic comes into goldilocks it will be tested against the tcp modules.
The connection may be handed to the TLS module if it appears to be a TLS/SSL/HTTPS connection
and if the tls module terminates the traffic, the connection will be sent back to the TLS module.
Due to the complexity of node.js' networking stack it is not currently possible to tell which
port tls-terminated traffic came from, so only the SNI header (serername / domain name) may be used for
modules matching terminated TLS.
### tcp.proxy ### tcp.proxy
The proxy module routes traffic **after tls-termination** based on the servername (domain name) The proxy module routes traffic based on the servername contained in a SNI header.
contained in a SNI header. As such this only works to route TCP connections wrapped in a TLS stream. As such this only works to route TCP connections wrapped in a TLS stream.
It has the same options as the [HTTP proxy module](#httpproxy-how-to-reverse-proxy-ruby-python-etc). It has the same options as the [HTTP proxy module](#httpproxy-how-to-reverse-proxy-ruby-python-etc).
This is particularly useful for routing ssh and vpn traffic over tcp port 443 as wrapped TLS
connections in order to access one of your servers even when connecting from a harsh or potentially
misconfigured network environment (i.e. hotspots in public libraries and shopping malls).
Example config: Example config:
```yml ```yml
tcp: tcp:
modules: modules:
- type: proxy - type: proxy
domains: domains:
- ssh.example.com # Note: this domain would also listed in tls.acme.domains - ssh.example.com
host: localhost
port: 22 port: 22
- type: proxy
domains:
- vpn.example.com # Note: this domain would also listed in tls.acme.domains
host: localhost
port: 1194
``` ```
_Note_: In same cases network administrators purposefully block ssh and vpn connections using
Application Firewalls with DPI (deep packet inspection) enabled. You should read the ToS of the
network you are connected to to ensure that you aren't subverting policies that are purposefully
in place on such networks.
#### Using with ssh
In order to use this to route SSH connections you will need to use `ssh`'s In order to use this to route SSH connections you will need to use `ssh`'s
`ProxyCommand` option. For example to use the TLS certificate for `ssh.example.com` `ProxyCommand` option. For example to ssh into `ssh.example.com` you could use
to wrap an ssh connection you could use the following command: the following command.
```bash ```bash
ssh user@example.com -o ProxyCommand='openssl s_client -quiet -connect example.com:443 -servername ssh.example.com' ssh user@ssh.example.com -o ProxyCommand='openssl s_client -quiet -connect ssh.example.com:443 -servername ssh.example.com'
``` ```
Alternatively you could add the following lines to your ssh config file. Alternatively you could add the following lines to your ssh config file.
``` ```
Host example.com Host ssh.example.com
ProxyCommand openssl s_client -quiet -connect example.com:443 -servername ssh.example.com ProxyCommand openssl s_client -quiet -connect ssh.example.com:443 -servername ssh.example.com
``` ```
#### Using with OpenVPN
There are two strategies that will work well for you:
1) [Use ssh](https://redfern.me/tunneling-openvpn-through-ssh/) with the config above to reverse proxy tcp port 1194 to you.
```bash
ssh -L 1194:localhost:1194 example.com
```
2) [Use stunnel]https://serverfault.com/questions/675553/stunnel-vpn-traffic-and-ensure-it-looks-like-ssl-traffic-on-port-443/681497)
```
[openvpn-over-goldilocks]
client = yes
accept = 127.0.0.1:1194
sni = vpn.example.com
connect = example.com:443
```
3) [Use stunnel.js](https://git.coolaj86.com/coolaj86/tunnel-client.js) as described in the "tunnel_server" section below.
### tcp.forward ### tcp.forward
The forward module routes traffic based on port number **without decrypting** it. The forward module routes traffic based on port number **without decrypting** it.
@ -616,7 +529,7 @@ mdns:
You can discover goldilocks with `mdig`. You can discover goldilocks with `mdig`.
``` ```
npm install -g git+https://git.coolaj86.com/coolaj86/mdig.js.git npm install -g git+https://git.daplie.com/Daplie/mdig.git
mdig _cloud._tcp.local mdig _cloud._tcp.local
``` ```
@ -645,7 +558,7 @@ TODO
* [ ] http - redirect based on domain name (not just path) * [ ] http - redirect based on domain name (not just path)
* [ ] tcp - bind should be able to specify localhost, uniquelocal, private, or ip * [ ] tcp - bind should be able to specify localhost, uniquelocal, private, or ip
* [ ] tcp - if destination host is omitted default to localhost, if dst port is missing, default to src * [ ] tcp - if destination host is omitted default to localhost, if dst port is missing, default to src
* [ ] sys - `curl https://coolaj86.com/goldilocks | bash -s example.com` * [ ] sys - `curl https://daplie.me/goldilocks | bash -s example.com`
* [ ] oauth3 - `example.com/.well-known/domains@oauth3.org/directives.json` * [ ] oauth3 - `example.com/.well-known/domains@oauth3.org/directives.json`
* [ ] oauth3 - commandline questionnaire * [ ] oauth3 - commandline questionnaire
* [x] modules - use consistent conventions (i.e. address vs host + port) * [x] modules - use consistent conventions (i.e. address vs host + port)

View File

@ -1,5 +0,0 @@
# /etc/tmpfiles.d/goldilocks.conf
# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
# Type Path Mode UID GID Age Argument
d /run/goldilocks 0755 MY_USER MY_GROUP - -

View File

@ -19,14 +19,14 @@ StartLimitBurst=3
# User and group the process will run as # User and group the process will run as
# (www-data is the de facto standard on most systems) # (www-data is the de facto standard on most systems)
User=MY_USER User=www-data
Group=MY_GROUP Group=www-data
# If we need to pass environment variables in the future # If we need to pass environment variables in the future
Environment=GOLDILOCKS_PATH=/srv/www NODE_PATH=/opt/goldilocks/lib/node_modules NPM_CONFIG_PREFIX=/opt/goldilocks Environment=GOLDILOCKS_PATH=/srv/www NODE_PATH=/opt/goldilocks/lib/node_modules NPM_CONFIG_PREFIX=/opt/goldilocks
# Set a sane working directory, sane flags, and specify how to reload the config file # Set a sane working directory, sane flags, and specify how to reload the config file
WorkingDirectory=/opt/goldilocks WorkingDirectory=/srv/www
ExecStart=/opt/goldilocks/bin/node /opt/goldilocks/bin/goldilocks --config /etc/goldilocks/goldilocks.yml ExecStart=/opt/goldilocks/bin/node /opt/goldilocks/bin/goldilocks --config /etc/goldilocks/goldilocks.yml
ExecReload=/bin/kill -USR1 $MAINPID ExecReload=/bin/kill -USR1 $MAINPID
@ -46,7 +46,7 @@ ProtectSystem=full
# … except TLS/SSL, ACME, and Let's Encrypt certificates # … except TLS/SSL, ACME, and Let's Encrypt certificates
# and /var/log/goldilocks, because we want a place where logs can go. # and /var/log/goldilocks, because we want a place where logs can go.
# This merely retains r/w access rights, it does not add any new. Must still be writable on the host! # This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks /opt/goldilocks ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks
# you may also want to add other directories such as /opt/goldilocks /etc/acme /etc/letsencrypt # you may also want to add other directories such as /opt/goldilocks /etc/acme /etc/letsencrypt
# Note: in v231 and above ReadWritePaths has been renamed to ReadWriteDirectories # Note: in v231 and above ReadWritePaths has been renamed to ReadWriteDirectories

View File

@ -0,0 +1,10 @@
# /etc/tmpfiles.d/goldilocks.conf
# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
# Type Path Mode UID GID Age Argument
d /etc/goldilocks 0755 www-data www-data - -
d /opt/goldilocks 0775 www-data www-data - -
d /srv/www 0775 www-data www-data - -
d /etc/ssl/goldilocks 0750 www-data www-data - -
d /var/log/goldilocks 0750 www-data www-data - -
#d /run/goldilocks 0755 www-data www-data - -

224
install.sh Normal file
View File

@ -0,0 +1,224 @@
#!/bin/bash
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
MY_ROOT=""
if [ -z "${PREFIX-}" ]; then
MY_ROOT=""
else
MY_ROOT="$PREFIX"
fi
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
###############################
# #
# http_get #
# boilerplate for curl / wget #
# #
###############################
# See https://git.daplie.com/Daplie/daplie-snippets/blob/master/bash/http-get.sh
http_get=""
http_opts=""
http_out=""
detect_http_get()
{
if type -p curl >/dev/null 2>&1; then
http_get="curl"
http_opts="-fsSL"
http_out="-o"
elif type -p wget >/dev/null 2>&1; then
http_get="wget"
http_opts="--quiet"
http_out="-O"
else
echo "Aborted, could not find curl or wget"
return 7
fi
}
dap_dl()
{
$http_get $http_opts $http_out "$2" "$1"
touch "$2"
}
dap_dl_bash()
{
dap_url=$1
#dap_args=$2
rm -rf dap-tmp-runner.sh
$http_get $http_opts $http_out dap-tmp-runner.sh "$dap_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
}
detect_http_get
## END HTTP_GET ##
###################
# #
# Install service #
# #
###################
my_app_name=goldilocks
my_app_pkg_name=com.daplie.goldilocks.web
my_app_dir=$(mktemp -d)
installer_base="https://git.daplie.com/Daplie/goldilocks.js/raw/master"
my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
my_app_etc_example_config="etc/${my_app_name}/${my_app_name}.example.yml"
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
install_for_systemd()
{
echo ""
echo "Installing as systemd service"
echo ""
mkdir -p $(dirname "$my_app_dir/$my_app_systemd_service")
dap_dl "$installer_base/$my_app_systemd_service" "$my_app_dir/$my_app_systemd_service"
$sudo_cmd mv "$my_app_dir/$my_app_systemd_service" "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_service"
mkdir -p $(dirname "$my_app_dir/$my_app_systemd_tmpfiles")
dap_dl "$installer_base/$my_app_systemd_tmpfiles" "$my_app_dir/$my_app_systemd_tmpfiles"
$sudo_cmd mv "$my_app_dir/$my_app_systemd_tmpfiles" "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null
$sudo_cmd systemctl daemon-reload
$sudo_cmd systemctl start "${my_app_name}.service"
$sudo_cmd systemctl enable "${my_app_name}.service"
echo "$my_app_name started with systemctl, check its status like so"
echo " $sudo_cmd systemctl status $my_app_name"
echo " $sudo_cmd journalctl -xe -u goldilocks"
}
install_for_launchd()
{
echo ""
echo "Installing as launchd service"
echo ""
# See http://www.launchd.info/
mkdir -p $(dirname "$my_app_dir/$my_app_launchd_service")
dap_dl "$installer_base/$my_app_launchd_service" "$my_app_dir/$my_app_launchd_service"
$sudo_cmd mv "$my_app_dir/$my_app_launchd_service" "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd chown root:wheel "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd chmod 0644 "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd launchctl unload -w "$MY_ROOT/$my_app_launchd_service" >/dev/null 2>/dev/null
$sudo_cmd launchctl load -w "$MY_ROOT/$my_app_launchd_service"
echo "$my_app_name started with launchd"
}
install_etc_config()
{
$sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_example_config")
mkdir -p $(dirname "$my_app_dir/$my_app_etc_example_config")
dap_dl "$installer_base/$my_app_etc_example_config" "$my_app_dir/$my_app_etc_example_config"
$sudo_cmd mv "$my_app_dir/$my_app_etc_example_config" "$MY_ROOT/$my_app_etc_example_config"
if [ ! -e "$MY_ROOT/$my_app_etc_config" ]; then
$sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_config")
mkdir -p $(dirname "$my_app_dir/$my_app_etc_config")
dap_dl "$installer_base/$my_app_etc_config" "$my_app_dir/$my_app_etc_config"
$sudo_cmd mv "$my_app_dir/$my_app_etc_config" "$MY_ROOT/$my_app_etc_config"
fi
# OS X
$sudo_cmd chown -R _www:_www $(dirname "$MY_ROOT/$my_app_etc_config") || true
# Linux
$sudo_cmd chown -R www-data:www-data $(dirname "$MY_ROOT/$my_app_etc_config") || true
$sudo_cmd chmod 775 $(dirname "$MY_ROOT/$my_app_etc_config")
$sudo_cmd chmod 664 "$MY_ROOT/$my_app_etc_config"
}
install_service()
{
install_etc_config
installable=""
if [ -d "$MY_ROOT/etc/systemd/system" ]; then
install_for_systemd
installable="true"
fi
if [ -d "/Library/LaunchDaemons" ]; then
install_for_launchd
installable="true"
fi
if [ -z "$installable" ]; then
echo ""
echo "Unknown system service init type. You must install as a system service manually."
echo '(please file a bug with the output of "uname -a")'
echo ""
fi
}
## END SERVICE_INSTALL ##
set -e
set -u
# Install
# TODO install to tmp location, then move to /opt
export NODE_PATH=/opt/goldilocks/lib/node_modules
export NPM_CONFIG_PREFIX=/opt/goldilocks
$sudo_cmd mkdir -p /etc/goldilocks
$sudo_cmd mkdir -p /var/log/goldilocks
$sudo_cmd mkdir -p /srv/www
$sudo_cmd mkdir -p /var/www
$sudo_cmd mkdir -p /opt/goldilocks/{lib,bin,etc}
# Dependencies
dap_dl_bash "https://git.daplie.com/Daplie/node-install-script/raw/master/setup-min.sh"
# Change to user perms
# OS X or Linux
$sudo_cmd chown -R $(whoami) /opt/goldilocks/ || true
my_npm="$NPM_CONFIG_PREFIX/bin/npm"
$my_npm install -g npm@4
$my_npm install -g 'git+https://git@git.daplie.com/Daplie/goldilocks.js.git'
# Finish up with submodule
pushd /opt/goldilocks/lib/node_modules/goldilocks
bash ./update-packages.sh
popd
# Change to admin perms
# OS X
$sudo_cmd chown -R _www:_www /var/www /srv/www /opt/goldilocks || true
# Linux
$sudo_cmd chown -R www-data:www-data /var/www /srv/www /opt/goldilocks || true
# make sure the files are all read/write for the owner and group, and then set
# the setuid and setgid bits so that any files/directories created inside these
# directories have the same owner and group.
$sudo_cmd chmod -R ug+rwX /opt/goldilocks
find /opt/goldilocks -type d -exec $sudo_cmd chmod ug+s {} \;
# Uninstall
dap_dl "https://git.daplie.com/Daplie/goldilocks.js/raw/master/uninstall.sh" "./goldilocks-uninstall"
$sudo_cmd chmod 755 "./goldilocks-uninstall"
# OS X
$sudo_cmd chown root:wheel "./goldilocks-uninstall" || true
# Linux
$sudo_cmd chown root:root "./goldilocks-uninstall" || true
$sudo_cmd mv "./goldilocks-uninstall" "/usr/local/bin/uninstall-goldilocks"
# Install Service
install_service

View File

@ -1,20 +0,0 @@
set -e
set -u
my_name=goldilocks
# TODO provide an option to supply my_ver and my_tmp
my_ver=master
my_tmp=$(mktemp -d)
mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
git clone https://git.coolaj86.com/coolaj86/goldilocks.js.git $my_tmp/opt/$my_name/lib/node_modules/$my_name
echo "Installing to $my_tmp (will be moved after install)"
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
git checkout $my_ver
source ./installer/install.sh
popd
echo "Installation successful, now cleaning up $my_tmp ..."
rm -rf $my_tmp
echo "Done"

View File

@ -1,48 +0,0 @@
###############################
# #
# http_get #
# boilerplate for curl / wget #
# #
###############################
# See https://git.coolaj86.com/coolaj86/snippets/blob/master/bash/http-get.sh
_h_http_get=""
_h_http_opts=""
_h_http_out=""
detect_http_get()
{
set +e
if type -p curl >/dev/null 2>&1; then
_h_http_get="curl"
_h_http_opts="-fsSL"
_h_http_out="-o"
elif type -p wget >/dev/null 2>&1; then
_h_http_get="wget"
_h_http_opts="--quiet"
_h_http_out="-O"
else
echo "Aborted, could not find curl or wget"
return 7
fi
set -e
}
http_get()
{
$_h_http_get $_h_http_opts $_h_http_out "$2" "$1"
touch "$2"
}
http_bash()
{
_http_url=$1
#dap_args=$2
rm -rf dap-tmp-runner.sh
$_h_http_get $_h_http_opts $_h_http_out dap-tmp-runner.sh "$_http_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
}
detect_http_get
## END HTTP_GET ##

View File

@ -1,17 +0,0 @@
set -u
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
echo ""
echo "Installing as launchd service"
echo ""
# See http://www.launchd.info/
safe_copy_config "$my_app_dist/$my_app_launchd_service" "$my_root/$my_app_launchd_service"
$sudo_cmd chown root:wheel "$my_root/$my_app_launchd_service"
$sudo_cmd launchctl unload -w "$my_root/$my_app_launchd_service" >/dev/null 2>/dev/null
$sudo_cmd launchctl load -w "$my_root/$my_app_launchd_service"
echo "$my_app_name started with launchd"

View File

@ -1,37 +0,0 @@
set -u
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
echo ""
echo "Installing as systemd service"
echo ""
sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_service" > "$my_app_dist/$my_app_systemd_service.2"
sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_service.2" > "$my_app_dist/$my_app_systemd_service"
rm "$my_app_dist/$my_app_systemd_service.2"
safe_copy_config "$my_app_dist/$my_app_systemd_service" "$my_root/$my_app_systemd_service"
$sudo_cmd chown root:root "$my_root/$my_app_systemd_service"
sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_tmpfiles" > "$my_app_dist/$my_app_systemd_tmpfiles.2"
sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_tmpfiles.2" > "$my_app_dist/$my_app_systemd_tmpfiles"
rm "$my_app_dist/$my_app_systemd_tmpfiles.2"
safe_copy_config "$my_app_dist/$my_app_systemd_tmpfiles" "$my_root/$my_app_systemd_tmpfiles"
$sudo_cmd chown root:root "$my_root/$my_app_systemd_tmpfiles"
$sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null || true
$sudo_cmd systemctl daemon-reload
$sudo_cmd systemctl start "${my_app_name}.service"
$sudo_cmd systemctl enable "${my_app_name}.service"
echo ""
echo ""
echo "Fun systemd commands to remember:"
echo " $sudo_cmd systemctl daemon-reload"
echo " $sudo_cmd systemctl restart $my_app_name.service"
echo ""
echo "$my_app_name started with systemctl, check its status like so:"
echo " $sudo_cmd systemctl status $my_app_name"
echo " $sudo_cmd journalctl -xefu $my_app_name"
echo ""
echo ""

View File

@ -1,37 +0,0 @@
safe_copy_config()
{
src=$1
dst=$2
$sudo_cmd mkdir -p $(dirname "$dst")
if [ -f "$dst" ]; then
$sudo_cmd rsync -a "$src" "$dst.latest"
# TODO edit config file with $my_user and $my_group
if [ "$(cat $dst)" == "$(cat $dst.latest)" ]; then
$sudo_cmd rm $dst.latest
else
echo "MANUAL INTERVENTION REQUIRED: check the systemd script update and manually decide what you want to do"
echo "diff $dst $dst.latest"
$sudo_cmd chown -R root:root "$dst.latest"
fi
else
$sudo_cmd rsync -a --ignore-existing "$src" "$dst"
fi
$sudo_cmd chown -R root:root "$dst"
$sudo_cmd chmod 644 "$dst"
}
installable=""
if [ -d "$my_root/etc/systemd/system" ]; then
source ./installer/install-for-systemd.sh
installable="true"
fi
if [ -d "/Library/LaunchDaemons" ]; then
source ./installer/install-for-launchd.sh
installable="true"
fi
if [ -z "$installable" ]; then
echo ""
echo "Unknown system service init type. You must install as a system service manually."
echo '(please file a bug with the output of "uname -a")'
echo ""
fi

View File

@ -1,150 +0,0 @@
#!/bin/bash
set -e
set -u
### IMPORTANT ###
### VERSION ###
my_name=goldilocks
my_app_pkg_name=com.coolaj86.goldilocks.web
my_app_ver="v1.1"
my_azp_oauth3_ver="v1.2.3"
export NODE_VERSION="v8.9.3"
if [ -z "${my_tmp-}" ]; then
my_tmp="$(mktemp -d)"
mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
echo "Installing to $my_tmp (will be moved after install)"
git clone ./ $my_tmp/opt/$my_name/lib/node_modules/$my_name
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
fi
#################
export NODE_PATH=$my_tmp/opt/$my_name/lib/node_modules
export PATH=$my_tmp/opt/$my_name/bin/:$PATH
export NPM_CONFIG_PREFIX=$my_tmp/opt/$my_name
my_npm="$NPM_CONFIG_PREFIX/bin/npm"
#################
my_app_dist=$my_tmp/opt/$my_name/lib/node_modules/$my_name/dist
installer_base="https://git.coolaj86.com/coolaj86/goldilocks.js/raw/$my_app_ver"
# Backwards compat
# some scripts still use the old names
my_app_dir=$my_tmp
my_app_name=$my_name
git checkout $my_app_ver
mkdir -p "$my_tmp/opt/$my_name"/{lib,bin,etc}
ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name
ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name.js
mkdir -p "$my_tmp/etc/$my_name"
chmod 775 "$my_tmp/etc/$my_name"
cat "$my_app_dist/etc/$my_name/$my_name.example.yml" > "$my_tmp/etc/$my_name/$my_name.example.yml"
chmod 664 "$my_tmp/etc/$my_name/$my_name.example.yml"
mkdir -p $my_tmp/srv/www
mkdir -p $my_tmp/var/www
mkdir -p $my_tmp/var/log/$my_name
#
# Helpers
#
source ./installer/sudo-cmd.sh
source ./installer/http-get.sh
#
# Dependencies
#
echo $NODE_VERSION > /tmp/NODEJS_VER
http_bash "https://git.coolaj86.com/coolaj86/node-installer.sh/raw/v1.1/install.sh"
$my_npm install -g npm@4
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
$my_npm install
popd
pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name/packages/assets
OAUTH3_GIT_URL="https://git.oauth3.org/OAuth3/oauth3.js.git"
git clone ${OAUTH3_GIT_URL} oauth3.org || true
ln -s oauth3.org org.oauth3
pushd oauth3.org
git remote set-url origin ${OAUTH3_GIT_URL}
git checkout $my_azp_oauth3_ver
#git pull
popd
mkdir -p jquery.com
ln -s jquery.com com.jquery
pushd jquery.com
http_get 'https://code.jquery.com/jquery-3.1.1.js' jquery-3.1.1.js
popd
mkdir -p google.com
ln -s google.com com.google
pushd google.com
http_get 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js' angular.1.6.2.min.js
popd
mkdir -p well-known
ln -s well-known .well-known
pushd well-known
ln -snf ../oauth3.org/well-known/oauth3 ./oauth3
popd
echo "installed dependencies"
popd
#
# System Service
#
source ./installer/my-root.sh
echo "Pre-installation to $my_tmp complete, now installing to $my_root/ ..."
set +e
if type -p tree >/dev/null 2>/dev/null; then
#tree -I "node_modules|include|share" $my_tmp
tree -L 6 -I "include|share|npm" $my_tmp
else
ls $my_tmp
fi
set -e
source ./installer/my-user-my-group.sh
echo "User $my_user Group $my_group"
source ./installer/install-system-service.sh
$sudo_cmd chown -R $my_user:$my_group $my_tmp/*
$sudo_cmd chown root:root $my_tmp/*
$sudo_cmd chown root:root $my_tmp
$sudo_cmd chmod 0755 $my_tmp
# don't change permissions on /, /etc, etc
$sudo_cmd rsync -a --ignore-existing $my_tmp/ $my_root/
$sudo_cmd rsync -a --ignore-existing $my_app_dist/etc/$my_name/$my_name.yml $my_root/etc/$my_name/$my_name.yml
# Change to admin perms
$sudo_cmd chown -R $my_user:$my_group $my_root/opt/$my_name
$sudo_cmd chown -R $my_user:$my_group $my_root/var/www $my_root/srv/www
# make sure the files are all read/write for the owner and group, and then set
# the setuid and setgid bits so that any files/directories created inside these
# directories have the same owner and group.
$sudo_cmd chmod -R ug+rwX $my_root/opt/$my_name
find $my_root/opt/$my_name -type d -exec $sudo_cmd chmod ug+s {} \;
echo ""
echo "$my_name installation complete!"
echo ""
echo ""
echo "Update the config at: /etc/$my_name/$my_name.yml"
echo ""
echo "Unistall: rm -rf /srv/$my_name/ /var/$my_name/ /etc/$my_name/ /opt/$my_name/ /var/log/$my_name/ /etc/tmpfiles.d/$my_name.conf /etc/systemd/system/$my_name.service /etc/ssl/$my_name"

View File

@ -1,8 +0,0 @@
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
my_root=""
if [ -z "${PREFIX-}" ]; then
my_root=""
else
my_root="$PREFIX"
fi

View File

@ -1,19 +0,0 @@
if type -p adduser >/dev/null 2>/dev/null; then
if [ -z "$(cat $my_root/etc/passwd | grep $my_app_name)" ]; then
$sudo_cmd adduser --home $my_root/opt/$my_app_name --gecos '' --disabled-password $my_app_name
fi
my_user=$my_app_name
my_group=$my_app_name
elif [ -n "$(cat /etc/passwd | grep www-data:)" ]; then
# Linux (Ubuntu)
my_user=www-data
my_group=www-data
elif [ -n "$(cat /etc/passwd | grep _www:)" ]; then
# Mac
my_user=_www
my_group=_www
else
# Unsure
my_user=$(whoami)
my_group=$(id -g -n)
fi

View File

@ -1,7 +0,0 @@
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
set +e
if type -p sudo >/dev/null 2>/dev/null; then
((EUID)) && [[ -z "${ANDROID_ROOT-}" ]] && sudo_cmd="sudo"
fi
set -e

View File

@ -1,122 +0,0 @@
'use strict';
// Much of this file was based on the `le-challenge-ddns` library (which we are not using
// here because it's method of setting records requires things we don't really want).
module.exports.create = function (deps, conf, utils) {
function getReleventSessionId(domain) {
var sessId;
utils.iterateAllModules(function (mod, domainList) {
// We return a truthy value in these cases because of the way the iterate function
// handles modules grouped by domain. By returning true we are saying these domains
// are "handled" and so if there are multiple modules we won't be given the rest.
if (sessId) { return true; }
if (domainList.indexOf(domain) < 0) { return true; }
// But if the domains are relevant but we don't know how to handle the module we
// return false to allow us to look at any other modules that might exist here.
if (mod.type !== 'dns@oauth3.org') { return false; }
sessId = mod.tokenId || mod.token_id;
return true;
});
return sessId;
}
function get(args, domain, challenge, done) {
done(new Error("Challenge.get() does not need an implementation for dns-01. (did you mean Challenge.loopback?)"));
}
// same as get, but external
function loopback(args, domain, challenge, done) {
var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
require('dns').resolveTxt(challengeDomain, done);
}
var activeChallenges = {};
async function removeAsync(args, domain) {
var data = activeChallenges[domain];
if (!data) {
console.warn(new Error('cannot remove DNS challenge for ' + domain + ': already removed'));
return;
}
var session = await utils.getSession(data.sessId);
var directives = await deps.OAUTH3.discover(session.token.aud);
var apiOpts = {
api: 'dns.unset'
, session: session
, type: 'TXT'
, value: data.keyAuthDigest
};
await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, data.splitDomain));
delete activeChallenges[domain];
}
async function setAsync(args, domain, challenge, keyAuth) {
if (activeChallenges[domain]) {
await removeAsync(args, domain, challenge);
}
var sessId = getReleventSessionId(domain);
if (!sessId) {
throw new Error('no DDNS module handles the domain ' + domain);
}
var session = await utils.getSession(sessId);
var directives = await deps.OAUTH3.discover(session.token.aud);
// I'm not sure what role challenge is supposed to play since even in the library
// this code is based on it was never used, but check for it anyway because ...
if (!challenge || keyAuth) {
console.warn(new Error('DDNS challenge missing challenge or keyAuth'));
}
var keyAuthDigest = require('crypto').createHash('sha256').update(keyAuth || '').digest('base64')
.replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
var splitDomain = (await utils.splitDomains(directives.api, [challengeDomain]))[0];
var apiOpts = {
api: 'dns.set'
, session: session
, type: 'TXT'
, value: keyAuthDigest
, ttl: args.ttl || 0
};
await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, splitDomain));
activeChallenges[domain] = {
sessId
, keyAuthDigest
, splitDomain
};
return new Promise(res => setTimeout(res, 1000));
}
// It might be slightly easier to use arguments and apply, but the library that will use
// this function counts the arguments we expect.
function set(a, b, c, d, done) {
setAsync(a, b, c, d).then(result => done(null, result), done);
}
function remove(a, b, c, done) {
removeAsync(a, b, c).then(result => done(null, result), done);
}
function getOptions() {
return {
oauth3: 'oauth3.org'
, debug: conf.debug
, acmeChallengeDns: '_acme-challenge.'
};
}
return {
getOptions
, set
, get
, remove
, loopback
};
};

View File

@ -1,6 +1,6 @@
'use strict'; 'use strict';
module.exports.create = function (deps, conf, utils) { module.exports.create = function (deps, conf) {
function dnsType(addr) { function dnsType(addr) {
if (/^\d+\.\d+\.\d+\.\d+$/.test(addr)) { if (/^\d+\.\d+\.\d+\.\d+$/.test(addr)) {
return 'A'; return 'A';
@ -10,6 +10,62 @@ module.exports.create = function (deps, conf, utils) {
} }
} }
var tldCache = {};
async function getTlds(provider) {
async function updateCache() {
var reqObj = {
url: deps.OAUTH3.url.normalize(provider)+'/api/com.daplie.domains/prices'
, method: 'GET'
, json: true
};
var resp = await deps.OAUTH3.request(reqObj);
var tldObj = {};
resp.data.forEach(function (tldInfo) {
if (tldInfo.enabled) {
tldObj[tldInfo.tld] = true;
}
});
tldCache[provider] = {
time: Date.now()
, tlds: tldObj
};
return tldObj;
}
// If we've never cached the results we need to return the promise that will fetch the recult,
// otherwise we can return the cached value. If the cached value has "expired", we can still
// return the cached value we just want to update the cache in parellel (making sure we only
// update once).
if (!tldCache[provider]) {
return updateCache();
}
if (!tldCache[provider].updating && Date.now() - tldCache[provider].time > 24*60*60*1000) {
tldCache[provider].updating = true;
updateCache();
}
return tldCache[provider].tlds;
}
async function splitDomains(provider, domains) {
var tlds = await getTlds(provider);
return domains.map(function (domain) {
var split = domain.split('.');
var tldSegCnt = tlds[split.slice(-2).join('.')] ? 2 : 1;
// Currently assuming that the sld can't contain dots, and that the tld can have at
// most one dot. Not 100% sure this is a valid assumption, but exceptions should be
// rare even if the assumption isn't valid.
return {
tld: split.slice(-tldSegCnt).join('.')
, sld: split.slice(-tldSegCnt-1, -tldSegCnt).join('.')
, sub: split.slice(0, -tldSegCnt-1).join('.')
};
});
}
async function setDeviceAddress(session, addr, domains) { async function setDeviceAddress(session, addr, domains) {
var directives = await deps.OAUTH3.discover(session.token.aud); var directives = await deps.OAUTH3.discover(session.token.aud);
@ -52,10 +108,10 @@ module.exports.create = function (deps, conf, utils) {
return record.value === addr && badAddrDomains.indexOf(record.host) < 0; return record.value === addr && badAddrDomains.indexOf(record.host) < 0;
}).map(record => record.host); }).map(record => record.host);
var requiredUpdates = domains.filter(function (domain) { var requiredUpdates = domains.filter(function (domain) {
return goodAddrDomains.indexOf(domain) < 0; return goodAddrDomains.indexOf(domain) !== -1;
}); });
var oldDns = await utils.splitDomains(directives.api, badAddrDomains); var oldDns = await splitDomains(directives.api, badAddrDomains);
var common = { var common = {
api: 'devices.detach' api: 'devices.detach'
, session: session , session: session
@ -64,11 +120,8 @@ module.exports.create = function (deps, conf, utils) {
await deps.PromiseA.all(oldDns.map(function (record) { await deps.PromiseA.all(oldDns.map(function (record) {
return deps.OAUTH3.api(directives.api, Object.assign({}, common, record)); return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
})); }));
if (conf.debug && badAddrDomains.length) {
console.log('removed bad DNS records for ' + badAddrDomains.join(', '));
}
var newDns = await utils.splitDomains(directives.api, requiredUpdates); var newDns = await splitDomains(directives.api, requiredUpdates);
common = { common = {
api: 'devices.attach' api: 'devices.attach'
, session: session , session: session
@ -79,9 +132,6 @@ module.exports.create = function (deps, conf, utils) {
await deps.PromiseA.all(newDns.map(function (record) { await deps.PromiseA.all(newDns.map(function (record) {
return deps.OAUTH3.api(directives.api, Object.assign({}, common, record)); return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
})); }));
if (conf.debug && requiredUpdates.length) {
console.log('set new DNS records for ' + requiredUpdates.join(', '));
}
} }
async function getDeviceAddresses(session) { async function getDeviceAddresses(session) {
@ -113,7 +163,7 @@ module.exports.create = function (deps, conf, utils) {
async function removeDomains(session, domains) { async function removeDomains(session, domains) {
var directives = await deps.OAUTH3.discover(session.token.aud); var directives = await deps.OAUTH3.discover(session.token.aud);
var oldDns = await utils.splitDomains(directives.api, domains); var oldDns = await splitDomains(directives.api, domains);
var common = { var common = {
api: 'devices.detach' api: 'devices.detach'
, session: session , session: session

View File

@ -3,20 +3,48 @@
module.exports.create = function (deps, conf) { module.exports.create = function (deps, conf) {
var dns = deps.PromiseA.promisifyAll(require('dns')); var dns = deps.PromiseA.promisifyAll(require('dns'));
var network = deps.PromiseA.promisifyAll(deps.recase.camelCopy(require('network'))); var network = deps.PromiseA.promisifyAll(deps.recase.camelCopy(require('network')));
var loopback = require('./loopback').create(deps, conf);
var dnsCtrl = require('./dns-ctrl').create(deps, conf);
var tunnelClients = require('./tunnel-client-manager').create(deps, conf);
var equal = require('deep-equal'); var equal = require('deep-equal');
var utils = require('./utils').create(deps, conf);
var loopback = require('./loopback').create(deps, conf, utils);
var dnsCtrl = require('./dns-ctrl').create(deps, conf, utils);
var challenge = require('./challenge-responder').create(deps, conf, utils);
var tunnelClients = require('./tunnel-client-manager').create(deps, conf, utils);
var loopbackDomain; var loopbackDomain;
function iterateAllModules(action, curConf) {
curConf = curConf || conf;
var promises = curConf.ddns.modules.map(function (mod) {
return action(mod, mod.domains);
});
curConf.domains.forEach(function (dom) {
if (!dom.modules || !Array.isArray(dom.modules.ddns) || !dom.modules.ddns.length) {
return null;
}
// For the time being all of our things should only be tried once (regardless if it succeeded)
// TODO: revisit this behavior when we support multiple ways of setting records, and/or
// if we want to allow later modules to run if early modules fail.
promises.push(dom.modules.ddns.reduce(function (prom, mod) {
if (prom) { return prom; }
return action(mod, dom.names);
}, null));
});
return deps.PromiseA.all(promises.filter(Boolean));
}
async function getSession(id) {
var session = await deps.storage.tokens.get(id);
if (!session) {
throw new Error('no user token with ID "'+id+'"');
}
return session;
}
var tunnelActive = false; var tunnelActive = false;
async function startTunnel(tunnelSession, mod, domainList) { async function startTunnel(tunnelSession, mod, domainList) {
try { try {
var dnsSession = await utils.getSession(mod.tokenId); var dnsSession = await getSession(mod.tokenId);
var tunnelDomain = await tunnelClients.start(tunnelSession || dnsSession, domainList); var tunnelDomain = await tunnelClients.start(tunnelSession || dnsSession, domainList);
var addrList; var addrList;
@ -32,9 +60,7 @@ module.exports.create = function (deps, conf) {
throw new Error('failed to lookup IP for tunnel domain "' + tunnelDomain + '"'); throw new Error('failed to lookup IP for tunnel domain "' + tunnelDomain + '"');
} }
if (!mod.disabled) { await dnsCtrl.setDeviceAddress(dnsSession, addrList[0], domainList);
await dnsCtrl.setDeviceAddress(dnsSession, addrList[0], domainList);
}
} catch (err) { } catch (err) {
console.log('error starting tunnel for', domainList.join(', ')); console.log('error starting tunnel for', domainList.join(', '));
console.log(err); console.log(err);
@ -48,7 +74,7 @@ module.exports.create = function (deps, conf) {
tunnelSession = await deps.storage.tokens.get(conf.ddns.tunnel.tokenId); tunnelSession = await deps.storage.tokens.get(conf.ddns.tunnel.tokenId);
} }
await utils.iterateAllModules(function (mod, domainList) { await iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return null; } if (mod.type !== 'dns@oauth3.org') { return null; }
return startTunnel(tunnelSession, mod, domainList); return startTunnel(tunnelSession, mod, domainList);
@ -64,7 +90,7 @@ module.exports.create = function (deps, conf) {
async function checkTunnelTokens() { async function checkTunnelTokens() {
var oldTokens = tunnelClients.current(); var oldTokens = tunnelClients.current();
var newTokens = await utils.iterateAllModules(function checkTokens(mod, domainList) { var newTokens = await iterateAllModules(function checkTokens(mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return null; } if (mod.type !== 'dns@oauth3.org') { return null; }
var domainStr = domainList.slice().sort().join(','); var domainStr = domainList.slice().sort().join(',');
@ -99,17 +125,8 @@ module.exports.create = function (deps, conf) {
// Since we can't detect the OS level events when a user plugs in an ethernet cable to recheck // Since we can't detect the OS level events when a user plugs in an ethernet cable to recheck
// what network environment we are in we check our local network address and the gateway to // what network environment we are in we check our local network address and the gateway to
// determine if we need to run the loopback check and router configuration again. // determine if we need to run the loopback check and router configuration again.
var gw = await network.getGatewayIpAsync();
var addr = await network.getPrivateIpAsync(); var addr = await network.getPrivateIpAsync();
// Until the author of the `network` package publishes the pull request we gave him
// checking the gateway on our units fails because we have the busybox versions of
// the linux commands. Gateway is realistically less important than address, so if
// we fail in getting it go ahead and use the null value.
var gw;
try {
gw = await network.getGatewayIpAsync();
} catch (err) {
gw = null;
}
if (localAddr === addr && gateway === gw) { if (localAddr === addr && gateway === gw) {
return; return;
} }
@ -162,10 +179,10 @@ module.exports.create = function (deps, conf) {
} }
publicAddress = addr; publicAddress = addr;
await utils.iterateAllModules(function setModuleDNS(mod, domainList) { await iterateAllModules(function setModuleDNS(mod, domainList) {
if (mod.type !== 'dns@oauth3.org' || mod.disabled) { return null; } if (mod.type !== 'dns@oauth3.org' || mod.disabled) { return null; }
return utils.getSession(mod.tokenId).then(function (session) { return getSession(mod.tokenId).then(function (session) {
return dnsCtrl.setDeviceAddress(session, addr, domainList); return dnsCtrl.setDeviceAddress(session, addr, domainList);
}).catch(function (err) { }).catch(function (err) {
console.log('error setting DNS records for', domainList.join(', ')); console.log('error setting DNS records for', domainList.join(', '));
@ -180,13 +197,13 @@ module.exports.create = function (deps, conf) {
// this returns a Promise, but since the functions we use are synchronous // this returns a Promise, but since the functions we use are synchronous
// and change our enclosed variables we don't need to wait for the return. // and change our enclosed variables we don't need to wait for the return.
utils.iterateAllModules(function (mod, domainList) { iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return; } if (mod.type !== 'dns@oauth3.org') { return; }
prevMods[mod.id] = { mod, domainList }; prevMods[mod.id] = { mod, domainList };
return true; return true;
}, prevConf); }, prevConf);
utils.iterateAllModules(function (mod, domainList) { iterateAllModules(function (mod, domainList) {
if (mod.type !== 'dns@oauth3.org') { return; } if (mod.type !== 'dns@oauth3.org') { return; }
curMods[mod.id] = { mod, domainList }; curMods[mod.id] = { mod, domainList };
@ -209,11 +226,8 @@ module.exports.create = function (deps, conf) {
// Then remove DNS records for the domains that we are no longer responsible for. // Then remove DNS records for the domains that we are no longer responsible for.
await Promise.all(Object.values(prevMods).map(function ({mod, domainList}) { await Promise.all(Object.values(prevMods).map(function ({mod, domainList}) {
// If the module was disabled before there should be any records that we need to clean up
if (mod.disabled) { return; }
var oldDomains; var oldDomains;
if (!curMods[mod.id] || curMods[mod.id].disabled || mod.tokenId !== curMods[mod.id].mod.tokenId) { if (!curMods[mod.id] || mod.tokenId !== curMods[mod.id].mod.tokenId) {
oldDomains = domainList.slice(); oldDomains = domainList.slice();
} else { } else {
oldDomains = domainList.filter(function (domain) { oldDomains = domainList.filter(function (domain) {
@ -227,7 +241,7 @@ module.exports.create = function (deps, conf) {
return; return;
} }
return utils.getSession(mod.tokenId).then(function (session) { return getSession(mod.tokenId).then(function (session) {
return dnsCtrl.removeDomains(session, oldDomains); return dnsCtrl.removeDomains(session, oldDomains);
}); });
}).filter(Boolean)); }).filter(Boolean));
@ -237,9 +251,6 @@ module.exports.create = function (deps, conf) {
// And add DNS records for any newly added domains. // And add DNS records for any newly added domains.
await Promise.all(Object.values(curMods).map(function ({mod, domainList}) { await Promise.all(Object.values(curMods).map(function ({mod, domainList}) {
// Don't set any new records if the module has been disabled.
if (mod.disabled) { return; }
var newDomains; var newDomains;
if (!prevMods[mod.id] || mod.tokenId !== prevMods[mod.id].mod.tokenId) { if (!prevMods[mod.id] || mod.tokenId !== prevMods[mod.id].mod.tokenId) {
newDomains = domainList.slice(); newDomains = domainList.slice();
@ -255,7 +266,7 @@ module.exports.create = function (deps, conf) {
return; return;
} }
return utils.getSession(mod.tokenId).then(function (session) { return getSession(mod.tokenId).then(function (session) {
return dnsCtrl.setDeviceAddress(session, publicAddress, newDomains); return dnsCtrl.setDeviceAddress(session, publicAddress, newDomains);
}); });
}).filter(Boolean)); }).filter(Boolean));
@ -321,6 +332,5 @@ module.exports.create = function (deps, conf) {
, getDeviceAddresses: dnsCtrl.getDeviceAddresses , getDeviceAddresses: dnsCtrl.getDeviceAddresses
, recheckPubAddr: recheckPubAddr , recheckPubAddr: recheckPubAddr
, updateConf: updateConf , updateConf: updateConf
, challenge
}; };
}; };

View File

@ -1,102 +0,0 @@
'use strict';
module.exports.create = function (deps, conf) {
async function getSession(id) {
var session = await deps.storage.tokens.get(id);
if (!session) {
throw new Error('no user token with ID "' + id + '"');
}
return session;
}
function iterateAllModules(action, curConf) {
curConf = curConf || conf;
var promises = [];
curConf.domains.forEach(function (dom) {
if (!dom.modules || !Array.isArray(dom.modules.ddns) || !dom.modules.ddns.length) {
return null;
}
// For the time being all of our things should only be tried once (regardless if it succeeded)
// TODO: revisit this behavior when we support multiple ways of setting records, and/or
// if we want to allow later modules to run if early modules fail.
promises.push(dom.modules.ddns.reduce(function (prom, mod) {
if (prom) { return prom; }
return action(mod, dom.names);
}, null));
});
curConf.ddns.modules.forEach(function (mod) {
promises.push(action(mod, mod.domains));
});
return Promise.all(promises.filter(Boolean));
}
var tldCache = {};
async function updateTldCache(provider) {
var reqObj = {
url: deps.OAUTH3.url.normalize(provider) + '/api/com.daplie.domains/prices'
, method: 'GET'
, json: true
};
var resp = await deps.OAUTH3.request(reqObj);
var tldObj = {};
resp.data.forEach(function (tldInfo) {
if (tldInfo.enabled) {
tldObj[tldInfo.tld] = true;
}
});
tldCache[provider] = {
time: Date.now()
, tlds: tldObj
};
return tldObj;
}
async function getTlds(provider) {
// If we've never cached the results we need to return the promise that will fetch the result,
// otherwise we can return the cached value. If the cached value has "expired", we can still
// return the cached value we just want to update the cache in parellel (making sure we only
// update once).
if (!tldCache[provider]) {
tldCache[provider] = {
updating: true
, tlds: updateTldCache(provider)
};
}
if (!tldCache[provider].updating && Date.now() - tldCache[provider].time > 24 * 60 * 60 * 1000) {
tldCache[provider].updating = true;
updateTldCache(provider);
}
return tldCache[provider].tlds;
}
async function splitDomains(provider, domains) {
var tlds = await getTlds(provider);
return domains.map(function (domain) {
var split = domain.split('.');
var tldSegCnt = tlds[split.slice(-2).join('.')] ? 2 : 1;
// Currently assuming that the sld can't contain dots, and that the tld can have at
// most one dot. Not 100% sure this is a valid assumption, but exceptions should be
// rare even if the assumption isn't valid.
return {
tld: split.slice(-tldSegCnt).join('.')
, sld: split.slice(-tldSegCnt - 1, -tldSegCnt).join('.')
, sub: split.slice(0, -tldSegCnt - 1).join('.')
};
});
}
return {
getSession
, iterateAllModules
, getTlds
, splitDomains
};
};

View File

@ -80,7 +80,6 @@ module.exports.create = function (deps, config) {
configEnabled = true; configEnabled = true;
} }
} }
process.nextTick(updateConf);
return { return {
curState curState

View File

@ -159,13 +159,11 @@ module.exports.create = function (deps, config) {
}); });
} }
process.nextTick(function () { modules = {};
modules = {}; modules.tcpHandler = tcpHandler;
modules.tcpHandler = tcpHandler; modules.proxy = require('./proxy-conn').create(deps, config);
modules.proxy = require('./proxy-conn').create(deps, config); modules.tls = require('./tls').create(deps, config, modules);
modules.tls = require('./tls').create(deps, config, modules); modules.http = require('./http').create(deps, config, modules);
modules.http = require('./http').create(deps, config, modules);
});
function updateListeners() { function updateListeners() {
var current = listeners.list(); var current = listeners.list();

View File

@ -86,7 +86,8 @@ module.exports.create = function (deps, config, tcpMods) {
, challenges: { , challenges: {
'http-01': require('le-challenge-fs').create({ debug: config.debug }) 'http-01': require('le-challenge-fs').create({ debug: config.debug })
, 'tls-sni-01': require('le-challenge-sni').create({ debug: config.debug }) , 'tls-sni-01': require('le-challenge-sni').create({ debug: config.debug })
, 'dns-01': deps.ddns.challenge // TODO dns-01
//, 'dns-01': require('le-challenge-ddns').create({ debug: config.debug })
} }
, challengeType: 'http-01' , challengeType: 'http-01'

2332
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,14 +1,14 @@
{ {
"name": "goldilocks", "name": "goldilocks",
"version": "1.1.6", "version": "1.0.0-placeholder",
"description": "The node.js webserver that's just right, Greenlock (HTTPS/TLS/SSL via ACME/Let's Encrypt) and tunneling (RVPN) included.", "description": "The node.js webserver that's just right, Greenlock (HTTPS/TLS/SSL via ACME/Let's Encrypt) and tunneling (RVPN) included.",
"main": "bin/goldilocks.js", "main": "bin/goldilocks.js",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git.coolaj86.com:coolaj86/goldilocks.js.git" "url": "git@git.daplie.com:Daplie/goldilocks.js.git"
}, },
"author": "AJ ONeal <coolaj86@gmail.com> (https://coolaj86.com/)", "author": "AJ ONeal <aj@daplie.com> (https://daplie.com/)",
"license": "(MIT OR Apache-2.0)", "license": "SEE LICENSE IN LICENSE.txt",
"scripts": { "scripts": {
"test": "node bin/goldilocks.js -p 8443 -d /tmp/" "test": "node bin/goldilocks.js -p 8443 -d /tmp/"
}, },
@ -34,41 +34,42 @@
"server" "server"
], ],
"bugs": { "bugs": {
"url": "https://git.coolaj86.com/coolaj86/goldilocks.js/issues" "url": "https://git.daplie.com/Daplie/server-https/issues"
}, },
"homepage": "https://git.coolaj86.com/coolaj86/goldilocks.js", "homepage": "https://git.daplie.com/Daplie/goldilocks.js#readme",
"dependencies": { "dependencies": {
"bluebird": "^3.4.6", "bluebird": "^3.4.6",
"body-parser": "1", "body-parser": "git+https://github.com/expressjs/body-parser.git#1.16.1",
"commander": "^2.9.0", "commander": "^2.9.0",
"deep-equal": "^1.0.1", "deep-equal": "^1.0.1",
"dns-suite": "1", "dns-suite": "git+https://git@git.daplie.com/Daplie/dns-suite#v1",
"express": "4", "express": "git+https://github.com/expressjs/express.git#4.x",
"finalhandler": "^0.4.0", "finalhandler": "^0.4.0",
"greenlock": "2.1", "greenlock": "git+https://git.daplie.com/Daplie/node-greenlock.git#master",
"http-proxy": "^1.16.2", "http-proxy": "^1.16.2",
"human-readable-ids": "1", "human-readable-ids": "git+https://git.daplie.com/Daplie/human-readable-ids-js#master",
"ipaddr.js": "v1.3", "ipaddr.js": "git+https://github.com/whitequark/ipaddr.js.git#v1.3.0",
"js-yaml": "^3.8.3", "js-yaml": "^3.8.3",
"jsonschema": "^1.2.0", "jsonschema": "^1.2.0",
"jsonwebtoken": "^7.4.0", "jsonwebtoken": "^7.4.0",
"le-challenge-fs": "2", "le-challenge-ddns": "git+https://git.daplie.com/Daplie/le-challenge-ddns.git#master",
"le-challenge-fs": "git+https://git.daplie.com/Daplie/le-challenge-webroot.git#master",
"le-challenge-sni": "^2.0.1", "le-challenge-sni": "^2.0.1",
"le-store-certbot": "2", "le-store-certbot": "git+https://git.daplie.com/Daplie/le-store-certbot.git#master",
"localhost.daplie.me-certificates": "^1.3.5", "localhost.daplie.me-certificates": "^1.3.5",
"network": "^0.4.0", "network": "^0.4.0",
"recase": "v1.0.4", "recase": "git+https://git.daplie.com/coolaj86/recase-js.git#v1.0.4",
"redirect-https": "^1.1.0", "redirect-https": "^1.1.0",
"request": "^2.81.0", "request": "^2.81.0",
"scmp": "1", "scmp": "git+https://github.com/freewil/scmp.git#1.x",
"serve-index": "^1.7.0", "serve-index": "^1.7.0",
"serve-static": "^1.10.0", "serve-static": "^1.10.0",
"server-destroy": "^1.0.1", "server-destroy": "^1.0.1",
"sni": "^1.0.0", "sni": "^1.0.0",
"socket-pair": "^1.0.3", "socket-pair": "^1.0.3",
"socksv5": "0.0.6", "socksv5": "0.0.6",
"stunnel": "1.0", "stunnel": "git+https://git.daplie.com/Daplie/node-tunnel-client.git#v1",
"stunneld": "0.9", "stunneld": "git+https://git.daplie.com/Daplie/node-tunnel-server.git#v1",
"tunnel-packer": "^1.3.0", "tunnel-packer": "^1.3.0",
"ws": "^2.3.1" "ws": "^2.3.1"
} }

3
terms.sh Normal file
View File

@ -0,0 +1,3 @@
# adding TOS to TXT DNS Record
daplie dns:set -n _terms._cloud.localhost.foo.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600
daplie dns:set -n _terms._cloud.localhost.alpha.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600

17
test-chain.sh Executable file
View File

@ -0,0 +1,17 @@
#!/bin/bash
node serve.js \
--port 8443 \
--key node_modules/localhost.daplie.me-certificates/privkey.pem \
--cert node_modules/localhost.daplie.me-certificates/fullchain.pem \
--root node_modules/localhost.daplie.me-certificates/root.pem \
-c "$(cat node_modules/localhost.daplie.me-certificates/root.pem)" &
PID=$!
sleep 1
curl -s --insecure http://localhost.daplie.me:8443 > ./root.pem
curl -s https://localhost.daplie.me:8443 --cacert ./root.pem
rm ./root.pem
kill $PID 2>/dev/null

48
uninstall.sh Normal file
View File

@ -0,0 +1,48 @@
#!/bin/bash
# something or other about android and tmux using PREFIX
#: "${PREFIX:=''}"
MY_ROOT=""
if [ -z "${PREFIX-}" ]; then
MY_ROOT=""
else
MY_ROOT="$PREFIX"
fi
# Not every platform has or needs sudo, gotta save them O(1)s...
sudo_cmd=""
((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
# you don't want any oopsies when an rm -rf is involved...
set -e
set -u
my_app_name=goldilocks
my_app_pkg_name=com.daplie.goldilocks.web
my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
my_app_upstart_service="etc/init.d/${my_app_name}.conf"
$sudo_cmd rm -f /usr/local/bin/$my_app_name
$sudo_cmd rm -f /usr/local/bin/uninstall-$my_app_name
$sudo_cmd rm -rf /usr/local/lib/node_modules/$my_app_name
$sudo_cmd rm -f "$MY_ROOT/$my_app_etc_config"
$sudo_cmd rmdir -p $(dirname "$MY_ROOT/$my_app_etc_config") 2>/dev/null || true
$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_service"
$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_tmpfiles"
$sudo_cmd rm -f "$MY_ROOT/$my_app_launchd_service"
$sudo_cmd rm -f "$MY_ROOT/$my_app_upstart_service"
$sudo_cmd rm -rf /opt/$my_app_name
$sudo_cmd rm -rf /var/log/$my_app_name
# TODO flag for --purge
#rm -rf /etc/goldilocks
# TODO trap uninstall function
echo "uninstall complete: $my_app_name"

31
update-packages.sh Executable file
View File

@ -0,0 +1,31 @@
#!/bin/bash
set -e
set -u
pushd $(dirname ${0})/packages/assets
OAUTH3_GIT_URL="https://git.daplie.com/Oauth3/oauth3.js.git"
git clone ${OAUTH3_GIT_URL} org.oauth3 || true
pushd org.oauth3
git remote set-url origin ${OAUTH3_GIT_URL}
git checkout master
git pull
popd
mkdir -p com.jquery
pushd com.jquery
curl -o jquery-3.1.1.js 'https://code.jquery.com/jquery-3.1.1.js'
popd
mkdir -p com.google
pushd com.google
curl -o angular.1.6.2.min.js 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js'
popd
mkdir -p well-known
pushd well-known
ln -snf ../org.oauth3/well-known/oauth3 ./oauth3
popd
popd