CHANGELOG

@@ -1,12 +0,0 @@
-v1.1.5 - Implemented dns-01 ACME challenges
-
-v1.1.4 - Improved responsiveness to config updates
-  * changed which TCP/UDP ports are bound to on config update
-  * update tunnel server settings on config update
-  * update socks5 setting on config update
-
-v1.1.3 - Better late than never... here's some stuff we've got
-  * fixed (probably) network settings not being readable
-  * supports timeouts in loopback check
-  * loopback check less likely to fail / throw errors, will try again
-  * supports ddns using audience of token

LICENSE

@@ -1,41 +0,0 @@
-Copyright 2017 Daplie, Inc
-
-This is open source software; you can redistribute it and/or modify it under the
-terms of either:
-
-   a) the "MIT License"
-   b) the "Apache-2.0 License"
-
-MIT License
-
-   Permission is hereby granted, free of charge, to any person obtaining a copy
-   of this software and associated documentation files (the "Software"), to deal
-   in the Software without restriction, including without limitation the rights
-   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-   copies of the Software, and to permit persons to whom the Software is
-   furnished to do so, subject to the following conditions:
-
-   The above copyright notice and this permission notice shall be included in all
-   copies or substantial portions of the Software.
-
-   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-   SOFTWARE.
-
-Apache-2.0 License Summary
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.

LICENSE.txt

@@ -0,0 +1,3 @@
+Hello all. We make our source code available to view, but we retain copyright.
+
+It's not because we're trying to be mean or anything, we just want to maintain our distribution channel.

README.md

@@ -20,51 +20,17 @@ The node.js netserver that's just right.
 Install Standalone
 -------
 
-### curl | bash
-
 ```bash
-curl -fsSL https://git.coolaj86.com/coolaj86/goldilocks.js/raw/v1.1/installer/get.sh | bash
+# v1 in npm
-```
+npm install -g goldilocks
-
-### git
-
-```bash
-git clone https://git.coolaj86.com/coolaj86/goldilocks.js
-pushd goldilocks.js
-git checkout v1.1
-bash installer/install.sh
-```
-
-### npm
-
-```bash
-# v1 in git (unauthenticated)
-npm install -g git+https://git@git.coolaj86.com:coolaj86/goldilocks.js#v1
 
 # v1 in git (via ssh)
-npm install -g git+ssh://git@git.coolaj86.com:coolaj86/goldilocks.js#v1
+npm install -g git+ssh://git@git.daplie.com:Daplie/goldilocks.js#v1
 
-# v1 in npm
+# v1 in git (unauthenticated)
-npm install -g goldilocks@v1
+npm install -g git+https://git@git.daplie.com:Daplie/goldilocks.js#v1
 ```
 
-### Uninstall
-
-Remove goldilocks and services:
-
-```
-rm -rf /opt/goldilocks/ /srv/goldilocks/ /var/goldilocks/ /var/log/goldilocks/ /etc/tmpfiles.d/goldilocks.conf /etc/systemd/system/goldilocks.service
-```
-
-Remove config as well
-
-```
-rm -rf /etc/goldilocks/ /etc/ssl/goldilocks
-```
-
-Usage
------
-
 ```bash
 goldilocks
 ```
@@ -81,7 +47,7 @@ We have service support for
 * launchd (macOS)
 
 ```bash
-curl https://git.coolaj86.com/coolaj86/goldilocks.js/raw/master/install.sh | bash
+curl https://git.daplie.com/Daplie/goldilocks.js/raw/master/install.sh | bash
 ```
 
 Modules & Configuration
@@ -305,16 +271,9 @@ tls:
       challenge_type: 'http-01'
 ```
 
-**NOTE:** If you specify `dns-01` as the challenge type there must also be a
-[DDNS module](#ddns) defined for all of the relevant domains (though not all
-domains handled by a single TLS module need to be handled by the same DDNS
-module). The DDNS module provides all of the information needed to actually
-set the DNS records needed to verify ownership.
-
 ### tcp
 
-The tcp system handles both *raw* and *tls-terminated* tcp network traffic
+The tcp system handles all tcp network traffic **before decryption** and may use port numbers
-(see the _Note_ section below the example). It may use port numbers
 or traffic sniffing to determine how the connection should be handled.
 
 It has the following options:
@@ -337,83 +296,37 @@ tcp:
       address: '127.0.0.1:2222'
 ```
 
-_Note_: When tcp traffic comes into goldilocks it will be tested against the tcp modules.
-The connection may be handed to the TLS module if it appears to be a TLS/SSL/HTTPS connection
-and if the tls module terminates the traffic, the connection will be sent back to the TLS module.
-Due to the complexity of node.js' networking stack it is not currently possible to tell which
-port tls-terminated traffic came from, so only the SNI header (serername / domain name) may be used for
-modules matching terminated TLS.
-
 ### tcp.proxy
 
-The proxy module routes traffic **after tls-termination** based on the servername (domain name)
+The proxy module routes traffic based on the servername contained in a SNI header.
-contained in a SNI header. As such this only works to route TCP connections wrapped in a TLS stream.
+As such this only works to route TCP connections wrapped in a TLS stream.
 
 It has the same options as the [HTTP proxy module](#httpproxy-how-to-reverse-proxy-ruby-python-etc).
 
-This is particularly useful for routing ssh and vpn traffic over tcp port 443 as wrapped TLS
-connections in order to access one of your servers even when connecting from a harsh or potentially
-misconfigured network environment (i.e. hotspots in public libraries and shopping malls).
-
 Example config:
 ```yml
 tcp:
   modules:
     - type: proxy
       domains:
-        - ssh.example.com      # Note: this domain would also listed in tls.acme.domains
+        - ssh.example.com
-      host: localhost
       port: 22
-    - type: proxy
-      domains:
-        - vpn.example.com      # Note: this domain would also listed in tls.acme.domains
-      host: localhost
-      port: 1194
 ```
 
-_Note_: In same cases network administrators purposefully block ssh and vpn connections using
-Application Firewalls with DPI (deep packet inspection) enabled. You should read the ToS of the
-network you are connected to to ensure that you aren't subverting policies that are purposefully
-in place on such networks.
-
-#### Using with ssh
-
 In order to use this to route SSH connections you will need to use `ssh`'s
-`ProxyCommand` option. For example to use the TLS certificate for `ssh.example.com`
+`ProxyCommand` option. For example to ssh into `ssh.example.com` you could use
-to wrap an ssh connection you could use the following command:
+the following command.
 
 ```bash
-ssh user@example.com -o ProxyCommand='openssl s_client -quiet -connect example.com:443 -servername ssh.example.com'
+ssh user@ssh.example.com -o ProxyCommand='openssl s_client -quiet -connect ssh.example.com:443 -servername ssh.example.com'
 ```
 
 Alternatively you could add the following lines to your ssh config file.
 ```
-Host example.com
+Host ssh.example.com
-  ProxyCommand openssl s_client -quiet -connect example.com:443 -servername ssh.example.com
+  ProxyCommand openssl s_client -quiet -connect ssh.example.com:443 -servername ssh.example.com
 ```
 
-#### Using with OpenVPN
-
-There are two strategies that will work well for you:
-
-1) [Use ssh](https://redfern.me/tunneling-openvpn-through-ssh/) with the config above to reverse proxy tcp port 1194 to you.
-
-```bash
-ssh -L 1194:localhost:1194 example.com
-```
-
-2) [Use stunnel]https://serverfault.com/questions/675553/stunnel-vpn-traffic-and-ensure-it-looks-like-ssl-traffic-on-port-443/681497)
-
-```
-[openvpn-over-goldilocks]
-client = yes
-accept = 127.0.0.1:1194
-sni = vpn.example.com
-connect = example.com:443
-```
-
-3) [Use stunnel.js](https://git.coolaj86.com/coolaj86/tunnel-client.js) as described in the "tunnel_server" section below.
-
 ### tcp.forward
 
 The forward module routes traffic based on port number **without decrypting** it.
@@ -616,7 +529,7 @@ mdns:
 You can discover goldilocks with `mdig`.
 
 ```
-npm install -g git+https://git.coolaj86.com/coolaj86/mdig.js.git
+npm install -g git+https://git.daplie.com/Daplie/mdig.git
 
 mdig _cloud._tcp.local
 ```
@@ -645,7 +558,7 @@ TODO
 * [ ] http - redirect based on domain name (not just path)
 * [ ] tcp - bind should be able to specify localhost, uniquelocal, private, or ip
 * [ ] tcp - if destination host is omitted default to localhost, if dst port is missing, default to src
-* [ ] sys - `curl https://coolaj86.com/goldilocks | bash -s example.com`
+* [ ] sys - `curl https://daplie.me/goldilocks | bash -s example.com`
 * [ ] oauth3 - `example.com/.well-known/domains@oauth3.org/directives.json`
 * [ ] oauth3 - commandline questionnaire
 * [x] modules - use consistent conventions (i.e. address vs host + port)

dist/etc/tmpfiles.d/goldilocks.conf

@@ -1,5 +0,0 @@
-# /etc/tmpfiles.d/goldilocks.conf
-# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
-
-# Type Path           Mode UID      GID      Age Argument
-d /run/goldilocks     0755 MY_USER  MY_GROUP -   -

etc/systemd/system/goldilocks.service

@@ -19,14 +19,14 @@ StartLimitBurst=3
 
 # User and group the process will run as
 # (www-data is the de facto standard on most systems)
-User=MY_USER
+User=www-data
-Group=MY_GROUP
+Group=www-data
 
 # If we need to pass environment variables in the future
 Environment=GOLDILOCKS_PATH=/srv/www NODE_PATH=/opt/goldilocks/lib/node_modules NPM_CONFIG_PREFIX=/opt/goldilocks
 
 # Set a sane working directory, sane flags, and specify how to reload the config file
-WorkingDirectory=/opt/goldilocks
+WorkingDirectory=/srv/www
 ExecStart=/opt/goldilocks/bin/node /opt/goldilocks/bin/goldilocks --config /etc/goldilocks/goldilocks.yml
 ExecReload=/bin/kill -USR1 $MAINPID
 
@@ -46,7 +46,7 @@ ProtectSystem=full
 # … except TLS/SSL, ACME, and Let's Encrypt certificates
 #   and /var/log/goldilocks, because we want a place where logs can go.
 #   This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
-ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks /opt/goldilocks
+ReadWriteDirectories=/etc/goldilocks /etc/ssl /srv/www /var/log/goldilocks
 # you may also want to add other directories such as /opt/goldilocks /etc/acme /etc/letsencrypt
 
 # Note: in v231 and above ReadWritePaths has been renamed to ReadWriteDirectories

etc/tmpfiles.d/goldilocks.conf

@@ -0,0 +1,10 @@
+# /etc/tmpfiles.d/goldilocks.conf
+# See https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
+
+# Type Path           Mode UID      GID      Age Argument
+d /etc/goldilocks          0755 www-data www-data -   -
+d /opt/goldilocks          0775 www-data www-data -   -
+d /srv/www                 0775 www-data www-data -   -
+d /etc/ssl/goldilocks      0750 www-data www-data -   -
+d /var/log/goldilocks      0750 www-data www-data -   -
+#d /run/goldilocks          0755 www-data www-data -   -

install.sh

@@ -0,0 +1,224 @@
+#!/bin/bash
+
+# something or other about android and tmux using PREFIX
+#: "${PREFIX:=''}"
+MY_ROOT=""
+if [ -z "${PREFIX-}" ]; then
+  MY_ROOT=""
+else
+  MY_ROOT="$PREFIX"
+fi
+# Not every platform has or needs sudo, gotta save them O(1)s...
+sudo_cmd=""
+((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
+
+###############################
+#                             #
+#         http_get            #
+# boilerplate for curl / wget #
+#                             #
+###############################
+
+# See https://git.daplie.com/Daplie/daplie-snippets/blob/master/bash/http-get.sh
+
+http_get=""
+http_opts=""
+http_out=""
+
+detect_http_get()
+{
+  if type -p curl >/dev/null 2>&1; then
+    http_get="curl"
+    http_opts="-fsSL"
+    http_out="-o"
+  elif type -p wget >/dev/null 2>&1; then
+    http_get="wget"
+    http_opts="--quiet"
+    http_out="-O"
+  else
+    echo "Aborted, could not find curl or wget"
+    return 7
+  fi
+}
+
+dap_dl()
+{
+  $http_get $http_opts $http_out "$2" "$1"
+  touch "$2"
+}
+
+dap_dl_bash()
+{
+  dap_url=$1
+  #dap_args=$2
+  rm -rf dap-tmp-runner.sh
+  $http_get $http_opts $http_out dap-tmp-runner.sh "$dap_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
+}
+
+detect_http_get
+
+## END HTTP_GET ##
+
+
+
+###################
+#                 #
+# Install service #
+#                 #
+###################
+
+my_app_name=goldilocks
+my_app_pkg_name=com.daplie.goldilocks.web
+my_app_dir=$(mktemp -d)
+installer_base="https://git.daplie.com/Daplie/goldilocks.js/raw/master"
+
+my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
+my_app_etc_example_config="etc/${my_app_name}/${my_app_name}.example.yml"
+my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
+my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
+my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
+
+install_for_systemd()
+{
+  echo ""
+  echo "Installing as systemd service"
+  echo ""
+  mkdir -p $(dirname "$my_app_dir/$my_app_systemd_service")
+  dap_dl "$installer_base/$my_app_systemd_service" "$my_app_dir/$my_app_systemd_service"
+  $sudo_cmd mv "$my_app_dir/$my_app_systemd_service" "$MY_ROOT/$my_app_systemd_service"
+  $sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_service"
+  $sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_service"
+
+  mkdir -p $(dirname "$my_app_dir/$my_app_systemd_tmpfiles")
+  dap_dl "$installer_base/$my_app_systemd_tmpfiles" "$my_app_dir/$my_app_systemd_tmpfiles"
+  $sudo_cmd mv "$my_app_dir/$my_app_systemd_tmpfiles" "$MY_ROOT/$my_app_systemd_tmpfiles"
+  $sudo_cmd chown -R root:root "$MY_ROOT/$my_app_systemd_tmpfiles"
+  $sudo_cmd chmod 644 "$MY_ROOT/$my_app_systemd_tmpfiles"
+
+  $sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null
+  $sudo_cmd systemctl daemon-reload
+  $sudo_cmd systemctl start "${my_app_name}.service"
+  $sudo_cmd systemctl enable "${my_app_name}.service"
+
+  echo "$my_app_name started with systemctl, check its status like so"
+  echo "  $sudo_cmd systemctl status $my_app_name"
+  echo "  $sudo_cmd journalctl -xe -u goldilocks"
+}
+
+install_for_launchd()
+{
+  echo ""
+  echo "Installing as launchd service"
+  echo ""
+  # See http://www.launchd.info/
+  mkdir -p $(dirname "$my_app_dir/$my_app_launchd_service")
+  dap_dl "$installer_base/$my_app_launchd_service" "$my_app_dir/$my_app_launchd_service"
+  $sudo_cmd mv "$my_app_dir/$my_app_launchd_service" "$MY_ROOT/$my_app_launchd_service"
+  $sudo_cmd chown root:wheel "$MY_ROOT/$my_app_launchd_service"
+  $sudo_cmd chmod 0644 "$MY_ROOT/$my_app_launchd_service"
+  $sudo_cmd launchctl unload -w "$MY_ROOT/$my_app_launchd_service" >/dev/null 2>/dev/null
+  $sudo_cmd launchctl load -w "$MY_ROOT/$my_app_launchd_service"
+
+  echo "$my_app_name started with launchd"
+}
+
+install_etc_config()
+{
+  $sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_example_config")
+  mkdir -p $(dirname "$my_app_dir/$my_app_etc_example_config")
+  dap_dl "$installer_base/$my_app_etc_example_config" "$my_app_dir/$my_app_etc_example_config"
+  $sudo_cmd mv "$my_app_dir/$my_app_etc_example_config" "$MY_ROOT/$my_app_etc_example_config"
+
+  if [ ! -e "$MY_ROOT/$my_app_etc_config" ]; then
+    $sudo_cmd mkdir -p $(dirname "$MY_ROOT/$my_app_etc_config")
+    mkdir -p $(dirname "$my_app_dir/$my_app_etc_config")
+    dap_dl "$installer_base/$my_app_etc_config" "$my_app_dir/$my_app_etc_config"
+    $sudo_cmd mv "$my_app_dir/$my_app_etc_config" "$MY_ROOT/$my_app_etc_config"
+  fi
+
+  # OS X
+  $sudo_cmd chown -R _www:_www $(dirname "$MY_ROOT/$my_app_etc_config") || true
+  # Linux
+  $sudo_cmd chown -R www-data:www-data $(dirname "$MY_ROOT/$my_app_etc_config") || true
+  $sudo_cmd chmod 775 $(dirname "$MY_ROOT/$my_app_etc_config")
+  $sudo_cmd chmod 664 "$MY_ROOT/$my_app_etc_config"
+}
+
+install_service()
+{
+  install_etc_config
+
+  installable=""
+  if [ -d "$MY_ROOT/etc/systemd/system" ]; then
+    install_for_systemd
+    installable="true"
+  fi
+  if [ -d "/Library/LaunchDaemons" ]; then
+    install_for_launchd
+    installable="true"
+  fi
+  if [ -z "$installable" ]; then
+    echo ""
+    echo "Unknown system service init type. You must install as a system service manually."
+    echo '(please file a bug with the output of "uname -a")'
+    echo ""
+  fi
+}
+
+## END SERVICE_INSTALL ##
+
+
+
+set -e
+set -u
+
+# Install
+# TODO install to tmp location, then move to /opt
+export NODE_PATH=/opt/goldilocks/lib/node_modules
+export NPM_CONFIG_PREFIX=/opt/goldilocks
+$sudo_cmd mkdir -p /etc/goldilocks
+$sudo_cmd mkdir -p /var/log/goldilocks
+$sudo_cmd mkdir -p /srv/www
+$sudo_cmd mkdir -p /var/www
+$sudo_cmd mkdir -p /opt/goldilocks/{lib,bin,etc}
+
+# Dependencies
+dap_dl_bash "https://git.daplie.com/Daplie/node-install-script/raw/master/setup-min.sh"
+
+# Change to user perms
+
+# OS X or Linux
+$sudo_cmd chown -R $(whoami) /opt/goldilocks/ || true
+
+my_npm="$NPM_CONFIG_PREFIX/bin/npm"
+$my_npm install -g npm@4
+$my_npm install -g 'git+https://git@git.daplie.com/Daplie/goldilocks.js.git'
+# Finish up with submodule
+pushd /opt/goldilocks/lib/node_modules/goldilocks
+bash ./update-packages.sh
+popd
+
+# Change to admin perms
+
+# OS X
+$sudo_cmd chown -R _www:_www /var/www /srv/www /opt/goldilocks || true
+# Linux
+$sudo_cmd chown -R www-data:www-data /var/www /srv/www /opt/goldilocks || true
+
+# make sure the files are all read/write for the owner and group, and then set
+# the setuid and setgid bits so that any files/directories created inside these
+# directories have the same owner and group.
+$sudo_cmd chmod -R ug+rwX /opt/goldilocks
+find /opt/goldilocks -type d -exec $sudo_cmd chmod ug+s {} \;
+
+# Uninstall
+dap_dl "https://git.daplie.com/Daplie/goldilocks.js/raw/master/uninstall.sh" "./goldilocks-uninstall"
+$sudo_cmd chmod 755 "./goldilocks-uninstall"
+# OS X
+$sudo_cmd chown root:wheel "./goldilocks-uninstall" || true
+# Linux
+$sudo_cmd chown root:root "./goldilocks-uninstall" || true
+$sudo_cmd mv "./goldilocks-uninstall" "/usr/local/bin/uninstall-goldilocks"
+
+# Install Service
+install_service

installer/get.sh

@@ -1,20 +0,0 @@
-set -e
-set -u
-
-my_name=goldilocks
-# TODO provide an option to supply my_ver and my_tmp
-my_ver=master
-my_tmp=$(mktemp -d)
-
-mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
-git clone https://git.coolaj86.com/coolaj86/goldilocks.js.git $my_tmp/opt/$my_name/lib/node_modules/$my_name
-
-echo "Installing to $my_tmp (will be moved after install)"
-pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
-  git checkout $my_ver
-  source ./installer/install.sh
-popd
-
-echo "Installation successful, now cleaning up $my_tmp ..."
-rm -rf $my_tmp
-echo "Done"

installer/http-get.sh

@@ -1,48 +0,0 @@
-###############################
-#                             #
-#         http_get            #
-# boilerplate for curl / wget #
-#                             #
-###############################
-
-# See https://git.coolaj86.com/coolaj86/snippets/blob/master/bash/http-get.sh
-
-_h_http_get=""
-_h_http_opts=""
-_h_http_out=""
-
-detect_http_get()
-{
-  set +e
-  if type -p curl >/dev/null 2>&1; then
-    _h_http_get="curl"
-    _h_http_opts="-fsSL"
-    _h_http_out="-o"
-  elif type -p wget >/dev/null 2>&1; then
-    _h_http_get="wget"
-    _h_http_opts="--quiet"
-    _h_http_out="-O"
-  else
-    echo "Aborted, could not find curl or wget"
-    return 7
-  fi
-  set -e
-}
-
-http_get()
-{
-  $_h_http_get $_h_http_opts $_h_http_out "$2" "$1"
-  touch "$2"
-}
-
-http_bash()
-{
-  _http_url=$1
-  #dap_args=$2
-  rm -rf dap-tmp-runner.sh
-  $_h_http_get $_h_http_opts $_h_http_out dap-tmp-runner.sh "$_http_url"; bash dap-tmp-runner.sh; rm dap-tmp-runner.sh
-}
-
-detect_http_get
-
-## END HTTP_GET ##

installer/install-for-launchd.sh

@@ -1,17 +0,0 @@
-set -u
-
-my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
-
-echo ""
-echo "Installing as launchd service"
-echo ""
-
-# See http://www.launchd.info/
-safe_copy_config "$my_app_dist/$my_app_launchd_service" "$my_root/$my_app_launchd_service"
-
-$sudo_cmd chown root:wheel "$my_root/$my_app_launchd_service"
-
-$sudo_cmd launchctl unload -w "$my_root/$my_app_launchd_service" >/dev/null 2>/dev/null
-$sudo_cmd launchctl load -w "$my_root/$my_app_launchd_service"
-
-echo "$my_app_name started with launchd"

installer/install-for-systemd.sh

@@ -1,37 +0,0 @@
-set -u
-
-my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
-my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
-
-echo ""
-echo "Installing as systemd service"
-echo ""
-
-sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_service" > "$my_app_dist/$my_app_systemd_service.2"
-sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_service.2" > "$my_app_dist/$my_app_systemd_service"
-rm "$my_app_dist/$my_app_systemd_service.2"
-safe_copy_config "$my_app_dist/$my_app_systemd_service" "$my_root/$my_app_systemd_service"
-$sudo_cmd chown root:root "$my_root/$my_app_systemd_service"
-
-sed "s/MY_USER/$my_user/g" "$my_app_dist/$my_app_systemd_tmpfiles" > "$my_app_dist/$my_app_systemd_tmpfiles.2"
-sed "s/MY_GROUP/$my_group/g" "$my_app_dist/$my_app_systemd_tmpfiles.2" > "$my_app_dist/$my_app_systemd_tmpfiles"
-rm "$my_app_dist/$my_app_systemd_tmpfiles.2"
-safe_copy_config "$my_app_dist/$my_app_systemd_tmpfiles" "$my_root/$my_app_systemd_tmpfiles"
-$sudo_cmd chown root:root "$my_root/$my_app_systemd_tmpfiles"
-
-$sudo_cmd systemctl stop "${my_app_name}.service" >/dev/null 2>/dev/null || true
-$sudo_cmd systemctl daemon-reload
-$sudo_cmd systemctl start "${my_app_name}.service"
-$sudo_cmd systemctl enable "${my_app_name}.service"
-
-echo ""
-echo ""
-echo "Fun systemd commands to remember:"
-echo "  $sudo_cmd systemctl daemon-reload"
-echo "  $sudo_cmd systemctl restart $my_app_name.service"
-echo ""
-echo "$my_app_name started with systemctl, check its status like so:"
-echo "  $sudo_cmd systemctl status $my_app_name"
-echo "  $sudo_cmd journalctl -xefu $my_app_name"
-echo ""
-echo ""

installer/install-system-service.sh

@@ -1,37 +0,0 @@
-safe_copy_config()
-{
-  src=$1
-  dst=$2
-  $sudo_cmd mkdir -p $(dirname "$dst")
-  if [ -f "$dst" ]; then
-    $sudo_cmd rsync -a "$src" "$dst.latest"
-    # TODO edit config file with $my_user and $my_group
-    if [ "$(cat $dst)" == "$(cat $dst.latest)" ]; then
-      $sudo_cmd rm $dst.latest
-    else
-      echo "MANUAL INTERVENTION REQUIRED: check the systemd script update and manually decide what you want to do"
-      echo "diff $dst $dst.latest"
-      $sudo_cmd chown -R root:root "$dst.latest"
-    fi
-  else
-    $sudo_cmd rsync -a --ignore-existing "$src" "$dst"
-  fi
-  $sudo_cmd chown -R root:root "$dst"
-  $sudo_cmd chmod 644 "$dst"
-}
-
-installable=""
-if [ -d "$my_root/etc/systemd/system" ]; then
-  source ./installer/install-for-systemd.sh
-  installable="true"
-fi
-if [ -d "/Library/LaunchDaemons" ]; then
-  source ./installer/install-for-launchd.sh
-  installable="true"
-fi
-if [ -z "$installable" ]; then
-  echo ""
-  echo "Unknown system service init type. You must install as a system service manually."
-  echo '(please file a bug with the output of "uname -a")'
-  echo ""
-fi

installer/install.sh

@@ -1,150 +0,0 @@
-#!/bin/bash
-
-set -e
-set -u
-
-
-### IMPORTANT ###
-###  VERSION  ###
-my_name=goldilocks
-my_app_pkg_name=com.coolaj86.goldilocks.web
-my_app_ver="v1.1"
-my_azp_oauth3_ver="v1.2.3"
-export NODE_VERSION="v8.9.3"
-
-if [ -z "${my_tmp-}" ]; then
-  my_tmp="$(mktemp -d)"
-  mkdir -p $my_tmp/opt/$my_name/lib/node_modules/$my_name
-  echo "Installing to $my_tmp (will be moved after install)"
-  git clone ./ $my_tmp/opt/$my_name/lib/node_modules/$my_name
-  pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
-fi
-
-#################
-export NODE_PATH=$my_tmp/opt/$my_name/lib/node_modules
-export PATH=$my_tmp/opt/$my_name/bin/:$PATH
-export NPM_CONFIG_PREFIX=$my_tmp/opt/$my_name
-my_npm="$NPM_CONFIG_PREFIX/bin/npm"
-#################
-
-
-my_app_dist=$my_tmp/opt/$my_name/lib/node_modules/$my_name/dist
-installer_base="https://git.coolaj86.com/coolaj86/goldilocks.js/raw/$my_app_ver"
-
-# Backwards compat
-# some scripts still use the old names
-my_app_dir=$my_tmp
-my_app_name=$my_name
-
-
-
-git checkout $my_app_ver
-
-mkdir -p "$my_tmp/opt/$my_name"/{lib,bin,etc}
-ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name
-ln -s ../lib/node_modules/$my_name/bin/$my_name.js $my_tmp/opt/$my_name/bin/$my_name.js
-mkdir -p "$my_tmp/etc/$my_name"
-chmod 775 "$my_tmp/etc/$my_name"
-cat "$my_app_dist/etc/$my_name/$my_name.example.yml" > "$my_tmp/etc/$my_name/$my_name.example.yml"
-chmod 664 "$my_tmp/etc/$my_name/$my_name.example.yml"
-mkdir -p $my_tmp/srv/www
-mkdir -p $my_tmp/var/www
-mkdir -p $my_tmp/var/log/$my_name
-
-
-
-#
-# Helpers
-#
-source ./installer/sudo-cmd.sh
-source ./installer/http-get.sh
-
-
-
-#
-# Dependencies
-#
-echo $NODE_VERSION > /tmp/NODEJS_VER
-http_bash "https://git.coolaj86.com/coolaj86/node-installer.sh/raw/v1.1/install.sh"
-$my_npm install -g npm@4
-pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name
-  $my_npm install
-popd
-pushd $my_tmp/opt/$my_name/lib/node_modules/$my_name/packages/assets
-  OAUTH3_GIT_URL="https://git.oauth3.org/OAuth3/oauth3.js.git"
-  git clone ${OAUTH3_GIT_URL} oauth3.org || true
-  ln -s oauth3.org org.oauth3
-  pushd oauth3.org
-    git remote set-url origin ${OAUTH3_GIT_URL}
-    git checkout $my_azp_oauth3_ver
-    #git pull
-  popd
-
-  mkdir -p jquery.com
-  ln -s jquery.com com.jquery
-  pushd jquery.com
-    http_get 'https://code.jquery.com/jquery-3.1.1.js' jquery-3.1.1.js
-  popd
-
-  mkdir -p google.com
-  ln -s google.com com.google
-  pushd google.com
-    http_get 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js' angular.1.6.2.min.js
-  popd
-
-  mkdir -p well-known
-  ln -s well-known .well-known
-  pushd well-known
-    ln -snf ../oauth3.org/well-known/oauth3 ./oauth3
-  popd
-  echo "installed dependencies"
-popd
-
-
-
-#
-# System Service
-#
-source ./installer/my-root.sh
-echo "Pre-installation to $my_tmp complete, now installing to $my_root/ ..."
-set +e
-if type -p tree >/dev/null 2>/dev/null; then
-  #tree -I "node_modules|include|share" $my_tmp
-  tree -L 6 -I "include|share|npm" $my_tmp
-else
-  ls $my_tmp
-fi
-set -e
-
-source ./installer/my-user-my-group.sh
-echo "User $my_user Group $my_group"
-
-source ./installer/install-system-service.sh
-
-$sudo_cmd chown -R $my_user:$my_group $my_tmp/*
-$sudo_cmd chown root:root $my_tmp/*
-$sudo_cmd chown root:root $my_tmp
-$sudo_cmd chmod 0755 $my_tmp
-# don't change permissions on /, /etc, etc
-$sudo_cmd rsync -a --ignore-existing $my_tmp/ $my_root/
-$sudo_cmd rsync -a --ignore-existing $my_app_dist/etc/$my_name/$my_name.yml $my_root/etc/$my_name/$my_name.yml
-
-# Change to admin perms
-$sudo_cmd chown -R $my_user:$my_group $my_root/opt/$my_name
-$sudo_cmd chown -R $my_user:$my_group $my_root/var/www $my_root/srv/www
-
-# make sure the files are all read/write for the owner and group, and then set
-# the setuid and setgid bits so that any files/directories created inside these
-# directories have the same owner and group.
-$sudo_cmd chmod -R ug+rwX $my_root/opt/$my_name
-find $my_root/opt/$my_name -type d -exec $sudo_cmd chmod ug+s {} \;
-
-
-
-echo ""
-echo "$my_name installation complete!"
-echo ""
-echo ""
-echo "Update the config at: /etc/$my_name/$my_name.yml"
-echo ""
-echo "Unistall: rm -rf /srv/$my_name/ /var/$my_name/ /etc/$my_name/ /opt/$my_name/ /var/log/$my_name/ /etc/tmpfiles.d/$my_name.conf /etc/systemd/system/$my_name.service /etc/ssl/$my_name"

installer/my-root.sh

@@ -1,8 +0,0 @@
-# something or other about android and tmux using PREFIX
-#: "${PREFIX:=''}"
-my_root=""
-if [ -z "${PREFIX-}" ]; then
-  my_root=""
-else
-  my_root="$PREFIX"
-fi

installer/my-user-my-group.sh

@@ -1,19 +0,0 @@
-if type -p adduser >/dev/null 2>/dev/null; then
-  if [ -z "$(cat $my_root/etc/passwd | grep $my_app_name)" ]; then
-    $sudo_cmd adduser --home $my_root/opt/$my_app_name --gecos '' --disabled-password $my_app_name
-  fi
-  my_user=$my_app_name
-  my_group=$my_app_name
-elif [ -n "$(cat /etc/passwd | grep www-data:)" ]; then
-  # Linux (Ubuntu)
-  my_user=www-data
-  my_group=www-data
-elif [ -n "$(cat /etc/passwd | grep _www:)" ]; then
-  # Mac
-  my_user=_www
-  my_group=_www
-else
-  # Unsure
-  my_user=$(whoami)
-  my_group=$(id -g -n)
-fi

installer/sudo-cmd.sh

@@ -1,7 +0,0 @@
-# Not every platform has or needs sudo, gotta save them O(1)s...
-sudo_cmd=""
-set +e
-if type -p sudo >/dev/null 2>/dev/null; then
-  ((EUID)) && [[ -z "${ANDROID_ROOT-}" ]] && sudo_cmd="sudo"
-fi
-set -e

lib/ddns/challenge-responder.js

@@ -1,122 +0,0 @@
-'use strict';
-
-// Much of this file was based on the `le-challenge-ddns` library (which we are not using
-// here because it's method of setting records requires things we don't really want).
-module.exports.create = function (deps, conf, utils) {
-
-  function getReleventSessionId(domain) {
-    var sessId;
-
-    utils.iterateAllModules(function (mod, domainList) {
-      // We return a truthy value in these cases because of the way the iterate function
-      // handles modules grouped by domain. By returning true we are saying these domains
-      // are "handled" and so if there are multiple modules we won't be given the rest.
-      if (sessId) { return true; }
-      if (domainList.indexOf(domain) < 0) { return true; }
-
-      // But if the domains are relevant but we don't know how to handle the module we
-      // return false to allow us to look at any other modules that might exist here.
-      if (mod.type !== 'dns@oauth3.org')  { return false; }
-
-      sessId = mod.tokenId || mod.token_id;
-      return true;
-    });
-
-    return sessId;
-  }
-
-  function get(args, domain, challenge, done) {
-    done(new Error("Challenge.get() does not need an implementation for dns-01. (did you mean Challenge.loopback?)"));
-  }
-  // same as get, but external
-  function loopback(args, domain, challenge, done) {
-    var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
-    require('dns').resolveTxt(challengeDomain, done);
-  }
-
-  var activeChallenges = {};
-  async function removeAsync(args, domain) {
-    var data = activeChallenges[domain];
-    if (!data) {
-      console.warn(new Error('cannot remove DNS challenge for ' + domain + ': already removed'));
-      return;
-    }
-
-    var session = await utils.getSession(data.sessId);
-    var directives = await deps.OAUTH3.discover(session.token.aud);
-    var apiOpts = {
-      api: 'dns.unset'
-    , session: session
-    , type: 'TXT'
-    , value: data.keyAuthDigest
-    };
-    await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, data.splitDomain));
-
-    delete activeChallenges[domain];
-  }
-  async function setAsync(args, domain, challenge, keyAuth) {
-    if (activeChallenges[domain]) {
-      await removeAsync(args, domain, challenge);
-    }
-
-    var sessId = getReleventSessionId(domain);
-    if (!sessId) {
-      throw new Error('no DDNS module handles the domain ' + domain);
-    }
-    var session = await utils.getSession(sessId);
-    var directives = await deps.OAUTH3.discover(session.token.aud);
-
-    // I'm not sure what role challenge is supposed to play since even in the library
-    // this code is based on it was never used, but check for it anyway because ...
-    if (!challenge || keyAuth) {
-      console.warn(new Error('DDNS challenge missing challenge or keyAuth'));
-    }
-    var keyAuthDigest = require('crypto').createHash('sha256').update(keyAuth || '').digest('base64')
-      .replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
-
-    var challengeDomain = (args.test || '') + args.acmeChallengeDns + domain;
-    var splitDomain = (await utils.splitDomains(directives.api, [challengeDomain]))[0];
-
-    var apiOpts = {
-      api: 'dns.set'
-    , session: session
-    , type: 'TXT'
-    , value: keyAuthDigest
-    , ttl: args.ttl || 0
-    };
-    await deps.OAUTH3.api(directives.api, Object.assign({}, apiOpts, splitDomain));
-
-    activeChallenges[domain] = {
-      sessId
-    , keyAuthDigest
-    , splitDomain
-    };
-
-    return new Promise(res => setTimeout(res, 1000));
-  }
-
-  // It might be slightly easier to use arguments and apply, but the library that will use
-  // this function counts the arguments we expect.
-  function set(a, b, c, d, done) {
-    setAsync(a, b, c, d).then(result => done(null, result), done);
-  }
-  function remove(a, b, c, done) {
-    removeAsync(a, b, c).then(result => done(null, result), done);
-  }
-
-  function getOptions() {
-    return {
-      oauth3: 'oauth3.org'
-    , debug: conf.debug
-    , acmeChallengeDns: '_acme-challenge.'
-    };
-  }
-
-  return {
-    getOptions
-  , set
-  , get
-  , remove
-  , loopback
-  };
-};

lib/ddns/dns-ctrl.js

@@ -1,6 +1,6 @@
 'use strict';
 
-module.exports.create = function (deps, conf, utils) {
+module.exports.create = function (deps, conf) {
   function dnsType(addr) {
     if (/^\d+\.\d+\.\d+\.\d+$/.test(addr)) {
       return 'A';
@@ -10,6 +10,62 @@ module.exports.create = function (deps, conf, utils) {
     }
   }
 
+  var tldCache = {};
+  async function getTlds(provider) {
+    async function updateCache() {
+      var reqObj = {
+        url: deps.OAUTH3.url.normalize(provider)+'/api/com.daplie.domains/prices'
+      , method: 'GET'
+      , json: true
+      };
+
+      var resp = await deps.OAUTH3.request(reqObj);
+      var tldObj = {};
+      resp.data.forEach(function (tldInfo) {
+        if (tldInfo.enabled) {
+          tldObj[tldInfo.tld] = true;
+        }
+      });
+
+      tldCache[provider] = {
+        time: Date.now()
+      , tlds: tldObj
+      };
+      return tldObj;
+    }
+
+    // If we've never cached the results we need to return the promise that will fetch the recult,
+    // otherwise we can return the cached value. If the cached value has "expired", we can still
+    // return the cached value we just want to update the cache in parellel (making sure we only
+    // update once).
+    if (!tldCache[provider]) {
+      return updateCache();
+    }
+    if (!tldCache[provider].updating && Date.now() - tldCache[provider].time > 24*60*60*1000) {
+      tldCache[provider].updating = true;
+      updateCache();
+    }
+
+    return tldCache[provider].tlds;
+  }
+
+  async function splitDomains(provider, domains) {
+    var tlds = await getTlds(provider);
+    return domains.map(function (domain) {
+      var split = domain.split('.');
+      var tldSegCnt = tlds[split.slice(-2).join('.')] ? 2 : 1;
+
+      // Currently assuming that the sld can't contain dots, and that the tld can have at
+      // most one dot. Not 100% sure this is a valid assumption, but exceptions should be
+      // rare even if the assumption isn't valid.
+      return {
+        tld: split.slice(-tldSegCnt).join('.')
+      , sld: split.slice(-tldSegCnt-1, -tldSegCnt).join('.')
+      , sub: split.slice(0, -tldSegCnt-1).join('.')
+      };
+    });
+  }
+
   async function setDeviceAddress(session, addr, domains) {
     var directives = await deps.OAUTH3.discover(session.token.aud);
 
@@ -52,10 +108,10 @@ module.exports.create = function (deps, conf, utils) {
       return record.value === addr && badAddrDomains.indexOf(record.host) < 0;
     }).map(record => record.host);
     var requiredUpdates = domains.filter(function (domain) {
-      return goodAddrDomains.indexOf(domain) < 0;
+      return goodAddrDomains.indexOf(domain) !== -1;
     });
 
-    var oldDns = await utils.splitDomains(directives.api, badAddrDomains);
+    var oldDns = await splitDomains(directives.api, badAddrDomains);
     var common = {
       api: 'devices.detach'
     , session: session
@@ -64,11 +120,8 @@ module.exports.create = function (deps, conf, utils) {
     await deps.PromiseA.all(oldDns.map(function (record) {
       return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
     }));
-    if (conf.debug && badAddrDomains.length) {
-      console.log('removed bad DNS records for ' + badAddrDomains.join(', '));
-    }
 
-    var newDns = await utils.splitDomains(directives.api, requiredUpdates);
+    var newDns = await splitDomains(directives.api, requiredUpdates);
     common = {
       api: 'devices.attach'
     , session: session
@@ -79,9 +132,6 @@ module.exports.create = function (deps, conf, utils) {
     await deps.PromiseA.all(newDns.map(function (record) {
       return deps.OAUTH3.api(directives.api, Object.assign({}, common, record));
     }));
-    if (conf.debug && requiredUpdates.length) {
-      console.log('set new DNS records for ' + requiredUpdates.join(', '));
-    }
   }
 
   async function getDeviceAddresses(session) {
@@ -113,7 +163,7 @@ module.exports.create = function (deps, conf, utils) {
   async function removeDomains(session, domains) {
     var directives = await deps.OAUTH3.discover(session.token.aud);
 
-    var oldDns = await utils.splitDomains(directives.api, domains);
+    var oldDns = await splitDomains(directives.api, domains);
     var common = {
       api: 'devices.detach'
     , session: session

lib/ddns/index.js

@@ -3,20 +3,48 @@
 module.exports.create = function (deps, conf) {
   var dns = deps.PromiseA.promisifyAll(require('dns'));
   var network = deps.PromiseA.promisifyAll(deps.recase.camelCopy(require('network')));
+  var loopback = require('./loopback').create(deps, conf);
+  var dnsCtrl = require('./dns-ctrl').create(deps, conf);
+  var tunnelClients = require('./tunnel-client-manager').create(deps, conf);
   var equal = require('deep-equal');
 
-  var utils = require('./utils').create(deps, conf);
-  var loopback = require('./loopback').create(deps, conf, utils);
-  var dnsCtrl = require('./dns-ctrl').create(deps, conf, utils);
-  var challenge = require('./challenge-responder').create(deps, conf, utils);
-  var tunnelClients = require('./tunnel-client-manager').create(deps, conf, utils);
-
   var loopbackDomain;
 
+  function iterateAllModules(action, curConf) {
+    curConf = curConf || conf;
+    var promises = curConf.ddns.modules.map(function (mod) {
+      return action(mod, mod.domains);
+    });
+
+    curConf.domains.forEach(function (dom) {
+      if (!dom.modules || !Array.isArray(dom.modules.ddns) || !dom.modules.ddns.length) {
+        return null;
+      }
+
+      // For the time being all of our things should only be tried once (regardless if it succeeded)
+      // TODO: revisit this behavior when we support multiple ways of setting records, and/or
+      // if we want to allow later modules to run if early modules fail.
+      promises.push(dom.modules.ddns.reduce(function (prom, mod) {
+        if (prom) { return prom; }
+        return action(mod, dom.names);
+      }, null));
+    });
+
+    return deps.PromiseA.all(promises.filter(Boolean));
+  }
+
+  async function getSession(id) {
+    var session = await deps.storage.tokens.get(id);
+    if (!session) {
+      throw new Error('no user token with ID "'+id+'"');
+    }
+    return session;
+  }
+
   var tunnelActive = false;
   async function startTunnel(tunnelSession, mod, domainList) {
     try {
-      var dnsSession = await utils.getSession(mod.tokenId);
+      var dnsSession = await getSession(mod.tokenId);
       var tunnelDomain = await tunnelClients.start(tunnelSession || dnsSession, domainList);
 
       var addrList;
@@ -32,9 +60,7 @@ module.exports.create = function (deps, conf) {
         throw new Error('failed to lookup IP for tunnel domain "' + tunnelDomain + '"');
       }
 
-      if (!mod.disabled) {
+      await dnsCtrl.setDeviceAddress(dnsSession, addrList[0], domainList);
-        await dnsCtrl.setDeviceAddress(dnsSession, addrList[0], domainList);
-      }
     } catch (err) {
       console.log('error starting tunnel for', domainList.join(', '));
       console.log(err);
@@ -48,7 +74,7 @@ module.exports.create = function (deps, conf) {
       tunnelSession = await deps.storage.tokens.get(conf.ddns.tunnel.tokenId);
     }
 
-    await utils.iterateAllModules(function (mod, domainList) {
+    await iterateAllModules(function (mod, domainList) {
       if (mod.type !== 'dns@oauth3.org') { return null; }
 
       return startTunnel(tunnelSession, mod, domainList);
@@ -64,7 +90,7 @@ module.exports.create = function (deps, conf) {
   async function checkTunnelTokens() {
     var oldTokens = tunnelClients.current();
 
-    var newTokens = await utils.iterateAllModules(function checkTokens(mod, domainList) {
+    var newTokens = await iterateAllModules(function checkTokens(mod, domainList) {
       if (mod.type !== 'dns@oauth3.org') { return null; }
 
       var domainStr = domainList.slice().sort().join(',');
@@ -99,17 +125,8 @@ module.exports.create = function (deps, conf) {
     // Since we can't detect the OS level events when a user plugs in an ethernet cable to recheck
     // what network environment we are in we check our local network address and the gateway to
     // determine if we need to run the loopback check and router configuration again.
+    var gw = await network.getGatewayIpAsync();
     var addr = await network.getPrivateIpAsync();
-    // Until the author of the `network` package publishes the pull request we gave him
-    // checking the gateway on our units fails because we have the busybox versions of
-    // the linux commands. Gateway is realistically less important than address, so if
-    // we fail in getting it go ahead and use the null value.
-    var gw;
-    try {
-      gw = await network.getGatewayIpAsync();
-    } catch (err) {
-      gw = null;
-    }
     if (localAddr === addr && gateway === gw) {
       return;
     }
@@ -162,10 +179,10 @@ module.exports.create = function (deps, conf) {
     }
     publicAddress = addr;
 
-    await utils.iterateAllModules(function setModuleDNS(mod, domainList) {
+    await iterateAllModules(function setModuleDNS(mod, domainList) {
       if (mod.type !== 'dns@oauth3.org' || mod.disabled) { return null; }
 
-      return utils.getSession(mod.tokenId).then(function (session) {
+      return getSession(mod.tokenId).then(function (session) {
         return dnsCtrl.setDeviceAddress(session, addr, domainList);
       }).catch(function (err) {
         console.log('error setting DNS records for', domainList.join(', '));
@@ -180,13 +197,13 @@ module.exports.create = function (deps, conf) {
 
     // this returns a Promise, but since the functions we use are synchronous
     // and change our enclosed variables we don't need to wait for the return.
-    utils.iterateAllModules(function (mod, domainList) {
+    iterateAllModules(function (mod, domainList) {
       if (mod.type !== 'dns@oauth3.org') { return; }
 
       prevMods[mod.id] = { mod, domainList };
       return true;
     }, prevConf);
-    utils.iterateAllModules(function (mod, domainList) {
+    iterateAllModules(function (mod, domainList) {
       if (mod.type !== 'dns@oauth3.org') { return; }
 
       curMods[mod.id] = { mod, domainList };
@@ -209,11 +226,8 @@ module.exports.create = function (deps, conf) {
 
     // Then remove DNS records for the domains that we are no longer responsible for.
     await Promise.all(Object.values(prevMods).map(function ({mod, domainList}) {
-      // If the module was disabled before there should be any records that we need to clean up
-      if (mod.disabled) { return; }
-
       var oldDomains;
-      if (!curMods[mod.id] || curMods[mod.id].disabled || mod.tokenId !== curMods[mod.id].mod.tokenId) {
+      if (!curMods[mod.id] || mod.tokenId !== curMods[mod.id].mod.tokenId) {
         oldDomains = domainList.slice();
       } else {
         oldDomains = domainList.filter(function (domain) {
@@ -227,7 +241,7 @@ module.exports.create = function (deps, conf) {
         return;
       }
 
-      return utils.getSession(mod.tokenId).then(function (session) {
+      return getSession(mod.tokenId).then(function (session) {
         return dnsCtrl.removeDomains(session, oldDomains);
       });
     }).filter(Boolean));
@@ -237,9 +251,6 @@ module.exports.create = function (deps, conf) {
 
     // And add DNS records for any newly added domains.
     await Promise.all(Object.values(curMods).map(function ({mod, domainList}) {
-      // Don't set any new records if the module has been disabled.
-      if (mod.disabled) { return; }
-
       var newDomains;
       if (!prevMods[mod.id] || mod.tokenId !== prevMods[mod.id].mod.tokenId) {
         newDomains = domainList.slice();
@@ -255,7 +266,7 @@ module.exports.create = function (deps, conf) {
         return;
       }
 
-      return utils.getSession(mod.tokenId).then(function (session) {
+      return getSession(mod.tokenId).then(function (session) {
         return dnsCtrl.setDeviceAddress(session, publicAddress, newDomains);
       });
     }).filter(Boolean));
@@ -321,6 +332,5 @@ module.exports.create = function (deps, conf) {
   , getDeviceAddresses: dnsCtrl.getDeviceAddresses
   , recheckPubAddr:     recheckPubAddr
   , updateConf:         updateConf
-  , challenge
   };
 };

lib/ddns/utils.js

@@ -1,102 +0,0 @@
-'use strict';
-
-module.exports.create = function (deps, conf) {
-
-  async function getSession(id) {
-    var session = await deps.storage.tokens.get(id);
-    if (!session) {
-      throw new Error('no user token with ID "' + id + '"');
-    }
-    return session;
-  }
-
-  function iterateAllModules(action, curConf) {
-    curConf = curConf || conf;
-    var promises = [];
-
-    curConf.domains.forEach(function (dom) {
-      if (!dom.modules || !Array.isArray(dom.modules.ddns) || !dom.modules.ddns.length) {
-        return null;
-      }
-
-      // For the time being all of our things should only be tried once (regardless if it succeeded)
-      // TODO: revisit this behavior when we support multiple ways of setting records, and/or
-      // if we want to allow later modules to run if early modules fail.
-      promises.push(dom.modules.ddns.reduce(function (prom, mod) {
-        if (prom) { return prom; }
-        return action(mod, dom.names);
-      }, null));
-    });
-
-    curConf.ddns.modules.forEach(function (mod) {
-      promises.push(action(mod, mod.domains));
-    });
-
-    return Promise.all(promises.filter(Boolean));
-  }
-
-  var tldCache = {};
-  async function updateTldCache(provider) {
-    var reqObj = {
-      url: deps.OAUTH3.url.normalize(provider) + '/api/com.daplie.domains/prices'
-    , method: 'GET'
-    , json: true
-    };
-
-    var resp = await deps.OAUTH3.request(reqObj);
-    var tldObj = {};
-    resp.data.forEach(function (tldInfo) {
-      if (tldInfo.enabled) {
-        tldObj[tldInfo.tld] = true;
-      }
-    });
-
-    tldCache[provider] = {
-      time: Date.now()
-    , tlds: tldObj
-    };
-    return tldObj;
-  }
-  async function getTlds(provider) {
-    // If we've never cached the results we need to return the promise that will fetch the result,
-    // otherwise we can return the cached value. If the cached value has "expired", we can still
-    // return the cached value we just want to update the cache in parellel (making sure we only
-    // update once).
-    if (!tldCache[provider]) {
-      tldCache[provider] = {
-        updating: true
-      , tlds: updateTldCache(provider)
-      };
-    }
-    if (!tldCache[provider].updating && Date.now() - tldCache[provider].time > 24 * 60 * 60 * 1000) {
-      tldCache[provider].updating = true;
-      updateTldCache(provider);
-    }
-
-    return tldCache[provider].tlds;
-  }
-
-  async function splitDomains(provider, domains) {
-    var tlds = await getTlds(provider);
-    return domains.map(function (domain) {
-      var split = domain.split('.');
-      var tldSegCnt = tlds[split.slice(-2).join('.')] ? 2 : 1;
-
-      // Currently assuming that the sld can't contain dots, and that the tld can have at
-      // most one dot. Not 100% sure this is a valid assumption, but exceptions should be
-      // rare even if the assumption isn't valid.
-      return {
-        tld: split.slice(-tldSegCnt).join('.')
-      , sld: split.slice(-tldSegCnt - 1, -tldSegCnt).join('.')
-      , sub: split.slice(0, -tldSegCnt - 1).join('.')
-      };
-    });
-  }
-
-  return {
-    getSession
-  , iterateAllModules
-  , getTlds
-  , splitDomains
-  };
-};

lib/socks5-server.js

@@ -80,7 +80,6 @@ module.exports.create = function (deps, config) {
       configEnabled = true;
     }
   }
-  process.nextTick(updateConf);
 
   return {
     curState

lib/tcp/index.js

@@ -159,13 +159,11 @@ module.exports.create = function (deps, config) {
     });
   }
 
-  process.nextTick(function () {
+  modules = {};
-    modules = {};
+  modules.tcpHandler = tcpHandler;
-    modules.tcpHandler = tcpHandler;
+  modules.proxy = require('./proxy-conn').create(deps, config);
-    modules.proxy = require('./proxy-conn').create(deps, config);
+  modules.tls   = require('./tls').create(deps, config, modules);
-    modules.tls   = require('./tls').create(deps, config, modules);
+  modules.http  = require('./http').create(deps, config, modules);
-    modules.http  = require('./http').create(deps, config, modules);
-  });
 
   function updateListeners() {
     var current = listeners.list();

lib/tcp/tls.js

@@ -86,7 +86,8 @@ module.exports.create = function (deps, config, tcpMods) {
   , challenges: {
       'http-01': require('le-challenge-fs').create({ debug: config.debug })
     , 'tls-sni-01': require('le-challenge-sni').create({ debug: config.debug })
-    , 'dns-01': deps.ddns.challenge
+      // TODO dns-01
+      //, 'dns-01': require('le-challenge-ddns').create({ debug: config.debug })
     }
   , challengeType: 'http-01'
 

package.json

@@ -1,14 +1,14 @@
 {
   "name": "goldilocks",
-  "version": "1.1.6",
+  "version": "1.0.0-placeholder",
   "description": "The node.js webserver that's just right, Greenlock (HTTPS/TLS/SSL via ACME/Let's Encrypt) and tunneling (RVPN) included.",
   "main": "bin/goldilocks.js",
   "repository": {
     "type": "git",
-    "url": "git.coolaj86.com:coolaj86/goldilocks.js.git"
+    "url": "git@git.daplie.com:Daplie/goldilocks.js.git"
   },
-  "author": "AJ ONeal <coolaj86@gmail.com> (https://coolaj86.com/)",
+  "author": "AJ ONeal <aj@daplie.com> (https://daplie.com/)",
-  "license": "(MIT OR Apache-2.0)",
+  "license": "SEE LICENSE IN LICENSE.txt",
   "scripts": {
     "test": "node bin/goldilocks.js -p 8443 -d /tmp/"
   },
@@ -34,41 +34,42 @@
     "server"
   ],
   "bugs": {
-    "url": "https://git.coolaj86.com/coolaj86/goldilocks.js/issues"
+    "url": "https://git.daplie.com/Daplie/server-https/issues"
   },
-  "homepage": "https://git.coolaj86.com/coolaj86/goldilocks.js",
+  "homepage": "https://git.daplie.com/Daplie/goldilocks.js#readme",
   "dependencies": {
     "bluebird": "^3.4.6",
-    "body-parser": "1",
+    "body-parser": "git+https://github.com/expressjs/body-parser.git#1.16.1",
     "commander": "^2.9.0",
     "deep-equal": "^1.0.1",
-    "dns-suite": "1",
+    "dns-suite": "git+https://git@git.daplie.com/Daplie/dns-suite#v1",
-    "express": "4",
+    "express": "git+https://github.com/expressjs/express.git#4.x",
     "finalhandler": "^0.4.0",
-    "greenlock": "2.1",
+    "greenlock": "git+https://git.daplie.com/Daplie/node-greenlock.git#master",
     "http-proxy": "^1.16.2",
-    "human-readable-ids": "1",
+    "human-readable-ids": "git+https://git.daplie.com/Daplie/human-readable-ids-js#master",
-    "ipaddr.js": "v1.3",
+    "ipaddr.js": "git+https://github.com/whitequark/ipaddr.js.git#v1.3.0",
     "js-yaml": "^3.8.3",
     "jsonschema": "^1.2.0",
     "jsonwebtoken": "^7.4.0",
-    "le-challenge-fs": "2",
+    "le-challenge-ddns": "git+https://git.daplie.com/Daplie/le-challenge-ddns.git#master",
+    "le-challenge-fs": "git+https://git.daplie.com/Daplie/le-challenge-webroot.git#master",
     "le-challenge-sni": "^2.0.1",
-    "le-store-certbot": "2",
+    "le-store-certbot": "git+https://git.daplie.com/Daplie/le-store-certbot.git#master",
     "localhost.daplie.me-certificates": "^1.3.5",
     "network": "^0.4.0",
-    "recase": "v1.0.4",
+    "recase": "git+https://git.daplie.com/coolaj86/recase-js.git#v1.0.4",
     "redirect-https": "^1.1.0",
     "request": "^2.81.0",
-    "scmp": "1",
+    "scmp": "git+https://github.com/freewil/scmp.git#1.x",
     "serve-index": "^1.7.0",
     "serve-static": "^1.10.0",
     "server-destroy": "^1.0.1",
     "sni": "^1.0.0",
     "socket-pair": "^1.0.3",
     "socksv5": "0.0.6",
-    "stunnel": "1.0",
+    "stunnel": "git+https://git.daplie.com/Daplie/node-tunnel-client.git#v1",
-    "stunneld": "0.9",
+    "stunneld": "git+https://git.daplie.com/Daplie/node-tunnel-server.git#v1",
     "tunnel-packer": "^1.3.0",
     "ws": "^2.3.1"
   }

terms.sh

@@ -0,0 +1,3 @@
+# adding TOS to TXT DNS Record
+daplie dns:set -n _terms._cloud.localhost.foo.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600
+daplie dns:set -n _terms._cloud.localhost.alpha.daplie.me -t TXT -a '{"url":"oauth3.org/tos/draft","explicit":true}' --ttl 3600

test-chain.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+node serve.js \
+  --port 8443 \
+  --key node_modules/localhost.daplie.me-certificates/privkey.pem \
+  --cert node_modules/localhost.daplie.me-certificates/fullchain.pem \
+  --root node_modules/localhost.daplie.me-certificates/root.pem \
+  -c "$(cat node_modules/localhost.daplie.me-certificates/root.pem)" &
+
+PID=$!
+
+sleep 1
+curl -s --insecure http://localhost.daplie.me:8443 > ./root.pem
+curl -s https://localhost.daplie.me:8443 --cacert ./root.pem
+
+rm ./root.pem
+kill $PID 2>/dev/null

uninstall.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+
+# something or other about android and tmux using PREFIX
+#: "${PREFIX:=''}"
+MY_ROOT=""
+if [ -z "${PREFIX-}" ]; then
+  MY_ROOT=""
+else
+  MY_ROOT="$PREFIX"
+fi
+# Not every platform has or needs sudo, gotta save them O(1)s...
+sudo_cmd=""
+((EUID)) && [[ -z "$ANDROID_ROOT" ]] && sudo_cmd="sudo"
+
+# you don't want any oopsies when an rm -rf is involved...
+set -e
+set -u
+
+my_app_name=goldilocks
+my_app_pkg_name=com.daplie.goldilocks.web
+
+my_app_etc_config="etc/${my_app_name}/${my_app_name}.yml"
+my_app_systemd_service="etc/systemd/system/${my_app_name}.service"
+my_app_systemd_tmpfiles="etc/tmpfiles.d/${my_app_name}.conf"
+my_app_launchd_service="Library/LaunchDaemons/${my_app_pkg_name}.plist"
+my_app_upstart_service="etc/init.d/${my_app_name}.conf"
+
+
+$sudo_cmd rm -f /usr/local/bin/$my_app_name
+$sudo_cmd rm -f /usr/local/bin/uninstall-$my_app_name
+$sudo_cmd rm -rf /usr/local/lib/node_modules/$my_app_name
+
+$sudo_cmd rm -f "$MY_ROOT/$my_app_etc_config"
+$sudo_cmd rmdir -p $(dirname "$MY_ROOT/$my_app_etc_config") 2>/dev/null || true
+$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_service"
+$sudo_cmd rm -f "$MY_ROOT/$my_app_systemd_tmpfiles"
+$sudo_cmd rm -f "$MY_ROOT/$my_app_launchd_service"
+$sudo_cmd rm -f "$MY_ROOT/$my_app_upstart_service"
+
+$sudo_cmd rm -rf /opt/$my_app_name
+$sudo_cmd rm -rf /var/log/$my_app_name
+
+# TODO flag for --purge
+#rm -rf /etc/goldilocks
+
+# TODO trap uninstall function
+
+echo "uninstall complete: $my_app_name"

update-packages.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+set -e
+set -u
+
+pushd $(dirname ${0})/packages/assets
+
+OAUTH3_GIT_URL="https://git.daplie.com/Oauth3/oauth3.js.git"
+git clone ${OAUTH3_GIT_URL} org.oauth3 || true
+pushd org.oauth3
+git remote set-url origin ${OAUTH3_GIT_URL}
+git checkout master
+git pull
+popd
+
+mkdir -p com.jquery
+pushd com.jquery
+curl -o jquery-3.1.1.js 'https://code.jquery.com/jquery-3.1.1.js'
+popd
+
+mkdir -p com.google
+pushd com.google
+curl -o  angular.1.6.2.min.js 'https://ajax.googleapis.com/ajax/libs/angularjs/1.6.2/angular.min.js'
+popd
+
+mkdir -p well-known
+pushd well-known
+ln -snf ../org.oauth3/well-known/oauth3 ./oauth3
+popd
+
+popd