A lightweight IOT application server with a hard shell written for node.js
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
AJ ONeal e2e99a5c77 merge 1 year ago
bin removed unused modules from package.json 1 year ago
boot stop double printing uncaught error stacks 1 year ago
dist fix installer once and for all? 1 year ago
etc include special and auto cert dirs 3 years ago
installer update urls 1 year ago
lib [WIP] retrieve sub from db 1 year ago
snippets add undo instructions 4 years ago
tests made leading comma style more consistent 1 year ago
.gitignore com.daplie.walnut -> walnut@daplie.com 1 year ago
.jshintrc add .jshintrc 2 years ago
API.md made some minor fixes in the markdown files 1 year ago
CHANGELOG bump ver, add standard files 1 year ago
INSTALL.md updated with repo to clone for issuer@oauth3.org in the docs 1 year ago
LICENSE bump ver, add standard files 1 year ago
README.md update urls 1 year ago
add-subtree.sh add subtree script 2 years ago
package.json update urls 1 year ago
setup-dev-deps.sh refactoring to use fs config 3 years ago
uninstall.sh update installer 2 years ago
walnut.js removed letsencrypt and other 2 years ago

README.md

walnut

An opinionated, constrained, secure application framework with a hard shell - kinda like iOS, but for a server.

Applications are written in express, but instead of using require for generic packages, they use req.getSiteCapability(pkg) and are restricted to packages that have been allowed by app, device, site, or user permission. Any configuration for the capability (external passwords, api keys, etc) will be set up beforehand so that they are not exposed to the application.

Security Features

  • JSON-only APIs
  • JWT (not cookie*) authentication
  • no server-rendered html
  • disallows urlencoded forms, except for secured webhooks
  • disallows cookies, except for protected static assets
  • api.* subdomain for apis
  • assets.* subdomain for protected assets
  • must sit behind a trusted https proxy (such as Goldilocks)
  • HTTPS-only (checks for X-Forwarded-For)
  • AES, RSA, and ECDSA encryption and signing
  • Safe against CSRF, XSS, and SQL injection
  • Safe against Compression attacks

*Cookies are used only for GETs and only where using a token would be less secure - such as images which would otherwise require the token to be passed into the img src. They are also scoped such that CSRF attacks are not possible.

Application Features

Currently being tested with Ubuntu, Raspbian, and Debian on Digital Ocean, Raspberry Pi, and Heroku.

Installation

We’re still in a stage where the installation generally requires many manual steps.

curl https://git.coolaj86.com/coolaj86/walnut.js/raw/v1.2/installer/get.sh | bash

See INSTALL.md

Uninstall

rm -rf /srv/walnut/ /var/walnut/ /etc/walnut/ /opt/walnut/ /var/log/walnut/ /etc/systemd/system/walnut.service /etc/tmpfiles.d/walnut.conf

Usage

Here’s how you run the thing, once installed:

/opt/walnut/bin/node /srv/walnut/core/bin/walnut.js

It listens on all addresses, port 3000.

TODO: Add config to restrict listening to localhost.

API

The API is still in flux, but you can take a peek anyway.

See API.md

Understanding Walnut

/srv/walnut/
├── setup.sh (in-progress)
├── core
│   ├── bin
│   ├── boot
│   └── lib
├── etc
│   └── client-api-grants
├── node_modules
├── packages
│   ├── apis
│   ├── pages
│   ├── rest
│   └── services
└── var
    └── sites
  • core contains all walnut code
  • node_modules is a flat installation of all dependencies
  • certs is a directory for Let’s Encrypt (or custom) certificates
  • var is a directory for database files and such
  • packages contains 3 types of packages

Will install to

/srv/walnut/core/
/etc/walnut
/opt/walnut
/var/log/walnut
/etc/systemd/system/walnut.service
/etc/tmpfiles.d/walnut.conf

Initialization

needs to know its primary domain

POST https://api.<domain.tld>/api/walnut@oauth3.org/init

{ "domain": "<domain.tld>" }

The following domains are required to point to WALNUT server

cloud.<domain.tld>
api.cloud.<domain.tld>

and

<domain.tld>
www.<domain.tld>

api.<domain.tld>
assets.<domain.tld>

The domains can be setup through the OAuth3 Desktop App or with oauth3-tools

# set device address and attach primary domain
oauth3 devices:attach -d foodevice -n example.com -a 127.0.0.1

# attach all other domains with same device/address
oauth3 devices:attach -d foodevice -n www.example.com
oauth3 devices:attach -d foodevice -n api.example.com
oauth3 devices:attach -d foodevice -n assets.example.com
oauth3 devices:attach -d foodevice -n cloud.example.com
oauth3 devices:attach -d foodevice -n api.cloud.example.com

Example /etc/goldilocks/goldilocks.yml:

tls:
  email: domains@example.com
  servernames:
    - example.com
    - www.example.com
    - api.example.com
    - assets.example.com
    - cloud.example.com
    - api.cloud.example.com

http:
  trust_proxy: true
  modules:
    - name: proxy
      domains:
        - '*'
      address: '127.0.0.1:3000'

Resetting the Initialization

Once you run the app the initialization files will appear in these locations

/srv/walnut/var/walnut+config@oauth3.org.sqlite3
/srv/walnut/config/<domain.tld>/config.json

Deleting those files and restarting walnut will reset it to its bootstrap state.

Accessing static apps

Static apps are stored in packages/pages

# App ID as files with a list of packages they should load
# note that '#' is used in place of '/' because files and folders may not contain '/' in their names
/srv/walnut/packages/pages/<domain.tld#path>          # https://domain.tld/path
/srv/walnut/packages/pages/<domain.tld>               # https://domain.tld and https://domain.tld/foo match

# packages are directories with email-style name      # For the sake of debugging these packages can be accessed directly, without a site by
/srv/walnut/packages/pages/<package@domain.tld>       # matches apps.<domain.tld>/<package-name> and <domain.tld>/apps/<package-name>

Accessing REST APIs

# Apps are granted access to use a package by listing it in the grants file by the name of the app url (domain.tld)
/srv/walnut/packages/client-api-grants/<domain.tld>   # matches api.<domain.tld>/api/ and contains a list of allowed REST APIs
                                                      # the REST apis themselves are submatched as api.<domain.tld>/api/<tld.domain.package>

# packages are directories with reverse dns name, a package.json, and an index.js
/srv/walnut/packages/rest/<tld.domain.package>

Example tree with contents:

Here com.example.hello is a package with a REST API and a static page and foobar.me is a WALNUT-configured domain (smithfam.net, etc).

The packages:

/srv/walnut/packages/
├── api
├── rest
│   └── com.example.hello
│      ├── package.json
│      └── index.js
│           '''
│           'use strict';
│
│           module.exports.create = function (conf, deps, app) {
│
│             app.use('/', function (req, res) {
│               console.log('[com.example.hello] req.url', req.url);
│               res.send({ message: 'hello' });
│             });
│
│             return deps.Promise.resolve();
│           };
│
│           '''
│
└── services
/srv/walnut/packages/
└── pages
    └── demo@example.com
        └── index.html
              '''
              <html>
                <head><title>demo@example.com</title></head>
                <body>
                  <h1>demo@example.com</h1>
                </body>
              </html>
              '''

The permissions:

/srv/walnut/packages/
└── client-api-grants
    └── cloud.foobar.me
          '''
          hello@example.com     # refers to /srv/walnut/packages/rest/hello@example.com
          '''
/srv/walnut/var/
└── sites
    └── example.com
          '''
          seed@example.com      # refers to /srv/walnut/packages/pages/seed@example.com
          '''