298 lines
7.5 KiB
Markdown
298 lines
7.5 KiB
Markdown
walnut
|
|
======
|
|
|
|
An opinionated, constrained, secure application framework with a hard shell - kinda like iOS, but for a server.
|
|
|
|
Applications are written in express, but instead of using `require` for generic packages,
|
|
they use `req.getSiteCapability(pkg)` and are restricted to packages that have been
|
|
allowed by app, device, site, or user permission. Any configuration for the capability
|
|
(external passwords, api keys, etc) will be set up beforehand so that they are not exposed
|
|
to the application.
|
|
|
|
Security Features
|
|
-----------------
|
|
|
|
* JSON-only APIs
|
|
* JWT (not cookie*) authentication
|
|
* no server-rendered html
|
|
* disallows urlencoded forms, except for secured webhooks
|
|
* disallows cookies, except for protected static assets
|
|
* api.* subdomain for apis
|
|
* assets.* subdomain for protected assets
|
|
* *must* sit behind a trusted https proxy (such as [Goldilocks](https://git.daplie.com/Daplie/goldilocks.js)
|
|
* HTTPS-only (checks for X-Forwarded-For)
|
|
* AES, RSA, and ECDSA encryption and signing
|
|
* Safe against CSRF, XSS, and SQL injection
|
|
* Safe against Compression attacks
|
|
|
|
\*Cookies are used only for GETs and only where using a token would be less secure -
|
|
such as images which would otherwise require the token to be passed into the img src.
|
|
They are also scoped such that CSRF attacks are not possible.
|
|
|
|
Application Features
|
|
--------------------
|
|
|
|
* JSON-only expressjs APIs
|
|
* Capability-based permissions system for (oauth3-discoverable) packages such as
|
|
* large file access (files@daplie.com)
|
|
* database access (data@daplie.com)
|
|
* scheduling (for background tasks, alerts, alarms, calendars, reminders, etc) (events@daplie.com)
|
|
* payments (credit card) (payments@daplie.com)
|
|
* email (email@daplie.com)
|
|
* SMS (texting) (tel@daplie.com)
|
|
* voice (calls and answering machine) (tel@daplie.com)
|
|
* lamba-style functions (functions@daplie.com)
|
|
* Per-app, per-site, and per-user configurations
|
|
* Multi-Tentated Application Management
|
|
* Built-in OAuth2 & OAuth3 support
|
|
|
|
Install
|
|
-------
|
|
|
|
```bash
|
|
curl https://daplie.me/install-scripts | bash
|
|
|
|
daplie-install-walnut
|
|
```
|
|
|
|
You could also, of course, try installing from the repository directly
|
|
(especially if you have goldilocks or some similar already installed)
|
|
|
|
```bash
|
|
mkdir -p /srv/walnut/
|
|
git clone git@git.daplie.com:Daplie/walnut.js.git /srv/walnut/core
|
|
pushd /srv/walnut/core
|
|
git checkout v1
|
|
popd
|
|
bash /srv/walnut/core/install-helper.sh
|
|
```
|
|
|
|
Initial Configuration
|
|
-------------
|
|
|
|
Once installed and started you can visit <https://localhost.daplie.me:3000>.
|
|
|
|
```bash
|
|
curl -X POST http://localhost.daplie.me:3000 \
|
|
-H 'X-Forwarded-Proto: https' \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{
|
|
domain: 'example.com'
|
|
}'
|
|
```
|
|
|
|
API
|
|
---
|
|
|
|
API docs are here https://git.daplie.com/Daplie/com.example.hello
|
|
|
|
Structure
|
|
=====
|
|
|
|
Currently being tested with Ubuntu, Raspbian, and Debian on Digital Ocean, Raspberry Pi, and Heroku.
|
|
|
|
```
|
|
/srv/walnut/
|
|
├── setup.sh (in-progress)
|
|
├── core
|
|
│ ├── bin
|
|
│ ├── boot
|
|
│ ├── holepunch
|
|
│ └── lib
|
|
├── etc
|
|
│ └── client-api-grants
|
|
├── node_modules
|
|
├── packages
|
|
│ ├── apis
|
|
│ ├── pages
|
|
│ └── services
|
|
└── var
|
|
└── sites
|
|
```
|
|
|
|
* `core` contains all walnut code
|
|
* `node_modules` is a flat installation of all dependencies
|
|
* `certs` is a directory for Let's Encrypt (or custom) certificates
|
|
* `var` is a directory for database files and such
|
|
* `packages` contains 3 types of packages
|
|
|
|
Will install to
|
|
---------------
|
|
|
|
```
|
|
/srv/walnut/core/
|
|
/etc/walnut
|
|
/opt/walnut
|
|
/var/log/walnut
|
|
/etc/systemd/system/walnut.service
|
|
/etc/tmpfiles.d/walnut.conf
|
|
```
|
|
|
|
Implementation details
|
|
----------------
|
|
|
|
Initialization
|
|
--------------
|
|
|
|
needs to know its primary domain
|
|
|
|
```
|
|
POST https://api.<domain.tld>/api/walnut@daplie.com/init
|
|
|
|
{ "domain": "<domain.tld>" }
|
|
```
|
|
|
|
The following domains are required to point to WALNUT server
|
|
|
|
```
|
|
cloud.<domain.tld>
|
|
api.cloud.<domain.tld>
|
|
```
|
|
|
|
and
|
|
|
|
```
|
|
<domain.tld>
|
|
www.<domain.tld>
|
|
|
|
api.<domain.tld>
|
|
assets.<domain.tld>
|
|
```
|
|
|
|
Example `/etc/goldilocks/goldilocks.yml`:
|
|
```yml
|
|
tls:
|
|
email: domains@example.com
|
|
servernames:
|
|
- example.com
|
|
- www.example.com
|
|
- api.example.com
|
|
- assets.example.com
|
|
- cloud.example.com
|
|
- api.cloud.example.com
|
|
|
|
http:
|
|
trust_proxy: true
|
|
modules:
|
|
- name: proxy
|
|
domains:
|
|
- '*'
|
|
address: '127.0.0.1:3000'
|
|
```
|
|
|
|
Resetting the Initialization
|
|
----------------------------
|
|
|
|
Once you run the app the initialization files will appear in these locations
|
|
|
|
```
|
|
/srv/walnut/var/com.daplie.walnut.config.sqlite3
|
|
/srv/walnut/config/<domain.tld>/config.json
|
|
```
|
|
|
|
Deleting those files will rese
|
|
|
|
Accessing static apps
|
|
---------------------
|
|
|
|
Static apps are stored in `packages/pages`
|
|
|
|
```
|
|
# App ID as files with a list of packages they should load
|
|
# note that '#' is used in place of '/' because files and folders may not contain '/' in their names
|
|
/srv/walnut/packages/sites/<domain.tld#path> # https://domain.tld/path
|
|
/srv/walnut/packages/sites/<domain.tld> # https://domain.tld and https://domain.tld/foo match
|
|
|
|
# packages are directories with reverse dns name # For the sake of debugging these packages can be accessed directly, without a site by
|
|
/srv/walnut/packages/pages/<tld.domain.package> # matches apps.<domain.tld>/<package-name> and <domain.tld>/apps/<package-name>
|
|
```
|
|
|
|
Accessing REST APIs
|
|
-------------------
|
|
|
|
```
|
|
# Apps are granted access to use a package by listing it in the grants file by the name of the app url (domain.tld)
|
|
/srv/walnut/packages/client-api-grants/<domain.tld> # matches api.<domain.tld>/api/ and contains a list of allowed REST APIs
|
|
# the REST apis themselves are submatched as api.<domain.tld>/api/<tld.domain.package>
|
|
|
|
# packages are directories with reverse dns name, a package.json, and an index.js
|
|
/srv/walnut/packages/rest/<tld.domain.package>
|
|
```
|
|
|
|
Example tree with contents:
|
|
|
|
Here `com.example.hello` is a package with a REST API and a static page
|
|
and `foobar.me` is a WALNUT-configured domain (smithfam.net, etc).
|
|
|
|
|
|
The packages:
|
|
|
|
```
|
|
/srv/walnut/packages/
|
|
├── api
|
|
├── pages
|
|
│ └── com.example.hello
|
|
│ └── index.html
|
|
│ '''
|
|
│ <html>
|
|
│ <head><title>com.example.hello</title></head>
|
|
│ <body>
|
|
│ <h1>com.example.hello</h1>
|
|
│ </body>
|
|
│ </html>
|
|
│ '''
|
|
│
|
|
├── rest
|
|
│ └── com.example.hello
|
|
│ ├── package.json
|
|
│ └── index.js
|
|
│ '''
|
|
│ 'use strict';
|
|
│
|
|
│ module.exports.create = function (conf, deps, app) {
|
|
│
|
|
│ app.use('/', function (req, res) {
|
|
│ console.log('[com.example.hello] req.url', req.url);
|
|
│ res.send({ message: 'hello' });
|
|
│ });
|
|
│
|
|
│ return deps.Promise.resolve();
|
|
│ };
|
|
│
|
|
│ '''
|
|
│
|
|
└── services
|
|
```
|
|
|
|
The permissions:
|
|
|
|
```
|
|
/srv/walnut/packages/
|
|
├── client-api-grants
|
|
│ └── cloud.foobar.me
|
|
│ '''
|
|
│ com.example.hello # refers to /srv/walnut/packages/rest/com.example.hello
|
|
│ '''
|
|
│
|
|
└── sites
|
|
└── daplie.me
|
|
'''
|
|
com.example.hello # refers to /srv/walnut/packages/pages/com.example.hello
|
|
'''
|
|
```
|
|
|
|
API
|
|
---
|
|
|
|
```
|
|
req.apiUrlPrefix => https://api.example.com/api/tld.domain.pkg
|
|
```
|
|
|
|
TODO
|
|
----
|
|
|
|
* HTTPS Key Pinning
|
|
* Heroku (pending completion of PostgreSQL support)
|
|
* [GunDB](https://gundb.io) Support
|
|
* OpenID support
|