walnut.js/README.md

274 lines
7.4 KiB
Markdown
Raw Normal View History

2015-11-28 07:40:33 +00:00
walnut
======
2017-07-28 23:38:23 +00:00
An opinionated, constrained, secure application framework with a hard shell - kinda like iOS, but for a server.
Applications are written in express, but instead of using `require` for generic packages,
they use `req.getSiteCapability(pkg)` and are restricted to packages that have been
allowed by app, device, site, or user permission. Any configuration for the capability
(external passwords, api keys, etc) will be set up beforehand so that they are not exposed
to the application.
Security Features
2017-07-28 23:16:56 +00:00
-----------------
* JSON-only APIs
* JWT (not cookie*) authentication
* no server-rendered html
* disallows urlencoded forms, except for secured webhooks
* disallows cookies, except for protected static assets
* api.* subdomain for apis
* assets.* subdomain for protected assets
* *must* sit behind a trusted https proxy (such as [Goldilocks](https://git.daplie.com/Daplie/goldilocks.js)
* HTTPS-only (checks for X-Forwarded-For)
* AES, RSA, and ECDSA encryption and signing
* Safe against CSRF, XSS, and SQL injection
* Safe against Compression attacks
\*Cookies are used only for GETs and only where using a token would be less secure -
such as images which would otherwise require the token to be passed into the img src.
They are also scoped such that CSRF attacks are not possible.
Application Features
2017-07-28 23:16:56 +00:00
--------------------
* JSON-only expressjs APIs
* Capability-based permissions system for (oauth3-discoverable) packages such as
* large file access (files@daplie.com)
* database access (data@daplie.com)
* scheduling (for background tasks, alerts, alarms, calendars, reminders, etc) (events@daplie.com)
* payments (credit card) (payments@daplie.com)
* email (email@daplie.com)
* SMS (texting) (tel@daplie.com)
* voice (calls and answering machine) (tel@daplie.com)
* lamba-style functions (functions@daplie.com)
* Per-app, per-site, and per-user configurations
* Multi-Tentated Application Management
* Built-in OAuth2 & OAuth3 support
2015-11-28 07:40:33 +00:00
2017-08-02 21:45:21 +00:00
Currently being tested with Ubuntu, Raspbian, and Debian on Digital Ocean, Raspberry Pi, and Heroku.
2017-08-02 21:36:28 +00:00
Installation
------------
2017-07-28 23:16:56 +00:00
2017-08-02 22:07:10 +00:00
We're still in a stage where the installation generally requires many manual steps.
2017-08-02 21:36:28 +00:00
See [INSTALL.md](https://git.daplie.com/Daplie/walnut.js/blob/master/INSTALL.md)
2015-11-28 07:40:33 +00:00
2017-06-13 16:56:44 +00:00
API
---
2017-08-02 22:07:10 +00:00
The API is still in flux, but you can take a peek anyway.
2017-08-02 21:45:21 +00:00
See [API.md](https://git.daplie.com/Daplie/walnut.js/blob/master/API.md)
2017-06-13 16:56:44 +00:00
2017-08-02 21:45:21 +00:00
Understanding Walnut
====================
2015-11-28 07:40:33 +00:00
```
/srv/walnut/
├── setup.sh (in-progress)
├── core
2017-05-19 05:20:09 +00:00
│ ├── bin
│ ├── boot
│ └── lib
2017-06-13 16:52:26 +00:00
├── etc
│ └── client-api-grants
2015-11-28 07:40:33 +00:00
├── node_modules
├── packages
2017-05-19 05:20:09 +00:00
│ ├── apis
2017-08-01 15:50:27 +00:00
| ├── pages
2017-08-03 17:31:08 +00:00
| ├── rest
2017-05-19 05:20:09 +00:00
│ └── services
2015-11-28 07:40:33 +00:00
└── var
2017-06-13 16:52:26 +00:00
└── sites
2015-11-28 07:40:33 +00:00
```
* `core` contains all walnut code
* `node_modules` is a flat installation of all dependencies
* `certs` is a directory for Let's Encrypt (or custom) certificates
* `var` is a directory for database files and such
* `packages` contains 3 types of packages
2017-05-19 05:20:09 +00:00
Will install to
---------------
```
/srv/walnut/core/
/etc/walnut
/opt/walnut
/var/log/walnut
/etc/systemd/system/walnut.service
/etc/tmpfiles.d/walnut.conf
```
Initialization
--------------
2015-11-28 07:40:33 +00:00
2017-05-19 05:20:09 +00:00
needs to know its primary domain
```
2017-07-28 23:26:02 +00:00
POST https://api.<domain.tld>/api/walnut@daplie.com/init
2017-05-19 05:20:09 +00:00
{ "domain": "<domain.tld>" }
```
2017-05-22 17:17:55 +00:00
The following domains are required to point to WALNUT server
2017-07-31 22:21:33 +00:00
```
cloud.<domain.tld>
api.cloud.<domain.tld>
```
and
2017-05-22 17:17:55 +00:00
```
<domain.tld>
www.<domain.tld>
api.<domain.tld>
assets.<domain.tld>
```
The domains can be setup through the Daplie Desktop App or with `daplie-tools`
```bash
# set device address and attach primary domain
daplie devices:attach -d foodevice -n example.com -a 127.0.0.1
# attach all other domains with same device/address
daplie devices:attach -d foodevice -n www.example.com
daplie devices:attach -d foodevice -n api.example.com
daplie devices:attach -d foodevice -n assets.example.com
daplie devices:attach -d foodevice -n cloud.example.com
daplie devices:attach -d foodevice -n api.cloud.example.com
```
2017-05-22 17:17:55 +00:00
Example `/etc/goldilocks/goldilocks.yml`:
```yml
tls:
email: domains@example.com
servernames:
- example.com
- www.example.com
- api.example.com
- assets.example.com
- cloud.example.com
- api.cloud.example.com
http:
trust_proxy: true
modules:
- name: proxy
domains:
- '*'
address: '127.0.0.1:3000'
```
2017-05-19 05:20:09 +00:00
Resetting the Initialization
----------------------------
Once you run the app the initialization files will appear in these locations
```
/srv/walnut/var/com.daplie.walnut.config.sqlite3
2017-05-26 20:23:17 +00:00
/srv/walnut/config/<domain.tld>/config.json
2017-05-19 05:20:09 +00:00
```
2015-11-28 07:40:33 +00:00
2017-08-02 21:45:21 +00:00
Deleting those files and restarting walnut will reset it to its bootstrap state.
2017-05-19 07:45:41 +00:00
Accessing static apps
---------------------
2017-08-01 15:49:25 +00:00
Static apps are stored in `packages/pages`
2017-05-19 07:45:41 +00:00
```
# App ID as files with a list of packages they should load
2017-05-22 17:17:55 +00:00
# note that '#' is used in place of '/' because files and folders may not contain '/' in their names
2017-08-01 15:49:25 +00:00
/srv/walnut/packages/pages/<domain.tld#path> # https://domain.tld/path
/srv/walnut/packages/pages/<domain.tld> # https://domain.tld and https://domain.tld/foo match
2017-05-22 17:17:55 +00:00
2017-08-01 15:49:25 +00:00
# packages are directories with email-style name # For the sake of debugging these packages can be accessed directly, without a site by
/srv/walnut/packages/pages/<package@domain.tld> # matches apps.<domain.tld>/<package-name> and <domain.tld>/apps/<package-name>
2017-05-22 17:17:55 +00:00
```
Accessing REST APIs
-------------------
2017-05-19 07:45:41 +00:00
2017-05-22 17:17:55 +00:00
```
# Apps are granted access to use a package by listing it in the grants file by the name of the app url (domain.tld)
/srv/walnut/packages/client-api-grants/<domain.tld> # matches api.<domain.tld>/api/ and contains a list of allowed REST APIs
# the REST apis themselves are submatched as api.<domain.tld>/api/<tld.domain.package>
# packages are directories with reverse dns name, a package.json, and an index.js
/srv/walnut/packages/rest/<tld.domain.package>
```
Example tree with contents:
Here `com.example.hello` is a package with a REST API and a static page
and `foobar.me` is a WALNUT-configured domain (smithfam.net, etc).
2017-05-22 17:22:32 +00:00
The packages:
2017-05-22 17:17:55 +00:00
```
/srv/walnut/packages/
├── api
├── rest
│ └── com.example.hello
│ ├── package.json
│ └── index.js
│ '''
│ 'use strict';
│ module.exports.create = function (conf, deps, app) {
│ app.use('/', function (req, res) {
│ console.log('[com.example.hello] req.url', req.url);
│ res.send({ message: 'hello' });
│ });
│ return deps.Promise.resolve();
│ };
│ '''
2017-05-22 17:22:32 +00:00
└── services
2017-08-01 15:40:53 +00:00
```
```
2017-08-01 15:50:27 +00:00
/srv/walnut/packages/
└── pages
2017-08-01 15:40:01 +00:00
└── demo@example.com
└── index.html
'''
<html>
<head><title>demo@example.com</title></head>
<body>
<h1>demo@example.com</h1>
</body>
</html>
'''
2017-05-22 17:22:32 +00:00
```
The permissions:
```
/srv/walnut/packages/
2017-08-01 15:49:25 +00:00
└── client-api-grants
└── cloud.foobar.me
'''
hello@example.com # refers to /srv/walnut/packages/rest/hello@example.com
'''
```
```
/srv/walnut/var/
2017-05-22 17:17:55 +00:00
└── sites
└── daplie.me
'''
2017-08-01 15:49:25 +00:00
seed@example.com # refers to /srv/walnut/packages/pages/seed@example.com
2017-05-22 17:17:55 +00:00
'''
2017-08-02 21:36:28 +00:00
```