walnut.js/API.md

• Bootstrap Initialization
• Package Discovery
• Package Layout
• Package APIs
• RESTful API constraints

Bootstrap Initialization

Before walnut is configured it starts up in a bootstrap mode with a single API exposed to set its primary domain.

# Set up with example.com as the primary domain
curl -X POST http://api.localhost.daplie.me:3000/api/walnut@daplie.com/init \
  -H 'X-Forwarded-Proto: https' \
  -H 'Content-Type: application/json' \
  -d '{ "domain": "example.com" }'

From this point forward you can now interact with Walnut at that domain.

OAuth3 Package Discovery

Unlike most package systems such as npm (node.js), gem (ruby), pip (python), etc, which rely on a single, centralized closed-source repository, walnut packages use the OAuth3 Package Specification which allows for open and closed, public and private, free and paid packages, according to the desire of the publisher.

In this model the name of a package is all that is necessary to install it from its publisher.

Let's hello@example.com as an example:

hello@example.com specifies that hello is a submodule of the example.com package. As you might guess, the publisher example.com is responsible for this package.

https://example.com/.well-known/packages@oauth3.org/ is the known location where package types can be discovered.

Since we're using walnut.js which is published by daplie.com, we can find walnut packages at

https://example.com/.well-known/packages@oauth3.org/walnut.js@daplie.com.json

This file tells us where example.com publishes packages that adhere to the walnut.js@daplie.com package spec. (you can imagine that if walnut were to be implemented in ruby the ruby packages could be found at walnut.rb@daplie.com or if walnut were not protected by trademark and another company were to create a similar, but incompatible package system for it, it would be walnut.go@acme.co or some such)

For publishers with a long list of packages you might find a URL to describe where more information about a package can be found.

Template variables

:package_name
:package_version

{ "package_url": "https://packages.example.com/indexes/:package_name.json"
, "package_index": "https://packages.example.com/index.json"
, "pingback_url": "https://api.example.com/api/pingback@oauth3.org/:package_name?version=:package_version"
}

For publishers with a short list of packages you might find that all of the packages are listed directly.

Template variables

:package_name
:package_version
:payment_token

{ "package_url": null
, "package_index": null
, "pingback_url": "https://api.example.com/api/pingback@oauth3.org/:package_name?version=:package_version"
, "packages": [
  { "name": "hello@example.com"
  , "license": "Physical-Source-v2@licenses.org"
  , "requires_payment": true
  , "payment_url": "https://author.tld/api/payments@oauth3.org/schemas/packages/walnut.js@daplie.com/:package_name"
  , "zip_url": "https://cdn.tld/api/orders@cdn.tld/:package_name-:package_version.zip?authorization=:payment_token"
  , "git_https_url":"https://git.cdn.tld/author.tld/:package_name.git#:package_version?authorization=:payment_token"
  , "git_ssh_url":":payment_token@git.cdn.tld:author.tld/:package_name.git#:package_version"
  }
, { "name": "gizmo@example.com"
  , "license": "MIT@licenses.org"
  , "requires_payment": false
  , "zip_url": "https://example.com/packages/:package_name-:package_version.zip"
  , "git_https_url":"https://git.cdn.tld/author.tld/:package_name.git#:package_version"
  , "git_ssh_url":"git@git.cdn.tld:author.tld/:package_name.git#:package_version"
  }
] }

Note: It is not expected that the package manage will directly query the publisher - a centralized caching service may be used. However, it is intended that a package manager could query the publisher, even if the publisher points back to a centralized cdn.

Package Layout

Packages have data model, api, and RESTful components.

/srv/walnut/packages/rest/hello@example.com/
  package.json
  api.js
  models.js
  rest.js

Each package must be enabled on a per-domain basis.

/srv/walnut/packages/client-api-grants/provider.example.com
  '''
  hello@example.com
  '''

When a package is enabled for example.com it becomes immediately available via https as https://api.example.com/api/package@publisher.tld/.

Note: although hot-loading of packages is supported, reloading still requires restarting the walnut server - for now at least

Package APIs

Packages are intended to be functional, however, they allow for instantiation as a matter of not putting ourselves in a box and finding out later that it's very, very, very hard to open the box back up.

rest.js:

module.exports.create = function (conf, deps, app) {
  var API = require('./api.js');
  var REST = {
    hello: function (req, res/*, next*/) {
      var promise = API.hello(deps, req.Models, req.oauth3/*, opts*/);

      app.handlePromise(req, res, promise, "[hello@example.com]");
    }
  }
};

Special methods for app:

app.handlePromise(request, response, promise, message);

handlePromise will respond to the request with the result of promise as JSON. If there is an error, it will include message in order to help you debug.

Special properties of request:

req.getSiteCapability(pkg)          // Promises a capability on behalf of the current site (req.experienceId) without exposing secrets

req.webhookParser(pkg, req, opts)   // Allows the use of potentially dangerous parsers (i.e. urlencoded) for the sake of webhooks

req.apiUrlPrefix                    // This represents the full package path without any package specific endpoints
                                    // This is particularly useful when constructing webhook URLs
                                    // i.e. https://api.example.com/api/pkg@domain.tld
                                    //      (of https://api.example.com/api/pkg@domain.tld/public/foo)

req.experienceId                    // The instance name of an app as a whole, where an app is mounted
                                    // i.e. the 'example.com' part of https://example.com/foo
                                    //      OR 'example.com#foo' if '/foo' is part of the app's mount point

req.clientApiUri                    // The api URL for the instance of an app
                                    // i.e. the 'api.example.com' part of https://api.example.com/api/hello@example.com/kv/foo

req.pkgId                           // The name of the package being accessed
                                    // i.e. the 'hello@example.com' part of https://api.example.com/api/hello@example.com/kv/foo

req.oauth3.accountIdx               // The system id of the account represented by the token
                                    // i.e. this is the user

Internal (and/or deprecated) APIs that you will very likely encounter

req.getSiteStore().then(function (models) {
  req.Models = models;
});

//
// Consider Models for a package 'hello@example.com', the would be named like so
//
req.Models.HelloExampleComData.create(obj)
req.Models.ComExampleHelloData.save(obj)
req.Models.ComExampleHelloData.find(params)
req.Models.ComExampleHelloData.destroy(objOrId)

//
// These should be scoped in such a way that the only hand back data specific
// to the experience and not expose secrets
//

req.getSiteConfig('com.example.hello').then(function (config) {
    // the com.example.hello section of /srv/walnut/etc/:domain/config.json
});
req.getSitePackageConfig

//
// Deprecated
//
// These helper methods should be moved to a capability

req.Stripe
req.Mandrill
req.Mailchimp

req.getSiteMailer().then(function (mailer) {});

RESTful API Contstraints

Walnut will reject requests to all domains and subdomains except those that begin with the subdomain api, assets, and webhooks.

• api is for JSON APIs and must use JWT in HTTP Authorization headers for authentication
  • secured by disallowing cookies
  • secured by disallowing non-JSON form types
  • secured by requiring authentication in header
• assets is for protected access to large files and other blobs and must use JWT in Cookies for authentication
  • warning: allows implicit authorization via cookies for hotlinking and the like
  • secured by not exposing tokens when users copy-paste
• webhooks is for 3rd-party API hooks and APIs with special requirements outside of the normal security model
  • warning: these are insecure and should be used with caution, prudence, and wisdom
  • JWT via query parameter
  • urlencoded forms
  • XML forms

Bare and www domains are DISALLOWED from being served by Walnut.

This enables scalability of static sites as the static assets are never on the same domain as generic APIs or authenticated assets. It also enforces security by disallowing 1990s web vulnerabilities by default.