sqlite3-cluster.js/README.md

205 lines
6.0 KiB
Markdown
Raw Permalink Normal View History

2015-07-24 20:47:16 +00:00
SQLite3 Cluster
===============
2015-07-22 00:58:34 +00:00
2015-07-24 20:47:16 +00:00
Works with node cluster, or completely and unrelated node processes.
2015-07-22 00:58:34 +00:00
2015-07-24 20:47:16 +00:00
**Note**: Most people would probably prefer to just use PostgreSQL rather than
wrap SQLite as a service... but I am not most people.
2015-07-22 00:58:34 +00:00
Node.js runs on a single core, which isn't very effective.
You can run multiple Node.js instances to take advantage of multiple cores,
but if you do that, you can't use SQLite in each process.
This module will either run client-server style in environments that benefit from it
(such as the Raspberry Pi 2 with 4 cores), or in-process for environments that don't
(such as the Raspberry Pi B and B+).
2015-07-24 20:47:16 +00:00
This also works with **SQLCipher**.
2015-07-22 00:58:34 +00:00
Usage
=====
2015-07-24 20:47:16 +00:00
The default behavior is to try to connect to a master and, if that fails, to become the master.
However, if you are in fact using the `cluster` rather than spinning up random instances,
you'll probably prefer to use this pattern:
2015-07-22 00:58:34 +00:00
```js
2015-07-24 20:47:16 +00:00
var cluster = require('cluster');
var sqlite = require('sqlite3-cluster');
var numCores = require('os').cpus().length;
2015-07-22 00:58:34 +00:00
var opts = {
2015-07-24 20:47:16 +00:00
filename: '/tmp/mydb.sqlcipher'
, sock: '/tmp/mydb.sqlcipher.sock'
2015-07-22 00:58:34 +00:00
, verbose: false
2015-07-24 20:47:16 +00:00
// a good default to use for instances where you might want
// to cluster or to run standalone, but with the same API
, serve: cluster.isMaster
, connect: cluster.isWorker
, standalone: (1 === numCores) // overrides serve and connect
2015-07-24 20:48:25 +00:00
// if using SQLCipher, you can supply the key and desired bit-length and the
// appropriate PRAGMA statements will be issued before the database is returned
2015-07-24 20:47:16 +00:00
, key: '00000000000000000000000000000000'
, bits: 128
2015-07-22 00:58:34 +00:00
};
sqlite.create(opts).then(function (db) {
2015-07-24 20:47:16 +00:00
// same api as new sqlite3.Database(options.filename)
db.run("SELECT ?", ['Hello World!'], function (err) {
2015-07-24 20:47:16 +00:00
if (err) {
console.error('[ERROR]', cluster.isMaster && '0' || cluster.worker.id);
console.error(err);
return;
}
console.log('[this]', cluster.isMaster && '0' || cluster.worker.id);
console.log(this);
});
2015-07-22 00:58:34 +00:00
});
process.on('unhandledPromiseRejection', function (err) {
console.error('Unhandled Promise Rejection');
console.error(err);
console.error(err.stack);
throw err;
});
2015-07-22 00:58:34 +00:00
```
2015-07-24 20:47:16 +00:00
If you wish to always use clustering, even on a single core system, see `test-cluster.js`.
Likewise, if you wish to use standalone mode in a particular worker process see `test-standalone.js`.
SQLCipher Considerations
========================
In (hopefully) most cases your AES key won't be available at the time that you want your service
to start listening. (And if it is you might be using a form of
"[encraption](https://twitter.com/nmacdona/status/532677876685217795)"
where you were intending to use a form of "encryption" and should
look into that before going any further.)
To account for this you can pass the `bits` option on `create` and then call `init({ key: key })`
when you receive your key from user input, the key server, etc.
Calling any normal methods will result in an error until `init` is called.
**NOTE:** Because the server process (the master) will use `node-sqlite3` directly,
without any wrapper to protect it, *you* must make sure that it doesn't
make any calls before the key is supplied with `init`.
For this reason it is recommended to not use your master process as an http server, etc.
```js
var cluster = require('cluster');
var sqlite = require('sqlite3-cluster');
var numCores = require('os').cpus().length;
var opts = {
filename: '/tmp/mydb.sqlcipher'
, key: null
, bits: 128
};
sqlite.create(opts).then(function (db) {
// same api as new sqlite3.Database(options.filename)
db.init({
bits: 128
, key: '00000000000000000000000000000000'
}).then(function (db) {
db.run("SELECT ?", ['Hello World!'], function (err) {
if (err) {
console.error('[ERROR]', cluster.isMaster && '0' || cluster.worker.id);
console.error(err);
return;
}
console.log('[this]', cluster.isMaster && '0' || cluster.worker.id);
console.log(this);
});
});
});
```
2015-07-24 20:47:16 +00:00
API
===
The API is exactly the same as `node-sqlite`, with these few exceptions:
1 Database Creation
-------------------
Instead of this:
```js
var db = new require('sqlite3').Database(filename);
```
You must do this:
```js
require('sqlite3-cluster').create(filename);
```
2015-07-24 20:53:58 +00:00
Also, `verbose` becomes an option to pass into the create function,
rather than a function to call. Even though `verbose` is passed on
create, it still behaves globally across all databases.
2015-07-24 20:47:16 +00:00
2 db.escape
-----------
This is an additional helper function.
If you need at any time to concatonate strings with user input
(which you should rarely need to do since `db.run(stmt, arr, fn)`
is usually sufficient), you can use the escape function.
```js
var sqlEscape = require('sqlite3-cluster').escape;
```
also
```js
require('sqlite3-cluster').create(options).then(function (db) {
// obligatory xkcd reference https://xkcd.com/327/
var userInput = db.escape("Robert'); DROP TABLE Students;");
});
```
3 serialize / parallelize
-------------------------
`db.serialize(fn)` and `db.parallelize(fn)` are not supported because it would require
copying a chunk of code from `node-sqlite3` and adapting it.
It wouldn't be a difficult task, just tedious and generally no longer necessary since
recent versions of node include native `Promise`s.
Standalone / Master Mode is raw sqlite3
========================
The `master` in the cluster (meaning `opts.serve = true`) will have a direct connection
to the sqlite3 database using `node-sqlite`, directly.
Likewise, when only one process is being used (`opts.standalone = true`) the listener is
not started and the connection is direct.
If you take a look at `wrapper.js` you'll see that it simply resolves with an instance of
`node-sqlite3`.
Security Warning
================
Note that any application on the system could connect to the socket.
In the future I may add a `secret` field in the options object to be
used for authentication across processes. This would not be difficult,
it's just not necessary for my use case at the moment.