2015-07-24 20:47:16 +00:00
|
|
|
SQLite3 Cluster
|
|
|
|
===============
|
2015-07-22 00:58:34 +00:00
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
Works with node cluster, or completely and unrelated node processes.
|
2015-07-22 00:58:34 +00:00
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
**Note**: Most people would probably prefer to just use PostgreSQL rather than
|
|
|
|
wrap SQLite as a service... but I am not most people.
|
2015-07-22 00:58:34 +00:00
|
|
|
|
|
|
|
Node.js runs on a single core, which isn't very effective.
|
|
|
|
|
|
|
|
You can run multiple Node.js instances to take advantage of multiple cores,
|
|
|
|
but if you do that, you can't use SQLite in each process.
|
|
|
|
|
|
|
|
This module will either run client-server style in environments that benefit from it
|
|
|
|
(such as the Raspberry Pi 2 with 4 cores), or in-process for environments that don't
|
|
|
|
(such as the Raspberry Pi B and B+).
|
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
This also works with **SQLCipher**.
|
|
|
|
|
2015-07-22 00:58:34 +00:00
|
|
|
Usage
|
|
|
|
=====
|
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
The default behavior is to try to connect to a master and, if that fails, to become the master.
|
|
|
|
|
|
|
|
However, if you are in fact using the `cluster` rather than spinning up random instances,
|
|
|
|
you'll probably prefer to use this pattern:
|
|
|
|
|
2015-07-22 00:58:34 +00:00
|
|
|
```js
|
2015-07-24 20:47:16 +00:00
|
|
|
var cluster = require('cluster');
|
|
|
|
var sqlite = require('sqlite3-cluster');
|
|
|
|
var numCores = require('os').cpus().length;
|
|
|
|
|
2015-07-22 00:58:34 +00:00
|
|
|
var opts = {
|
2015-07-24 20:47:16 +00:00
|
|
|
filename: '/tmp/mydb.sqlcipher'
|
|
|
|
, sock: '/tmp/mydb.sqlcipher.sock'
|
2015-07-22 00:58:34 +00:00
|
|
|
, verbose: false
|
2015-07-24 20:47:16 +00:00
|
|
|
|
|
|
|
// a good default to use for instances where you might want
|
|
|
|
// to cluster or to run standalone, but with the same API
|
|
|
|
, serve: cluster.isMaster
|
|
|
|
, connect: cluster.isWorker
|
|
|
|
, standalone: (1 === numCores) // overrides serve and connect
|
|
|
|
|
2015-07-24 20:48:25 +00:00
|
|
|
// if using SQLCipher, you can supply the key and desired bit-length and the
|
|
|
|
// appropriate PRAGMA statements will be issued before the database is returned
|
2015-07-24 20:47:16 +00:00
|
|
|
, key: '00000000000000000000000000000000'
|
|
|
|
, bits: 128
|
2015-07-22 00:58:34 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
sqlite.create(opts).then(function (db) {
|
2015-07-24 20:47:16 +00:00
|
|
|
// same api as new sqlite3.Database(options.filename)
|
|
|
|
|
2015-07-29 00:04:54 +00:00
|
|
|
db.run("SELECT ?", ['Hello World!'], function (err) {
|
2015-07-24 20:47:16 +00:00
|
|
|
if (err) {
|
|
|
|
console.error('[ERROR]', cluster.isMaster && '0' || cluster.worker.id);
|
|
|
|
console.error(err);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
console.log('[this]', cluster.isMaster && '0' || cluster.worker.id);
|
|
|
|
console.log(this);
|
|
|
|
});
|
2015-07-22 00:58:34 +00:00
|
|
|
});
|
2015-07-24 22:10:47 +00:00
|
|
|
|
|
|
|
process.on('unhandledPromiseRejection', function (err) {
|
|
|
|
console.error('Unhandled Promise Rejection');
|
|
|
|
console.error(err);
|
|
|
|
console.error(err.stack);
|
|
|
|
|
|
|
|
throw err;
|
|
|
|
});
|
2015-07-22 00:58:34 +00:00
|
|
|
```
|
2015-07-24 20:47:16 +00:00
|
|
|
|
|
|
|
If you wish to always use clustering, even on a single core system, see `test-cluster.js`.
|
|
|
|
|
|
|
|
Likewise, if you wish to use standalone mode in a particular worker process see `test-standalone.js`.
|
|
|
|
|
2015-07-29 00:04:54 +00:00
|
|
|
SQLCipher Considerations
|
|
|
|
========================
|
|
|
|
|
|
|
|
In (hopefully) most cases your AES key won't be available at the time that you want your service
|
|
|
|
to start listening. (And if it is you might be using a form of
|
|
|
|
"[encraption](https://twitter.com/nmacdona/status/532677876685217795)"
|
|
|
|
where you were intending to use a form of "encryption" and should
|
|
|
|
look into that before going any further.)
|
|
|
|
|
|
|
|
To account for this you can pass the `bits` option on `create` and then call `init({ key: key })`
|
|
|
|
when you receive your key from user input, the key server, etc.
|
|
|
|
|
|
|
|
Calling any normal methods will result in an error until `init` is called.
|
|
|
|
|
|
|
|
**NOTE:** Because the server process (the master) will use `node-sqlite3` directly,
|
|
|
|
without any wrapper to protect it, *you* must make sure that it doesn't
|
|
|
|
make any calls before the key is supplied with `init`.
|
|
|
|
For this reason it is recommended to not use your master process as an http server, etc.
|
|
|
|
|
|
|
|
```js
|
|
|
|
var cluster = require('cluster');
|
|
|
|
var sqlite = require('sqlite3-cluster');
|
|
|
|
var numCores = require('os').cpus().length;
|
|
|
|
|
|
|
|
var opts = {
|
|
|
|
filename: '/tmp/mydb.sqlcipher'
|
|
|
|
|
|
|
|
, key: null
|
|
|
|
, bits: 128
|
|
|
|
};
|
|
|
|
|
|
|
|
sqlite.create(opts).then(function (db) {
|
|
|
|
// same api as new sqlite3.Database(options.filename)
|
|
|
|
|
|
|
|
db.init({
|
|
|
|
bits: 128
|
|
|
|
, key: '00000000000000000000000000000000'
|
|
|
|
}).then(function (db) {
|
|
|
|
db.run("SELECT ?", ['Hello World!'], function (err) {
|
|
|
|
if (err) {
|
|
|
|
console.error('[ERROR]', cluster.isMaster && '0' || cluster.worker.id);
|
|
|
|
console.error(err);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
console.log('[this]', cluster.isMaster && '0' || cluster.worker.id);
|
|
|
|
console.log(this);
|
|
|
|
});
|
|
|
|
});
|
|
|
|
});
|
|
|
|
```
|
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
API
|
|
|
|
===
|
|
|
|
|
|
|
|
The API is exactly the same as `node-sqlite`, with these few exceptions:
|
|
|
|
|
|
|
|
1 Database Creation
|
|
|
|
-------------------
|
|
|
|
|
|
|
|
Instead of this:
|
|
|
|
|
|
|
|
```js
|
|
|
|
var db = new require('sqlite3').Database(filename);
|
|
|
|
```
|
|
|
|
|
|
|
|
You must do this:
|
|
|
|
|
|
|
|
```js
|
|
|
|
require('sqlite3-cluster').create(filename);
|
|
|
|
```
|
|
|
|
|
2015-07-24 20:53:58 +00:00
|
|
|
Also, `verbose` becomes an option to pass into the create function,
|
|
|
|
rather than a function to call. Even though `verbose` is passed on
|
|
|
|
create, it still behaves globally across all databases.
|
|
|
|
|
2015-07-24 20:47:16 +00:00
|
|
|
2 db.escape
|
|
|
|
-----------
|
|
|
|
|
|
|
|
This is an additional helper function.
|
|
|
|
|
|
|
|
If you need at any time to concatonate strings with user input
|
|
|
|
(which you should rarely need to do since `db.run(stmt, arr, fn)`
|
|
|
|
is usually sufficient), you can use the escape function.
|
|
|
|
|
|
|
|
```js
|
|
|
|
var sqlEscape = require('sqlite3-cluster').escape;
|
|
|
|
```
|
|
|
|
|
|
|
|
also
|
|
|
|
|
|
|
|
```js
|
|
|
|
require('sqlite3-cluster').create(options).then(function (db) {
|
|
|
|
// obligatory xkcd reference https://xkcd.com/327/
|
|
|
|
var userInput = db.escape("Robert'); DROP TABLE Students;");
|
|
|
|
});
|
|
|
|
```
|
|
|
|
|
|
|
|
3 serialize / parallelize
|
|
|
|
-------------------------
|
|
|
|
|
|
|
|
`db.serialize(fn)` and `db.parallelize(fn)` are not supported because it would require
|
|
|
|
copying a chunk of code from `node-sqlite3` and adapting it.
|
|
|
|
|
|
|
|
It wouldn't be a difficult task, just tedious and generally no longer necessary since
|
|
|
|
recent versions of node include native `Promise`s.
|
|
|
|
|
|
|
|
Standalone / Master Mode is raw sqlite3
|
|
|
|
========================
|
|
|
|
|
|
|
|
The `master` in the cluster (meaning `opts.serve = true`) will have a direct connection
|
|
|
|
to the sqlite3 database using `node-sqlite`, directly.
|
|
|
|
|
|
|
|
Likewise, when only one process is being used (`opts.standalone = true`) the listener is
|
|
|
|
not started and the connection is direct.
|
|
|
|
|
|
|
|
If you take a look at `wrapper.js` you'll see that it simply resolves with an instance of
|
|
|
|
`node-sqlite3`.
|
|
|
|
|
|
|
|
Security Warning
|
|
|
|
================
|
|
|
|
|
|
|
|
Note that any application on the system could connect to the socket.
|
|
|
|
|
|
|
|
In the future I may add a `secret` field in the options object to be
|
|
|
|
used for authentication across processes. This would not be difficult,
|
|
|
|
it's just not necessary for my use case at the moment.
|