request.js/README.md

443 lines
16 KiB
Markdown
Raw Normal View History

2021-08-21 04:57:43 +00:00
# [@root/request](https://git.rootprojects.org/root/request.js) | a [Root](https://rootprojects.org) project
2019-04-16 18:43:21 +00:00
> Minimalist HTTP client
2019-04-04 07:39:16 +00:00
2020-02-20 17:34:27 +00:00
A lightweight alternative to (and 80/20 drop-in replacement for) request.
Has the 20% of features that 80%+ of people need, in about 500 LoC.
2018-06-19 01:53:06 +00:00
2019-04-16 18:43:21 +00:00
Written from scratch, with zero-dependencies.
2018-06-19 01:39:19 +00:00
## Super simple to use
2021-08-21 04:57:43 +00:00
@root/request is designed to be a drop-in replacement for request. It also supports Promises and async/await by default.
2018-06-19 01:39:19 +00:00
```bash
2019-04-16 18:41:05 +00:00
npm install --save @root/request
2021-01-14 23:35:07 +00:00
# or npm install git+ssh://git@git.therootcompany.com/request.js
```
2018-06-19 01:39:19 +00:00
```js
2019-04-16 18:41:05 +00:00
var request = require('@root/request');
2021-01-14 23:07:48 +00:00
request('http://www.google.com', function (error, response, body) {
2019-10-29 20:31:30 +00:00
console.log('error:', error); // Print the error if one occurred
console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
console.log('body:', body); // Print the HTML for the Google homepage.
2018-06-19 01:39:19 +00:00
});
```
2018-06-19 01:53:06 +00:00
2019-06-04 04:19:11 +00:00
**Using Promises**
```js
var request = require('@root/request');
2019-10-29 20:31:30 +00:00
request('http://www.google.com')
2021-01-14 23:07:48 +00:00
.then(function (response) {
2019-10-29 20:31:30 +00:00
console.log('statusCode:', response.statusCode); // Print the response status code if a response was received
console.log('body:', response.body); // Print the HTML for the Google homepage.
})
2021-01-14 23:07:48 +00:00
.catch(function (error) {
2019-10-29 20:31:30 +00:00
console.log('error:', error); // Print the error if one occurred
});
2019-06-04 04:19:11 +00:00
```
2021-01-14 23:35:07 +00:00
**Streaming**
In order to keep this library lightweight, performant, and keep the code easy to
read, the streaming behavior is **_slightly different_** from that of
`request.js`.
2021-08-06 20:07:03 +00:00
```diff
-var request = require('request');
+var request = require('@root/request');
-var stream = request({ url, headers });
+var stream = await request({ url, headers });
let attachment = await new MailgunAPI.Attachment({
data: stream
})
```
Example:
2021-01-14 23:35:07 +00:00
```js
var request = require('@root/request');
var resp = await request({
url: 'http://www.google.com',
2021-08-06 20:07:03 +00:00
stream: true // true | 'filename.ext' | stream.Writable
2021-01-14 23:35:07 +00:00
});
2021-08-06 20:07:03 +00:00
// 'resp' itself is a ReadableStream
2021-01-14 23:35:07 +00:00
resp.on('data', function () {
// got some data
});
resp.on('end', function () {
// the data has ended
});
2021-08-06 20:07:03 +00:00
// 'resp.stream' is a Promise that is resolved when the read stream is destroyed
await resp.stream; // returns `undefined`
2021-01-14 23:35:07 +00:00
console.log('Done');
```
The difference is that we don't add an extra layer of stream abstraction.
You must use the response from await, a Promise, or the callback.
You can also give a file path:
```js
request({
url: 'http://www.google.com',
stream: '/tmp/google-index.html'
});
```
Which is equivalent to passing a write stream for the file:
```js
request({
url: 'http://www.google.com',
stream: fs.createWriteStream('/tmp/google-index.html')
});
```
## Table of contents
2019-10-29 20:31:30 +00:00
- [Forms](#forms)
- [HTTP Authentication](#http-authentication)
- [Custom HTTP Headers](#custom-http-headers)
- [Unix Domain Sockets](#unix-domain-sockets)
- [**All Available Options**](#requestoptions-callback)
## Forms
`urequest` supports `application/x-www-form-urlencoded` and `multipart/form-data` form uploads.
2019-10-29 20:31:30 +00:00
<!-- For `multipart/related` refer to the `multipart` API. -->
#### application/x-www-form-urlencoded (URL-Encoded Forms)
URL-encoded forms are simple.
```js
2019-10-29 20:31:30 +00:00
request.post('http://service.com/upload', { form: { key: 'value' } });
// or
2019-10-29 20:31:30 +00:00
request.post(
{ url: 'http://service.com/upload', form: { key: 'value' } },
2021-01-14 23:07:48 +00:00
function (err, httpResponse, body) {
2019-10-29 20:31:30 +00:00
/* ... */
}
);
```
2019-10-29 20:31:30 +00:00
<!--
// or
request.post('http://service.com/upload').form({key:'value'})
-->
#### multipart/form-data (Multipart Form Uploads)
For `multipart/form-data` we use the [form-data](https://github.com/form-data/form-data/tree/v2.5.1) library by [@felixge](https://github.com/felixge). For the most cases, you can pass your upload form data via the `formData` option.
To use `form-data`, you must install it separately:
```bash
npm install --save form-data@2.x
```
```js
var formData = {
2019-10-29 20:31:30 +00:00
// Pass a simple key-value pair
my_field: 'my_value',
// Pass data via Buffers
my_buffer: Buffer.from([1, 2, 3]),
// Pass data via Streams
my_file: fs.createReadStream(__dirname + '/unicycle.jpg'),
// Pass multiple values /w an Array
attachments: [
fs.createReadStream(__dirname + '/attachment1.jpg'),
fs.createReadStream(__dirname + '/attachment2.jpg')
],
// Pass optional meta-data with an 'options' object with style: {value: DATA, options: OPTIONS}
// Use case: for some types of streams, you'll need to provide "file"-related information manually.
// See the `form-data` README for more information about options: https://github.com/form-data/form-data
custom_file: {
value: fs.createReadStream('/dev/urandom'),
options: {
filename: 'topsecret.jpg',
contentType: 'image/jpeg'
}
}
};
2019-10-29 20:31:30 +00:00
request.post(
{ url: 'http://service.com/upload', formData: formData },
function optionalCallback(err, httpResponse, body) {
if (err) {
return console.error('upload failed:', err);
}
console.log('Upload successful! Server responded with:', body);
}
);
```
2019-10-29 20:31:30 +00:00
<!--
For advanced cases, you can access the form-data object itself via `r.form()`. This can be modified until the request is fired on the next cycle of the event-loop. (Note that this calling `form()` will clear the currently set form data for that request.)
```js
// NOTE: Advanced use-case, for normal use see 'formData' usage above
var r = request.post('http://service.com/upload', function optionalCallback(err, httpResponse, body) {...})
var form = r.form();
form.append('my_field', 'my_value');
form.append('my_buffer', Buffer.from([1, 2, 3]));
form.append('custom_file', fs.createReadStream(__dirname + '/unicycle.jpg'), {filename: 'unicycle.jpg'});
```
-->
See the [form-data README](https://github.com/form-data/form-data) for more information & examples.
2018-07-07 03:23:46 +00:00
---
## HTTP Authentication
<!--
request.get('http://some.server.com/').auth('username', 'password', false);
// or
request.get('http://some.server.com/').auth(null, null, true, 'bearerToken');
// or
-->
2019-10-29 20:31:30 +00:00
2018-07-07 03:23:46 +00:00
```js
request.get('http://some.server.com/', {
2019-10-29 20:31:30 +00:00
auth: {
user: 'username',
pass: 'password',
sendImmediately: false
}
2018-07-07 03:23:46 +00:00
});
// or
request.get('http://some.server.com/', {
2019-10-29 20:31:30 +00:00
auth: {
bearer: 'bearerToken'
}
2018-07-07 03:23:46 +00:00
});
```
If passed as an option, `auth` should be a hash containing values:
2019-10-29 20:31:30 +00:00
- `user` || `username`
- `pass` || `password`
- `bearer` (optional)
2018-07-07 03:23:46 +00:00
<!--
- `sendImmediately` (optional)
The method form takes parameters
`auth(username, password, sendImmediately, bearer)`.
`sendImmediately` defaults to `true`, which causes a basic or bearer
authentication header to be sent. If `sendImmediately` is `false`, then
`request` will retry with a proper authentication header after receiving a
`401` response from the server (which must contain a `WWW-Authenticate` header
indicating the required authentication method).
-->
Note that you can also specify basic authentication using the URL itself, as
detailed in [RFC 1738](http://www.ietf.org/rfc/rfc1738.txt). Simply pass the
`user:password` before the host with an `@` sign:
```js
var username = 'username',
password = 'password',
url = 'http://' + username + ':' + password + '@some.server.com';
2021-01-14 23:07:48 +00:00
request({ url: url }, function (error, response, body) {
2019-10-29 20:31:30 +00:00
// Do more stuff with 'body' here
2018-07-07 03:23:46 +00:00
});
```
<!--
Digest authentication is supported, but it only works with `sendImmediately`
set to `false`; otherwise `request` will send basic authentication on the
initial request, which will probably cause the request to fail.
-->
Bearer authentication is supported, and is activated when the `bearer` value is
available. The value may be either a `String` or a `Function` returning a
`String`. Using a function to supply the bearer token is particularly useful if
used in conjunction with `defaults` to allow a single function to supply the
last known token at the time of sending a request, or to compute one on the fly.
[back to top](#table-of-contents)
---
## Custom HTTP Headers
HTTP Headers, such as `User-Agent`, can be set in the `options` object.
In the example below, we call the github API to find out the number
of stars and forks for the request repository. This requires a
custom `User-Agent` header as well as https.
```js
var request = require('request');
var options = {
2019-10-29 20:31:30 +00:00
url: 'https://api.github.com/repos/request/request',
headers: {
'User-Agent': 'request'
}
};
function callback(error, response, body) {
2019-10-29 20:31:30 +00:00
if (!error && response.statusCode == 200) {
var info = JSON.parse(body);
console.log(info.stargazers_count + ' Stars');
console.log(info.forks_count + ' Forks');
}
}
request(options, callback);
```
[back to top](#table-of-contents)
---
## UNIX Domain Sockets
2018-07-07 02:41:35 +00:00
`urequest` supports making requests to [UNIX Domain Sockets](https://en.wikipedia.org/wiki/Unix_domain_socket). To make one, use the following URL scheme:
```js
2019-10-29 20:31:30 +00:00
/* Pattern */ 'http://unix:SOCKET:PATH';
/* Example */ request.get(
'http://unix:/absolute/path/to/unix.socket:/request/path'
);
```
Note: The `SOCKET` path is assumed to be absolute to the root of the host file system.
[back to top](#table-of-contents)
2018-06-19 01:53:06 +00:00
---
## request(options, callback)
The first argument can be either a `url` or an `options` object. The only required option is `uri`; all others are optional.
2019-10-29 20:31:30 +00:00
- `uri` || `url` - fully qualified uri or a parsed url object from `url.parse()`
- `method` - http method (default: `"GET"`)
- `headers` - http headers (default: `{}`)
2018-06-19 01:53:06 +00:00
2018-06-19 04:24:27 +00:00
<!-- TODO
- `baseUrl` - fully qualified uri string used as the base url. Most useful with `request.defaults`, for example when you want to do many requests to the same domain. If `baseUrl` is `https://example.com/api/`, then requesting `/end/point?test=true` will fetch `https://example.com/api/end/point?test=true`. When `baseUrl` is given, `uri` must also be a string.
-->
2018-06-19 01:53:06 +00:00
---
2019-10-29 20:31:30 +00:00
- `body` - entity body for PATCH, POST and PUT requests. Must be a `Buffer`, `String` or `ReadStream`. If `json` is `true`, then `body` must be a JSON-serializable object.
- `json` - sets `body` to JSON representation of value and adds `Content-type: application/json` header. Additionally, parses the response body as JSON.
2018-06-19 04:24:27 +00:00
<!-- TODO
- `form` - when passed an object or a querystring, this sets `body` to a querystring representation of value, and adds `Content-type: application/x-www-form-urlencoded` header. When passed no options, a `FormData` instance is returned (and is piped to request). See "Forms" section above.
2018-06-19 01:53:06 +00:00
- `formData` - data to pass for a `multipart/form-data` request. See
[Forms](#forms) section above.
- `multipart` - array of objects which contain their own headers and `body`
attributes. Sends a `multipart/related` request. See [Forms](#forms) section
above.
- Alternatively you can pass in an object `{chunked: false, data: []}` where
`chunked` is used to specify whether the request is sent in
[chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding)
In non-chunked requests, data items with body streams are not allowed.
- `preambleCRLF` - append a newline/CRLF before the boundary of your `multipart/form-data` request.
- `postambleCRLF` - append a newline/CRLF at the end of the boundary of your `multipart/form-data` request.
2018-06-19 04:24:27 +00:00
- `jsonReviver` - a [reviver function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse) that will be passed to `JSON.parse()` when parsing a JSON response body.
2018-06-19 01:53:06 +00:00
- `jsonReplacer` - a [replacer function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify) that will be passed to `JSON.stringify()` when stringifying a JSON request body.
-->
---
2019-10-29 20:31:30 +00:00
- `followRedirect` - follow HTTP 3xx responses as redirects (default: `true`). This property can also be implemented as function which gets `response` object as a single argument and should return `true` if redirects should continue or `false` otherwise.
- `followAllRedirects` - follow non-GET HTTP 3xx responses as redirects (default: `false`)
- `followOriginalHttpMethod` - by default we redirect to HTTP method GET. you can enable this property to redirect to the original HTTP method (default: `false`)
- `maxRedirects` - the maximum number of redirects to follow (default: `10`)
- `removeRefererHeader` - removes the referer header when a redirect happens (default: `false`). **Note:** if true, referer header set in the initial request is preserved during redirect chain.
2018-06-19 01:53:06 +00:00
---
2019-10-29 20:31:30 +00:00
- `encoding` - encoding to be used on `setEncoding` of response data. If `null`, the `body` is returned as a `Buffer`. Anything else **(including the default value of `undefined`)** will be passed as the [encoding](http://nodejs.org/api/buffer.html#buffer_buffer) parameter to `toString()` (meaning this is effectively `utf8` by default). (**Note:** if you expect binary data, you should set `encoding: null`.)
2018-06-19 04:24:27 +00:00
<!-- TODO
- `gzip` - if `true`, add an `Accept-Encoding` header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response. **Note:** Automatic decoding of the response content is performed on the body data returned through `request` (both through the `request` stream and passed to the callback function) but is not performed on the `response` stream (available from the `response` event) which is the unmodified `http.IncomingMessage` object which may contain compressed data. See example below.
2018-06-19 01:53:06 +00:00
- `jar` - if `true`, remember cookies for future use (or define your custom cookie jar; see examples section)
-->
---
## Convenience methods
There are also shorthand methods for different HTTP METHODs and some other conveniences.
### request.defaults(options)
This method **returns a wrapper** around the normal request API that defaults
to whatever options you pass to it.
**Note:** `request.defaults()` **does not** modify the global request API;
instead, it **returns a wrapper** that has your default settings applied to it.
**Note:** You can call `.defaults()` on the wrapper that is returned from
`request.defaults` to add/override defaults that were previously defaulted.
For example:
2019-10-29 20:31:30 +00:00
2018-06-19 01:53:06 +00:00
```js
//requests using baseRequest() will set the 'x-token' header
var baseRequest = request.defaults({
2019-10-29 20:31:30 +00:00
headers: { 'x-token': 'my-token' }
});
2018-06-19 01:53:06 +00:00
//requests using specialRequest() will include the 'x-token' header set in
//baseRequest and will also include the 'special' header
var specialRequest = baseRequest.defaults({
2019-10-29 20:31:30 +00:00
headers: { special: 'special value' }
});
2018-06-19 01:53:06 +00:00
```
### request.METHOD()
These HTTP method convenience functions act just like `request()` but with a default method already set for you:
2019-10-29 20:31:30 +00:00
- _request.get()_: Defaults to `method: "GET"`.
- _request.post()_: Defaults to `method: "POST"`.
- _request.put()_: Defaults to `method: "PUT"`.
- _request.patch()_: Defaults to `method: "PATCH"`.
- _request.del() / request.delete()_: Defaults to `method: "DELETE"`.
- _request.head()_: Defaults to `method: "HEAD"`.
- _request.options()_: Defaults to `method: "OPTIONS"`.
2018-06-19 01:53:06 +00:00
2018-07-07 03:23:46 +00:00
---
2018-06-19 01:53:06 +00:00
## Debugging
There are at least <!--three--> two ways to debug the operation of `request`:
1. Launch the node process like `NODE_DEBUG=urequest node script.js`
(`lib,request,otherlib` works too).
2019-04-16 18:41:05 +00:00
2. Set `require('@root/request').debug = true` at any time (this does the same thing
2018-06-19 01:53:06 +00:00
as #1).
2018-06-19 04:24:27 +00:00
<!-- TODO
2018-06-19 01:53:06 +00:00
3. Use the [request-debug module](https://github.com/request/request-debug) to
view request and response headers and bodies.
[back to top](#table-of-contents)
-->
[back to top](#table-of-contents)