buff

The swiss-army-knife of byte manipulation.

Features:

Move between data formats with ease!
Encode / decode between Base58, Base64, Bech32 and more.
Buff object recognized as native Uint8Array.
Prepend, append, sort, split and join arrays of multiple formats.
Read and prefix varints.
Convert blobs of data into consumable streams.
Uses DataView.setUint8 for ultra-fast performace.
Supports endianess for all the things!

How to Import

This library is designed to support classic and modern ESM imports, in both a nodejs and browser environment.

Example install via NPM or yarn:

npm install @cmdcode/buff || yarn add @cmdcode/buff

Classic import into a nodejs project:

const { Buff, Bytes } = require('@cmdcode/buff')

Modern import into an nodejs project:

import { Buff, Bytes } from '@cmdcode/buff'

Classic import into a browser-based project:

<script src="https://unpkg.com/@cmdcode/buff/dist/browser.js"></script>
<script>
  const { Buff, Bytes } = window.buff
</script>

Modern import into a browser-based project:

<script type="module">
  import { Buff, Bytes } from "https://unpkg.com/@cmdcode/buff/dist/module.mjs" 
</script>

How to Use

The Buff class is an extention of the base Uint8Array class. It provides the same default functionality of a Uint8Array, and can be used as a drop-in replacement for Uint8Array. Typescript will treat Buff as a Uint8Array object.

import { Buff, Bytes } from '@cmdcode/buff'

// Bytes covers value types that are convertable to Uint8Array.
type Bytes  = string | number | bigint | Uint8Array | Buff
// You can optionally specify the endianess of data.
type Endian = 'le' | 'be'

const bytes = new Buff (
  data    : Bytes | Bytes[] | ArrayBuffer,  
  size   ?: number, // Specify the size of the array (for padding)
  endian ?: Endian  // Specify the endianess of the array.
)

You can convert from many different types and formats into a Buff object.

Buff
  .any     = (data : any, size ?: number)        => Buff,
  .raw     = (data : Uint8Array, size ?: number) => Buff,
  .str     = (data : string, size ?: number)     => Buff,
  .hex     = (data : string, size ?: number)     => Buff,
  .bin     = (data : string, size ?: number)     => Buff,
  .num     = (data : number, size ?: number)     => Buff,
  .big     = (data : bigint, size ?: number)     => Buff,
  .bytes   = (data : Bytes,  size ?: number)     => Buff,
  .json    = (data : T,  replacer ?: Replacer)   => Buff,
  .b58chk  = (data : string)                     => Buff,
  .base64  = (data : string)                     => Buff,
  .b64url  = (data : string)                     => Buff,

  .bech32  = (
    data        : string,  // Data to be encoded
    limit      ?: number,  // Enforce a character limit.
    chk_prefix ?: string   // Enforce a certain prefix.
  ) => Buff,

  .bech32m = (
    data        : string,  // Data to be encoded
    limit      ?: number,  // Enforce a character limit.
    chk_prefix ?: string   // Enforce a certain prefix.
  ) => Buff
}

With Buff, you have access to an extensive API for converting between formats.

const bytes = new Buff(data)

/* Quicky convert into many formats using getters. */

bytes
  .arr     => number[]    // Convert to a number array.
  .num     => number      // Convert to a Number.
  .big     => bigint      // Convert to a BigInt.
  .str     => string      // Convert to a UTF8 string.
  .hex     => string      // Convert to a hex string.
  .raw     => Uint8Array  // Convert to a pure Uint8Array.
  .bin     => string      // Convert to a binary string.
  .b58chk  => string      // Convert to base58 with checksum.
  .base64  => string      // Convert to base64 string.
  .b64url  => string      // Convert to base64url string.
  .digest  => Buff        // Convert to a sha256 digest.
  .id      => string      // Convert to a digest (hex string).
  .stream  => Stream      // Convert to a Stream object.

/* There are a few export methods that support extra params. */

bytes
  .toNum     : (endian ?: Endian)                  => number
  .toBig     : (endian ?: Endian)                  => bigint
  .toBin     : ()                                  => string
  .toHash    : ()                                  => Buff
  .toJson    : (reviver ?: Reviver)                => T
  .toBech32  : (prefix : string, limit ?: number)  => string
  .toBech32m : (prefix : string, limit ?: number)  => string

In addition to format conversion, you can perform many other convenient tasks.

Buff = {
  // Standard UTF-8 string-to-bytes encoding.
  encode : (str : string) => Uint8Array,
  // Standard UTF-8 bytes-to-string decoding.
  decode : (bytes : Uint8Array) => string,
  // Same as Uint8Array.from(), but returns a Buff object.
  from (data : Uint8Array | number[]) => Buff
  // Same as Uint8Array.of(), but returns a Buff object.
  of (...data : number[]) => Buff,
  // Join together multiple arrays of bytes.
  join : (array : Bytes[]) => Buff,
  // Sort multiple arrays of bytes in lexicographic order.
  sort (arr : Bytes[], size ?: number) => Buff[],
  // Return a buffer object with random data (uses webcrypto).
  random (size : number) => Buff,
  // Converts a number into a 'varint' for byte streams.
  varInt : (num : number, endian ?: Endian) => Buff
}

const bytes = new Buff(data)

bytes
  // Append data to your ubber object
  .append (data : Bytes) => Buff
  // Prepend data to your buffer object.
  .prepend (data : Bytes) => Buff
  // Encode the size of your buffer as a varint and prepend it.
  .prefixSize (endian ?: Endian) => Buff
  // Same as Uint8Array.reverse(), but returns a Buff object.
  .reverse () => Buff
  // Identical to Uint8Array.set() method.
  .set (array : ArrayLike<number>, offset ?: number) => void
  // Same as Uint8Array.slice(), but returns a Buff object.
  .slice (start ?: number, end ?: number) => Buff
  // Same as Uint8Array.subarray(), but returns a Buff object.
  .subarray (begin ?: number, end ?: number) => Buff
  // Same as Uint8Array.set(), but allows Bytes as input.
  .write (data : Bytes, offset ?: number) => void

The Stream tool will take a blob of data and allow you to consume it byte-per-byte.

import { Stream } from '@cmdcode/buff'

// Convert data into a stream object.
const stream = new Stream(data)

// You can convert a buff object into a stream.
const stream = new Buff(data).stream

stream
  // Reads x number of bytes, does not consume the stream.
  .peek(size: number) => Buff
  // Reads x number of bytes, consumes the stream.
  .read(size: number) => bytes
  // Reads the next bytes(s) as a varint, returns the number value.
  .readSize (endian ?: Endian) => number

Dependencies

This library uses minimal dependences.

@scure/base
For performing encoding and decoding of many formats.
https://github.com/paulmillr/scure-base

@noble/hashes For creating hashes using sha256.
https://github.com/paulmillr/noble-hashes

Special thanks to Paul Miller for his wonderful work.

Bugs / Issues

Please feel free to post any questions or bug reports on the issues page!

Development / Testing

This project uses eslint and typescript for development, tape for unit tests, and rollup for bundling releases.

yarn test    || npm run test
yarn release || npm run release

Contributions

All contributions are welcome!

License

Use this code however you like! No warranty!

chrisalmeida/buff