TB v3.2.0

binary

Work with bytes: make them from hex or lists, pack and unpack numbers the way file formats and network protocols store them (like Python's struct), read little- and big-endian numbers at an offset, show a hex dump, measure entropy, and sniff what kind of file something is. Write bytes as b"..." in TB.

import binary

Example

import binary

# A tiny file header: magic, version (16-bit), record count (32-bit), little-endian
let header = binary.pack("<4sHI", b"TBDB", 3, 1200)
say(header.hex())
let [magic, version, count] = binary.unpack("<4sHI", header)
say(magic, version, count)

say(binary.read_u16be(b"\x01\xbb"))          # 443, as a network protocol stores it
say(binary.hexdump(b"MZ\x90\x00 tiny header"))
say(binary.entropy(b"aaaaaaaa"), binary.entropy(binary.from_list(range(256))))
say(binary.sniff(b"%PDF-1.7\n").type)

Functions

from_hex(text)

returns bytes

Bytes from hex text: from_hex("4d5a90") is b"MZ\x90". Spaces and colons are ignored.

Visualizehow binary.from_hex works

binary.from_hex("4d5a9000") b"MZ\x90\x00"

binary.from_hex: inputs flowing into the function and the resulttext"4d5a9000"binary.from_hexresult · bytes4dM5aZ90·00·text"4d:5a:90:00"binary.from_hexresult · bytes4dM5aZ90·00·text"7f454c46"binary.from_hexresult · bytes7f·45E4cL46F

Real result: worked out by running this example with TB while the site was built.

to_hex(data, separator?)

returns text

The bytes as hex text: to_hex(b"MZ") is "4d5a"; to_hex(b"MZ", " ") is "4d 5a".

Visualizehow binary.to_hex works

binary.to_hex(b"MZ\x90\x00", " ") "4d 5a 90 00"

Drawing…

Real result: worked out by running this example with TB while the site was built.

from_list(numbers)

returns bytes

Bytes from a list of numbers 0 to 255: from_list([77, 90]) is b"MZ".

Visualizehow binary.from_list works

binary.from_list([77, 90, 144, 0]) b"MZ\x90\x00"

binary.from_list: inputs flowing into the function and the resultnumbers77901440binary.from_listresult · bytes4dM5aZ90·00·

Real result: worked out by running this example with TB while the site was built.

from_text(text, encoding?)

returns bytes

The text as bytes (UTF-8 unless you name another encoding). Same as text.bytes().

Visualizehow binary.from_text works

binary.from_text("héllo", "utf-8") b"h\xc3\xa9llo"

Drawing…

Real result: worked out by running this example with TB while the site was built.

zeros(n)

returns bytes

n zero bytes.

Visualizehow binary.zeros works

binary.zeros(4) b"\x00\x00\x00\x00"

binary.zeros: inputs flowing into the function and the resultn2binary.zerosresult · bytes00·00·n4binary.zerosresult · bytes00·00·00·00·n8binary.zerosresult · bytes00·00·00·00·00·00·00·00·

Real result: worked out by running this example with TB while the site was built.

concat(data, ...)

returns bytes

Join bytes (or a list of bytes) into one.

Visualizehow binary.concat works

binary.concat(b"MZ", b"\x90\x00", b"PE") b"MZ\x90\x00PE"

binary.concat: inputs flowing into the function and the resultdatab"MZ"b"\x90\x00"b"PE"binary.concatresult · bytes4dM5aZ90·00·50P45E

Real result: worked out by running this example with TB while the site was built.

size(format)

returns number

How many bytes pack(format, ...) makes: size("<HI") is 6.

Visualizehow binary.size works

binary.size("<HI") 6

binary.size: inputs flowing into the function and the resultformat"<HI"binary.sizeresult · number6format"<B"binary.sizeresult · number1format"<Q"binary.sizeresult · number8format">HHI"binary.sizeresult · number8

Real result: worked out by running this example with TB while the site was built.

pack(format, value, ...)

returns bytes

Numbers (and bytes) packed the way file formats store them. pack("<HI", 1, 2) is 6 bytes, little-endian.

Format letters

  • < little-endian (the default).
  • > or !: big-endian.
  • B b H h I i Q q: whole numbers. A capital letter means unsigned.
    • B b: 8-bit
    • H h: 16-bit
    • I i: 32-bit
    • Q q: 64-bit
  • f d: 32-bit and 64-bit decimals.
  • Ns N bytes, like 4s.
  • x a zero byte.
Visualizehow binary.pack works

binary.pack("<HI", 1, 2) b"\x01\x00\x02\x00\x00\x00"

binary.pack: inputs flowing into the function and the resultformat"<HI"value1binary.packresult · bytes01·00·02·00·00·00·format">HI"value1binary.packresult · bytes00·01·00·00·00·02·

Real result: worked out by running this example with TB while the site was built.

unpack(format, data, offset?)

returns list

Read numbers out of bytes, starting at offset (default 0): unpack("<HI", data) gives [first, second]. Same format letters as pack().

Visualizehow binary.unpack works

binary.unpack("<HI", b"\x01\x00\x02\x00\x00\x00", 0) [1, 2]

binary.unpack: inputs flowing into the function and the resultformat"<HI"data01·00·02·00·00·00·offset?0binary.unpackresult · list12format">HI"data01·00·02·00·00·00·offset?0binary.unpackresult · list25633554432

Real result: worked out by running this example with TB while the site was built.

read_u8(data, offset?)

returns number

The unsigned 8-bit little-endian number at offset (default 0).

Visualizehow binary.read_u8 works

binary.read_u8(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 144

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i8(data, offset?)

returns number

The signed 8-bit little-endian number at offset (default 0).

Visualizehow binary.read_i8 works

binary.read_i8(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) -112

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u16le(data, offset?)

returns number

The unsigned 16-bit little-endian number at offset (default 0).

Visualizehow binary.read_u16le works

binary.read_u16le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 144

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i16le(data, offset?)

returns number

The signed 16-bit little-endian number at offset (default 0).

Visualizehow binary.read_i16le works

binary.read_i16le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 144

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u16be(data, offset?)

returns number

The unsigned 16-bit big-endian number at offset (default 0).

Visualizehow binary.read_u16be works

binary.read_u16be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 36864

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i16be(data, offset?)

returns number

The signed 16-bit big-endian number at offset (default 0).

Visualizehow binary.read_i16be works

binary.read_i16be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) -28672

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u32le(data, offset?)

returns number

The unsigned 32-bit little-endian number at offset (default 0).

Visualizehow binary.read_u32le works

binary.read_u32le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 196752

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i32le(data, offset?)

returns number

The signed 32-bit little-endian number at offset (default 0).

Visualizehow binary.read_i32le works

binary.read_i32le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 196752

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u32be(data, offset?)

returns number

The unsigned 32-bit big-endian number at offset (default 0).

Visualizehow binary.read_u32be works

binary.read_u32be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) 2415919872

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i32be(data, offset?)

returns number

The signed 32-bit big-endian number at offset (default 0).

Visualizehow binary.read_i32be works

binary.read_i32be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 2) -1879047424

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u64le(data, offset?)

returns number

The unsigned 64-bit little-endian number at offset (default 0).

Visualizehow binary.read_u64le works

binary.read_u64le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 0) 12894362189

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i64le(data, offset?)

returns number

The signed 64-bit little-endian number at offset (default 0).

Visualizehow binary.read_i64le works

binary.read_i64le(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 0) 12894362189

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_u64be(data, offset?)

returns number

The unsigned 64-bit big-endian number at offset (default 0).

Visualizehow binary.read_u64be works

binary.read_u64be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 0) 5573925818549141504

Drawing…

Real result: worked out by running this example with TB while the site was built.

read_i64be(data, offset?)

returns number

The signed 64-bit big-endian number at offset (default 0).

Visualizehow binary.read_i64be works

binary.read_i64be(b"MZ\x90\x00\x03\x00\x00\x00\xff\xff\xff\xfe\x01\x02\x03\x04", 0) 5573925818549141504

Drawing…

Real result: worked out by running this example with TB while the site was built.

hexdump(data, width?)

returns text

A classic hex dump: offset, hex bytes and the printable characters, width bytes per row (default 16).

Visualizehow binary.hexdump works

binary.hexdump(b"MZ\x90\x00This program cannot", 8) "00000000 4d 5a 90 00 54 68 69 73 |MZ..This|\n00000008 20 70 72 6f 67 72 61 6d | prog…

Drawing…

Real result: worked out by running this example with TB while the site was built.

entropy(data)

returns number

Shannon entropy from 0 (all the same byte) to 8 (random). Packed or encrypted data is usually above 7.2.

Visualizehow binary.entropy works

binary.entropy(b"AAAAAAAA") 0

binary.entropy: inputs flowing into the function and the resultdata41A41A41A41A41A41A41A41Abinary.entropyresult · number0data"hello world"binary.entropyresult · number2.8454databinary.from_hex("8f3a91c2e07d5b16")binary.entropyresult · number3

Real result: worked out by running this example with TB while the site was built.

xor(data, key)

returns bytes

Each byte XORed with the key (bytes, repeated; or one number 0-255). XOR twice to get the original back.

Visualizehow binary.xor works

binary.xor(b"hi", 42) b"BC"

Drawing…

Real result: worked out by running this example with TB while the site was built.

file_type(path)

returns map

What kind of file this is, from its first bytes (not its name): {type, description, mime, extension, path, size}. Knows PE, ELF, Mach-O, PDF, ZIP, PNG, JPEG, GIF, gzip and more.

Visualizehow binary.file_type works

binary.file_type("apps/data/sample.pcap") {"type": "pcap", "description": "Network capture (pcap)", "mime": "application/vnd.tcpdum…

binary.file_type: inputs flowing into the function and the resultpath"apps/data/sample.pcap"binary.file_typeresult · maptype"pcap"description"Network capture (pcap)"mime"application/vnd.tcpdump.pcap"extension".pcap"path"apps/data/sample.evtx"binary.file_typeresult · maptype"evtx"description"Windows event log (EVTX)"mime"application/x-ms-evtx"extension".evtx"path"notes.txt"binary.file_typeresult · maptype"text"description"Plain text"mime"text/plain"extension".txt"

Real result: worked out by running this example with TB while the site was built.

sniff(data)

returns map

Like file_type(), but for bytes you already have: {type, description, mime, extension}.

Visualizehow binary.sniff works

binary.sniff(binary.from_hex("504b0304")) {"type": "zip", "description": "ZIP archive", "mime": "application/zip", "extension": ".z…

binary.sniff: inputs flowing into the function and the resultdatabinary.from_hex("504b0304")binary.sniffresult · maptype"zip"description"ZIP archive"mime"application/zip"extension".zip"databinary.from_hex("25504446")binary.sniffresult · maptype"pdf"description"PDF document"mime"application/pdf"extension".pdf"data4dM5aZ90·00·binary.sniffresult · maptype"text"description"Plain text"mime"text/plain"extension".txt"

Real result: worked out by running this example with TB while the site was built.