pcap

Every packet in a pcap or pcapng file, as a list of maps. Fails if the file isn't a capture. Fails too past MAX_PACKETS packets or 512 MB of packet data: use pcap.each() for those.

Arguments

• limit read only the first limit packets.

Fields of each packet

• "number", "time", "when", "length"
• "protocol" like tcp, udp, icmp or arp.
• "src", "dst" the addresses.
• "sport", "dport" the ports.
• "flags" the TCP flags.
• "ttl"
• "app" the application protocol, like dns, http, tls or ssh.
• "dns", "http", "tls", "arp", "icmp" detail maps for that protocol.
• "payload" the payload bytes.
• "payload_length" the size of the payload.
• "summary" one readable line.
• "raw" the whole frame.

Like pcap.read() but for capture bytes already in memory.

Call fn(packet) for every packet in the file, one at a time, streaming from disk, so captures of any size use little memory (and there is no packet limit). .gz and one-file .zip captures are unpacked as they are read. Return false from fn to stop early. Gives back how many packets fn saw.

"pcap", "pcap-ns" (nanosecond) or "pcapng"; fails for other files.

Group packets into conversations, oldest first. A conversation is one protocol, addresses and ports, in both directions.

Fields of each flow

• "protocol", "src", "sport", "dst", "dport"
• "packets", "bytes", "client_bytes", "server_bytes"
• "first", "last" the first and last timestamps.
• "first_when", "last_when" the same times as text.
• "duration"
• "flags", "app", "service"
• "names" DNS names, HTTP hosts and SNI names seen.
• "state" for TCP, one of:
  • "established", "refused", "no_reply", "half_open" or "other"

The busiest addresses by bytes (default: top 10).

Fields of each address

• "ip", "packets", "bytes", "sent", "received", "peers"

Every DNS question asked, with the answers that came back.

Fields of each query

• "time", "when", "src", "dst", "name", "type"
• "answers" a list of the values that came back.
• "rcode", "answered"

Every plain HTTP request.

Fields of each request

• "time", "when", "src", "dst", "dport"
• "method", "host", "path", "url"
• "user_agent", "headers", "content_type"

The server names from TLS Client Hellos. They show what HTTPS connections are for, even though the traffic is encrypted.

Fields of each hello

• "time", "when", "src", "dst", "dport"
• "sni" the server name.
• "version", "alpn"

Sources that sent bare TCP SYNs to many ports of one host (default: 10 or more ports).

Fields of each scan

• "src", "dst", "ports", "count"
• "open_ports" the ports that answered SYN-ACK.
• "first_when", "last_when", "duration"

An overview of a capture.

Fields

• "format", "packets", "bytes"
• "first_when", "last_when", "duration"
• "hosts", "flows"
• "protocols" a map of name -> count.
• "apps"
• "ports" the top destination ports.
• "top_talkers", "dns_queries", "http_requests", "tls_hellos", "port_scans"

Keep the packets for which fn(packet) is true (same as packets.filter(fn); here so pipelines read naturally).

Save packets (from pcap.read, maybe filtered) as a classic pcap file; gives back how many were written. Lets you cut a huge capture down to the packets that matter and open the result in Wireshark.

Put fragmented IPv4 and IPv6 packets back together. Each complete set of fragments becomes one packet, decoded again (so TCP, UDP, DNS and HTTP inside it show up), in the place where its last fragment arrived. Sets with a piece missing are left as they are.

Fields added to a reassembled packet

• fragments the numbers of the packets it was built from.

Fragments that are left over keep a "fragment" map with "offset", "more" and "id".

Every TCP connection with the bytes each side sent, in order. Fragments are joined first, segments that arrived out of order are sorted, and repeated bytes (retransmissions and overlaps) are kept once; where the same bytes arrive twice, the first copy wins.

Options

• max_bytes the most bytes kept for each side of a connection (default: 16 MB).

Fields of each stream

• id 1 for the oldest connection, then 2, 3 and so on.
• src, sport the client (the side that sent the first SYN).
• dst, dport the server.
• state how the connection went, one of:
  • "established", "closing", "closed" (FIN both ways), "reset", "refused", "no_reply"
  • "partial" the capture started after the handshake.
• handshake true when both SYNs were seen.
• packets, first, last, first_when, last_when, duration
• app "http", "tls", "ssh" or nil, from the first bytes.
• service the usual name for the port.
• client, server what each side sent, as a map.

Fields of client and server

• data the bytes, in order.
• bytes, packets
• gaps holes where segments were never captured, as maps of "at" (where in data) and "length".
• missing how many bytes the gaps add up to.
• retransmissions, overlaps, out_of_order how many segments were repeats, partly repeats, or late.
• truncated true when max_bytes cut the data short.
• fin true when this side sent FIN.

Every HTTP request with its response, read from the reassembled TCP streams, so bodies split over many packets come out whole. Handles keep-alive, Content-Length, chunked transfer encoding and gzip or deflate content. Damaged packets and streams are skipped.

Options

• max_bytes the most bytes kept for each side of a connection (default: 16 MB). One call unpacks at most 4 times max_bytes of gzip and deflate in total; bodies past that come back as sent.

Fields of each exchange

• stream the id from pcap.streams().
• time, when when the request was sent.
• src, sport, dst, dport
• method, host, path, url, user_agent
• request_headers, request_body
• status, reason, response_headers, content_type, response_body
• request_decoded, response_decoded false when a gzip or deflate body comes back as sent, because it was broken, bigger than max_bytes, or over the call's unpacking limit.
• complete true when both bodies arrived in full.

A request with no answer has status nil.

The files carried over plain HTTP: response bodies (downloads) and request bodies (uploads), with their hashes, ready to check against threat intelligence or save with fs.write_bytes().

Options

• max_bytes the most bytes kept for each side of a connection (default: 16 MB). One call unpacks at most 4 times max_bytes of gzip and deflate in total.

Fields of each object

• name from the Content-Disposition header or the end of the URL. Never a folder path, and always a name Windows can save.
• direction "download" or "upload".
• url, host, status, content_type
• size the number of bytes.
• sha256, md5 hashes of the bytes, as hex text.
• data the bytes themselves.
• decoded false when gzip or deflate bytes were kept as sent (see max_bytes).
• stream, time, when
• src, dst, dport the client and the server.

The usual service name for a port (443 -> "https"), or nil.