Dissector

This presentations goal it to introduce the features of the Dissector and how to configure it.

Dissector

This presentations goal it to introduce the features of the Dissector and how to configure it.

The challenge

I want to dissect a field to different target fields.

from this:

[21]:

document = { "message": "Oct 17 11:54:21 dev-machine hv_kvp_daemon[3416730]: sh: 1: /usr/libexec/hypervkvpd/hv_get_dns_info: not found" }

to this:

[22]:

expected = {
    "message": "Oct 17 11:54:21 dev-machine hv_kvp_daemon[3416730]: sh: 1: /usr/libexec/hypervkvpd/hv_get_dns_info: not found",
    "@timestamp": "Oct 17 11:54:21",
    "hostname": "dev-machine",
    "process": {
        "name": "hv_kvp_daemon",
        "pid": 3416730
    },
    "sh": "/usr/libexec/hypervkvpd/hv_get_dns_info: not found"
}

Create rule and processor

create the rule:

[23]:

import sys
sys.path.append("../../../../../")
import tempfile
from pathlib import Path

rule_yaml = """---
filter: "message"
dissector:
  mapping:
    message: "%{@timestamp} %{+@timestamp} %{+@timestamp} %{hostname} %{process.name}[%{process.pid}]: %{?shell}: %{}: %{&shell}"
  convert_datatype:
    process.pid: int
"""

rule_path = Path(tempfile.gettempdir()) / "concatenator"
rule_path.mkdir(exist_ok=True)
rule_file = rule_path / "data-stream.yml"
rule_file.write_text(rule_yaml)

[23]:

create the processor config:

[24]:

processor_config = {
    "thealmightydissector":{
        "type": "dissector",
        "rules": [str(rule_path), "/dev"],
        }
    }

create the processor with the factory:

[25]:

from unittest import mock
from logprep.factory import Factory

mock_logger = mock.MagicMock()
dissector = Factory.create(processor_config)
dissector

[25]:

dissector

Process event

[26]:

from copy import deepcopy
mydocument = deepcopy(document)


print(f"before: {mydocument}")
dissector.process(mydocument)
print(f"after: {mydocument}")
print(mydocument == expected)

before: {'message': 'Oct 17 11:54:21 dev-machine hv_kvp_daemon[3416730]: sh: 1: /usr/libexec/hypervkvpd/hv_get_dns_info: not found'}
after: {'message': 'Oct 17 11:54:21 dev-machine hv_kvp_daemon[3416730]: sh: 1: /usr/libexec/hypervkvpd/hv_get_dns_info: not found', '@timestamp': 'Oct 17 11:54:21', 'hostname': 'dev-machine', 'process': {'name': 'hv_kvp_daemon', 'pid': 3416730}, 'sh': '/usr/libexec/hypervkvpd/hv_get_dns_info: not found'}
True