`ingest_log` silently drops events for AIDE (and likely other file-integrity log types)

[Mogriffs01]

ingest_log silently drops events for AIDE (and likely other file-integrity log types) — sourceFilename field missing from LogsInlineSource payload

Summary

ChronicleClient.ingest_log(...) constructs a LogsInlineSource payload for POST .../logTypes/{log_type}/logs:import without the sourceFilename field. For at least the AIDE log type, Chronicle's ingestion pipeline silently drops submissions that omit it — the REST call returns HTTP 200 with an empty {} body (the standard queued-acknowledgement response shape), but the event never reaches the parser and never surfaces in search_udm. No error is ever surfaced to the caller.

The REST reference at projects.locations.instances.logTypes.logs/import#logsinlinesource shows sourceFilename as a field on LogsInlineSource (sibling of logs[] and forwarder). The SDK does not expose it — source_filename / sourceFilename appears zero times in src/secops/chronicle/log_ingest.py, and ChronicleClient.ingest_log's signature has no corresponding parameter.

Environment

• secops 0.42.0 (also verified against 0.43.0 — no change to log_ingest.py between versions)
• Tenant: Chronicle instance in region europe
• Python 3.12
• Authenticated as a service account with roles/chronicle.admin

Reproduction

from secops import SecOpsClient

client = SecOpsClient().chronicle(
    customer_id="...", project_id="...", region="europe",
)

# AIDE sample — valid syslog RFC5424 line, well-formed, UTF-8 clean.
# Parses to exactly 1 UDM event via run_parser against the committed parser.
log = '<133>1 2026-04-19T06:04:02.070299+00:00 host aide 1234 - - file=/etc/shadow;Mtime_new=...'

resp = client.ingest_log(
    log_type="AIDE",
    log_message=log,
    labels={"source": "smoketest", "smoketesteventid": "abcd1234"},
)
# resp == {}          — reported success
# Subsequent search_udm for smoketesteventid="abcd1234" → zero hits, indefinitely.

The submitted event never surfaces. No exception raised, no indication anything went wrong. Reproduced across > 5 probes over several hours, including with uniquely-generated synthetic content (rules out deduplication).

Production forwarder-pushed AIDE events continue to ingest and parse normally (> 10,000/day on the same tenant), so it is not a parser or tenant-level issue — it's specific to the logs:import inline path.

Root cause

The body emitted by ingest_log (per src/secops/chronicle/log_ingest.py lines 878–886 in the current main) looks like:

payload = {
    "inline_source": {
        "logs": [
            {
                "data": "<base64>",
                "log_entry_time": "...",
                "collection_time": "...",
                "labels": {"source": {"value": "smoketest"}, ...}
            }
        ],
        "forwarder": "projects/.../forwarders/<uuid>",
    }
}

There's no sourceFilename set at the inline_source level. Chronicle accepts the call (HTTP 200) but doesn't process the event.

Fix (what we had to do)

We bypassed ingest_log and built the request directly via chronicle_request, setting sourceFilename at the correct level — sibling of logs[] and forwarder:

from secops.chronicle.utils.request_utils import chronicle_request
import base64

payload = {
    "inline_source": {
        "logs": [
            {
                "data": base64.b64encode(log.encode("utf-8")).decode("ascii"),
                "log_entry_time": ts, "collection_time": ts,
                "labels": {"source": {"value": "smoketest"}, ...},
            }
        ],
        "forwarder": "projects/.../forwarders/<uuid>",
        "sourceFilename": "smoketest/AIDE/abcd1234.log",
    }
}
chronicle_request(client, "POST", "logTypes/AIDE/logs:import", json=payload)

With this change the AIDE event lands immediately and appears in search_udm within the normal sub-minute window.

Two important placement notes from trial-and-error:

• sourceFilename belongs at inline_source level (sibling of logs + forwarder), per the REST reference.
• Placing it inside an individual log entry (inline_source.logs[0].sourceFilename) returns HTTP 400: Unknown name "sourceFilename" at 'inline_source.logs[0]': Cannot find field.

We separately verified that adding sourceFilename to a log type that was previously ingesting fine (GOANYWHERE_MFT in our case) does not break anything — it's safe to set unconditionally. (extensively tested /s - Matt)

Suggested fix for the SDK

(AI generated, apply salt liberally - Matt)

Add a source_filename: str | None = None parameter to ChronicleClient.ingest_log() and plumb it through to the payload:

def ingest_log(
    self, log_type, log_message, ..., source_filename: str | None = None,
):
    ...
    inline_source = {
        "logs": logs,
        "forwarder": forwarder_resource,
    }
    if source_filename is not None:
        inline_source["sourceFilename"] = source_filename
    payload = {"inline_source": inline_source}

Optionally, defaulting it to a generated value (e.g. sdk-<log_type>.log) rather than omitting it would make the SDK robust against the undocumented server-side requirement — AIDE and possibly other file-integrity / file-aware log types appear to require it, and users hitting the silent-drop behaviour have no feedback mechanism pointing at the field.

[Mogriffs01]

Big time vibe coder here. I'm fully prepared to be told I just never found the right function to do this. Some additional context. This was actually working fine until today with the missing field, so it might be a backend change to tighten against the spec that previously wasn't enforced.