Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logstash parser issues to fix for v23.11.0 #289

Closed
mmguero opened this issue Nov 9, 2023 · 1 comment
Closed

logstash parser issues to fix for v23.11.0 #289

mmguero opened this issue Nov 9, 2023 · 1 comment
Assignees
Labels
logstash Relating to Malcolm's use of Logstash zeek Relating to Malcolm's use of Zeek
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Nov 9, 2023

There are some logstash filters that need that need adjusted before v23.11.0:

malcolm-logstash-1            | [2023-11-09T16:19:40,088][WARN ][logstash.outputs.opensearch] Could not index event to OpenSearch. {:status=>400, :action=>["index", {:_id=>"210301-Z52L6XCXCfUTVTC8xdzQLg", :_index=>"arkime_sessions3-210301", :routing=>nil}, {"host"=>{"name"=>"hedgehog"}, "protocol"=>["1", "ldap"], "source"=>{"port"=>1815, "ip"=>"192.168.0.2"}, "length"=>0, "related"=>{"ip"=>["192.168.0.2", "192.168.0.1"]}, "rootId"=>"CyeW0n2wv7efqWruy3", "zeek"=>{"ts"=>"2021-03-01T07:01:10.177999936Z", "uid"=>"CyeW0n2wv7efqWruy3", "ldap_search"=>{"base_object"=>["0"], "result_count"=>"protocol error"}}, "@version"=>"1", "log"=>{"file"=>{"path"=>"ldap_search(LDAP,pcap,1699546206561720747).log"}}, "tags"=>["LDAP"], "lastPacket"=>1614582070177, "event"=>{"hash"=>"Z52L6XCXCfUTVTC8xdzQLg", "action"=>["search"], "end"=>"1614582070177", "provider"=>"zeek", "start"=>"1614582070177", "ingested"=>2023-11-09T16:19:35.727Z, "kind"=>"event", "id"=>["CyeW0n2wv7efqWruy3"], "dataset"=>"ldap_search"}, "ecs"=>{"version"=>"8.0.0"}, "@timestamp"=>2021-03-01T07:01:10.177999936Z, "timestamp"=>1614582070177, "firstPacket"=>1614582070177, "node"=>"hedgehog", "input"=>{}, "destination"=>{"port"=>389, "ip"=>"192.168.0.1"}, "network"=>{"protocol"=>["ldap"], "transport"=>["1"], "application"=>"ldap", "type"=>"ipv4", "direction"=>"internal"}, "agent"=>{"name"=>"hedgehog"}}], :response=>{"index"=>{"_index"=>"arkime_sessions3-210301", "_id"=>"210301-Z52L6XCXCfUTVTC8xdzQLg", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [zeek.ldap_search.result_count] of type [integer] in document with id '210301-Z52L6XCXCfUTVTC8xdzQLg'. Preview of field's value: 'protocol error'", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"protocol error\""}}}}}

Need to see what the "no such object" thing is, and probably just ignore that field if it's set as such. Or it's possible there's a new field I'm not looking for.

@mmguero mmguero self-assigned this Nov 9, 2023
@mmguero mmguero added logstash Relating to Malcolm's use of Logstash zeek Relating to Malcolm's use of Zeek labels Nov 9, 2023
@mmguero mmguero added this to Malcolm Nov 9, 2023
@mmguero mmguero moved this to Todo (develop) in Malcolm Nov 9, 2023
@mmguero mmguero added this to the v23.11.0 milestone Nov 9, 2023
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Nov 10, 2023
* replace master/slave with client/server for modbus (idaholab#291)
* modbus updates for icsnpp-modbus (idaholab#289)
* point some Zeek plugins back upstream
* added new visualizations to modbus dashboard
@mmguero
Copy link
Collaborator Author

mmguero commented Nov 10, 2023

Modbus is fixed, need to look at ldap now.

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Nov 10, 2023
@mmguero mmguero moved this from Todo (develop) to Done in Malcolm Nov 10, 2023
@mmguero mmguero closed this as completed Nov 10, 2023
This was referenced Dec 4, 2023
@mmguero mmguero moved this from Done to Review in Malcolm Dec 5, 2023
@mmguero mmguero moved this from Review to Released in Malcolm Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logstash Relating to Malcolm's use of Logstash zeek Relating to Malcolm's use of Zeek
Projects
Status: Released
Development

No branches or pull requests

1 participant