Listing all knowledge objects enabled in Splunk ES

[vpantangi]

Is there a search query to give the list of all the knowledge objects that are enabled in ES , i want to have list of all the correlation searches, macros , lookups and all searches.

[livehybrid]

Hi @vpantangi 

Below is a trimmed down export query that you could use to pull in Savedsearches (Including Correlation searches - which are ultimately saved searches),lookup files, lookup definitions, automatic lookups and macros.

This is part of a much bigger search which you can find at https://github.com/paychex/Splunk.Conf19/blob/master/CYA_Export_For_Core_Splunk_Query which was part of a Conf talk in 2019 given by Paychex which you can see the slides for here (https://conf.splunk.com/files/2019/slides/FN1315.pdf) and video of here (https://conf.splunk.com/files/2019/recordings/FN1315.mp4)

The the search below might generate quite a bit of data which might be hard to read, but you can strip out particular rest calls and run them individually.

| union maxtime=300 timeout=300
[| rest splunk_server="local" "/servicesNS/-/-/saved/searches" | search NOT search="| noop" | eval Type="Saved Searches/Alerts/Reports" | fields - type]
[| rest splunk_server="local" "/servicesNS/-/-/data/lookup-table-files" | eval Type="lookup table files" | fields - type]
[| rest splunk_server="local" "/servicesNS/-/-/data/transforms/lookups" | eval Type="Lookup Definitions" | fields - type fields_array]
[| rest splunk_server="local" "/servicesNS/-/-/data/props/lookups" | eval Type="Automatic lookups" | fields - type value | rex field=title " : [^\-]+-(?<title>[^\e]+)"]
[| rest splunk_server="local" "/servicesNS/-/-/admin/macros" | eval Type="Search Macros" | fields - type]
| eval updated=round(strptime(updated, "%Y-%m-%dT%H:%M:%S"),0)
| fieldformat updated= strftime(updated, "%x %X")
| eval _time=now()

| foreach title "eai:data" "eai:acl.sharing" "eai:acl.perms.read" "eai:acl.perms.write" search definition stanza value transform tag* filename fields_list collection external_type description *cron* is_scheduled schedule_window action* alert* args errormsg validation earliest_time latest_time header_label order display_location fields eventtypes REGEX link.* search.* display.*
    [ eval "<<FIELD>>"=if(mvcount('<<FIELD>>')>1, mvjoin('<<FIELD>>', ","), '<<FIELD>>')]
| table _time Type title eai:acl.appauthor eai:acl.perms.read eai:acl.perms.write eai:* *

Also - I wondered if this is what you were really looking for? Or if you just need a list of those enabled then you might find adding the reduces the noise!

| table _time Type title disabled 
| where disabled!=1

Remember that with ES not all the knowledge objects will be in the "SplunkEnterpriseSecuritySuite" app - infact barely any will be, this is because ES uses a collection of apps (often, but not always starting with SA-) so be careful not to exclude these if you end up scoping particular apps in part of the above, or other, queries!

Let us know if you need more info or have a slightly different intended view of the results and I can work through it.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[vpantangi]

Thank you for taking your time to reply so mostly i am looking for list of all knowledge objects we use in ES as we are ending our license on ES so looking to transfer all the knowledge objects.

[livehybrid]

Hi @vpantangi 

Thanks for the reply - hopefully the combination of searches in the thread have helped. 

One thing that just sprung to mind which I meant to mention earlier was if you dont have access to the conf files (e.g. on Splunk Cloud) then its worth looking at the "Admins Little Helper" app on Splunkbase. It allows you to run btool commands from the Splunk search box, so you can run 

btool savedsearches
btool macros

for an easily digestible and familiar output!

Dont forget to take an export of your KV stores, Threat Intel feed configuration and Assets and Identities mappings!

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[vpantangi]

Thank you very much i will look into it.

[kiran_panchavat]

@vpantangi 

If you want a single query to combine all these knowledge objects, you can use append or union, but the fields will differ slightly, so you’ll need to normalize them. Here’s an example:

| rest splunk_server=local /services/saved/searches 
| where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") AND disabled=0 
| eval type="Correlation Search" 
| rename title as name, eai:acl.app as app 
| table type, name, app, description, search 
| append 
[ | rest splunk_server=local /servicesNS/-/-/configs/conf-macros 
| eval type="Macro" 
| rename title as name, eai:acl.app as app 
| table type, name, app, definition 
| where disabled=0 ] 
| append 
[ | rest splunk_server=local /servicesNS/-/-/configs/conf-transforms 
| where match('filename', ".+") AND disabled=0 
| eval type="Lookup" 
| rename title as name, eai:acl.app as app 
| table type, name, app, filename ] 
| append 
[ | rest splunk_server=local /servicesNS/-/-/saved/searches 
| where disabled=0 AND NOT match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") 
| eval type="Saved Search" 
| rename title as name, eai:acl.app as app 
| table type, name, app, description, search ]

[vpantangi]

Thank you Kiran i will try all the queries you sent and thank you for taking time to create all of the searches.

[kiran_panchavat]

@vpantangi 

To list all enabled saved searches (including those not tagged as correlation searches):

| rest splunk_server=local /servicesNS/-/-/saved/searches 
| rename eai:acl.app as app, title as search_name 
| table search_name, app, search, disabled, description 
| where disabled=0

[kiran_panchavat]

@vpantangi 

Additionally, to list lookup files present in the filesystem:

| rest splunk_server=local /servicesNS/-/-/data/lookup-table-files 
| rename title as lookup_file, eai:acl.app as app 
| table lookup_file, app

[kiran_panchavat]

@vpantangi 

To list all lookup definitions (files and configurations) in Splunk ES:

| rest splunk_server=local /servicesNS/-/-/configs/conf-transforms 
| where match('filename', ".+") 
| rename title as lookup_name, eai:acl.app as app 
| table lookup_name, filename, app, disabled
| where disabled=0

[kiran_panchavat]

@vpantangi 

To list all macros defined in your Splunk environment (including those used by Splunk ES):

| rest splunk_server=local /servicesNS/-/-/configs/conf-macros 
| rename title as macro_name, eai:acl.app as app 
| table macro_name, definition, app, disabled
| where disabled=0

[kiran_panchavat]

@vpantangi 

Correlation Searches (Enabled)

| rest splunk_server=local count=0 /services/saved/searches
| where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]")
| rename eai:acl.app as app, title as correlation_search_name, action.correlationsearch.label as label, action.notable.param.security_domain as security_domain
| table correlation_search_name, label, app, security_domain, description, search, disabled
| where disabled=0

[kiran_panchavat]

@vpantangi 

Yes. Use a REST search to expose the information in a table with the fields you're interested in. Example:

| rest /services/alerts/correlationsearches | rename eai:acl.app as app, title as csearch_name | table app security_domain csearch_name description

| rest splunk_server=local count=0 /services/saved/searches
| where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]")
| rename action.correlationsearch.label as Alert_Name,action.correlationsearch.annotations as frameworks,updated as Last_Updated
| table Alert_Name, frameworks, disabled, Last_Updated
| spath input=frameworks

This can be handy for dumping a list of installed ES correlation searches with disabled status, description, frameworks etc.  Be sure your use has the permissions to all knowledge objects if you don't see any you are know are present in an app context.

| rest splunk_server=local count=0 /servicesNS/-/-/saved/searches 
| where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") 
| rename title as search_name, eai:acl.app as app, action.correlationsearch.annotations as frameworks
| table search_name, app, description, frameworks, disabled
| spath input=frameworks
| rename mitre_attack{} as mitre_attack, nist{} as nist, cis20{} as cis20, kill_chain_phases{} as kill_chain_phases
| table app, search_name, description, disabled, cis, kill_chain_phases, nist, mitre_attack