Hello, We've setup our Splunk Search Head to download snapshots from ThreatStream API directly, while troubleshooting, we observed that it was downloading the snapshots from hxxps://ts-optic.s3.amazonaws.com/snapshots/... but then had issues processing it.         2022-11-03 02:01:47,394 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong. 2022-11-03 02:01:47,443 18860 ERROR threatstream_app - threatstream_kvstore> Failed at add_kvs_batch - sz == 1, collection_name: ts_md5, data: [{'date_last': '2016-02-21T14:52:32.000Z', 'id': '0', '_key': '99929352'}] 2022-11-03 02:01:47,443 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong. 2022-11-03 02:01:47,464 18860 ERROR threatstream_app - threatstream_kvstore> Failed at add_kvs_batch - sz == 1, collection_name: ts_md5, data: [{'date_last': '2016-02-21T14:52:37.000Z', 'id': '0', '_key': '99929603'}] 2022-11-03 02:01:47,464 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong. 2022-11-03 02:01:48,677 18860 INFO threatstream_app - ioc_loader> 193571 items with id="0" saved to kvs: ts_md5 for deletion, time: 35505.908512592316 2022-11-03 02:01:48,678 18860 INFO threatstream_app - ioc_loader> 193571 items with id="0" saved to kvs: ts_md5 for deletion, time: 35505.908512592316 2022-11-03 02:01:49,059 18860 ERROR threatstream_app - ts_ioc_ingest> failed to download optic intelligence: Autologin succeeded, but there was an auth error on next request. Something is very wrong. 2022-11-03 02:01:49,059 18860 ERROR threatstream_app - ts_ioc_ingest> failed to download optic intelligence: Autologin succeeded, but there was an auth error on next request. Something is very wrong. 2022-11-03 02:01:49,933 18860 ERROR threatstream_app - ts_ioc_ingest> Traceback (most recent call last): File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 290, in wrapper return request_fun(self, *args, **kwargs) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 71, in new_f val = f(*args, **kwargs) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 622, in delete response = self.http.delete(path, self._auth_headers, **query) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1169, in delete return self.request(url, message) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1255, in request raise HTTPError(response) splunklib.binding.HTTPError: HTTP 401 Unauthorized -- call not properly authenticated During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 232, in _handle_auth_error yield File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 301, in wrapper return request_fun(self, *args, **kwargs) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 71, in new_f val = f(*args, **kwargs) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 622, in delete response = self.http.delete(path, self._auth_headers, **query) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1169, in delete return self.request(url, message) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1255, in request raise HTTPError(response) splunklib.binding.HTTPError: HTTP 401 Unauthorized -- call not properly authenticated During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/splunk/etc/apps/threatstream/bin/ts_ioc_ingest.py", line 284, in download_iocs TmDataManager(splunka=remote_splunk, logger=logger).process_data() File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 176, in process_data self._process_data() File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 245, in _process_data self.load_from_lookup_files() File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 508, in load_from_lookup_files iocs.load_iocs() File "/opt/splunk/etc/apps/threatstream/bin/ts/lookup_iocs.py", line 404, in load_iocs util.utils.remove_0_id_values(self.kvsm, kvs) File "/opt/splunk/etc/apps/threatstream/bin/util/utils.py", line 143, in remove_0_id_values remove_delete_id_values(kvsm, ioc_kvs_name, 'id', '0') File "/opt/splunk/etc/apps/threatstream/bin/util/utils.py", line 146, in remove_delete_id_values kvsm.delete_kvs(kvs, {id_name : delete_id_value}) File "/opt/splunk/etc/apps/threatstream/bin/util/kvs_manager.py", line 286, in delete_kvs collection.data.delete(query=json.dumps(query_dict)) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/client.py", line 3678, in delete return self._delete('', **({'query': query}) if query else {}) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/client.py", line 3631, in _delete return self.service.delete(self.path + url, owner=self.owner, app=self.app, sharing=self.sharing, **kwargs) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 301, in wrapper return request_fun(self, *args, **kwargs) File "/opt/splunk/lib/python3.7/contextlib.py", line 130, in __exit__ self.gen.throw(type, value, traceback) File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 235, in _handle_auth_error raise AuthenticationError(msg, he) splunklib.binding.AuthenticationError: Autologin succeeded, but there was an auth error on next request. Something is very wrong.         So I guess "Something is wrong"? but what? Anyone knows a solution or at least the cause of this?   ... View more

Hello, I'm getting the following error when trying to Trigger an alert to ServiceNow   08-18-2021 18:42:04.461 -0400 INFO sendmodalert - Invoking modular alert action=sn_sec_multi_incident_alert for search="test" sid="scheduler__nobody_RGlmZW5kYS1UaHJlYXRIdW50aW5n__test_at_1629326520_324_E6AF4A46-E741-44E5-8200-C53A9BA036B3" in app="Alert App" owner="nobody" type="saved" 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - ERROR: Unexpected error: Traceback (most recent call last): 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - File "/opt/splunk/etc/apps/TA-ServiceNow-SecOps/bin/sn_sec_multi_incident_alert.py", line 47, in <module> 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - for result in csv.DictReader(csvResult): 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - File "/opt/splunk/lib/python3.7/csv.py", line 111, in __next__ 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - self.fieldnames 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - File "/opt/splunk/lib/python3.7/csv.py", line 98, in fieldnames 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - self._fieldnames = next(self.reader) 08-18-2021 18:42:04.947 -0400 ERROR sendmodalert - action=sn_sec_multi_incident_alert STDERR - _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)   Not sure what the problem is here, we upgraded from Splunk 7.3.3 to Splunk 8.1.5 and upgraded the ServiceNow Security Operations Add-on from 1.23.3 to 1.23.4 ... View more

The new SentinelOne App 5.1.1, doesn't show a rank field for the threats anymore under sourcetype="sentinelone:channel:threats"   How can we determine the rank of a threat now? ... View more

Hello, We're trying to get a UF on a Domain Controller to monitor two different OUs in the AD as follows:   [admon://AdminAccounts] targetDc = dc01.mydomain.com startingNode = OU="Administrative Accounts", DC=mydomain, DC=com index = admon [admon://ElevatedPrivs] targetDc = dc01.mydomain.com startingNode = "OU=Elevated Privileges", DC=mydomain, DC=com index = admon     The UF is running under a Domain Service Account with full read access to the tree. We're getting the following errors:   ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0x20' ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent, ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed: (0x80004005)Unspecified error -- attempting to reload server path ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0x20' ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent, ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed again with DCName='dc01.mydomain.com': (0x80004005)Unspecified error -- no more retries ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - ADMonitor::init: Failed to initialize Active Directory usn context. ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - ADMonitorThread::launchADMonitor: Failed to initialize ADMonitor='admon://ElevatedPrivs', targedDC='dc01.mydomain.com' ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0x20' ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent, ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed: (0x80004005)Unspecified error -- attempting to reload server path ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0x20' ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent, ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed again with DCName='dc01.mydomain.com': (0x80004005)Unspecified error -- no more retries ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - ADMonitor::init: Failed to initialize Active Directory usn context. ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe"" splunk-admon - ADMonitorThread::launchADMonitor: Failed to initialize ADMonitor='admon://ElevatedPrivs', targedDC='dc01.mydomain.com'   We can't figure out what does (0x80004005)Unspecified error, or err='0x20' actually means. Are we missing something here? Is there a problem with having a space (" ") character in the OUs? Please advice ... View more

Our CrowdStrike Add-on stopped pulling logs via the API giving this error       2021-05-01 19:03:31,879 ERROR pid=31672 tid=MainThread file=base_modinput.py:log_error:309 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-crowdstrike-falcon-event-streams/bin/ta_crowdstrike_falcon_event_streams/aob_py2/modinput_wrapper/base_modinput.py", line 128, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-crowdstrike-falcon-event-streams/bin/crowdstrike_event_streams.py", line 71, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-crowdstrike-falcon-event-streams/bin/input_module_crowdstrike_event_streams.py", line 358, in collect_events crowdstrike_client() File "/opt/splunk/etc/apps/TA-crowdstrike-falcon-event-streams/bin/input_module_crowdstrike_event_streams.py", line 234, in crowdstrike_client num_feeds = len(response['resources']) TypeError: object of type 'NoneType' has no len()       I can't understand what happened or how to prevent it for happening again. Anyone out there with same issue?   ... View more

We are seeing logs like this that might contain a base64 encoded encrypted password and we would like to know if this is a risk of leaving this in Splunk since we don't know how strong this encryption is, and if possible to remove it from the source or should we make a filter in a transforms to discard this information Dec 9 15:54:07 10.X.Y.Z date=2020-12-09 time=15:54:08 devname="FORTIGATE" devid="FG100" logid="0100044547" type="event" subtype="system" level="information" vd="root" eventtime=1607547248698893780 tz="-0500" logdesc="Object attribute configured" user="admin" ui="GUI(10.X.Y.Z)" action="Add" cfgtid=10289326 cfgpath="user.local" cfgobj="USER" cfgattr="type[password]passwd[ENC JcB/fAvi7lxawB9OqPN2t8WE4MnLXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXvmv1EMRzPZPdMAhWppDbaqNyr1tGx5eDmxg==]" msg="Add user.local USER"   ... View more

Hello, We're running Splunk Enterprise 7.3.7.1 on Linux. We have a Deployment Server to manage our forwarders. We are using our own certificates to authenticate the server (DS) and clients (UFs, HFs) connecting to it, all working fine and WebGUI working fine on DS. But we keep seeing this error in the logs and we don't know what's causing it and what are the consequences. 10-27-2020 09:01:00.395 -0400 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py" ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:741)   Does anyone know? ... View more

I'm seeing this errors on our installation of the Sophos Add-on for Splunk 2020-03-20 13:44:07,936 ERROR pid=7703 tid=MainThread file=base_modinput.py:log_error:307 |***event data in json format*** It seems that some events are getting indexed but other's are erroring this way. Does anyone know what's going on and how to fix it? ... View more

My script works great On-Premise, but Splunk Cloud rejects it I use the requests module to do HTTPS requests like this one response = requests.post(url, auth=(user, pwd), headers=headers ,data=data) In Splunk Cloud it gets rejected because it thinks it might be insecure, when passing parameters to the requests module, I must do parameter= Possible insecure HTTP Connection. Match: requests.post Positional arguments, ["?"]; Keyword arguments, {"auth": "?", "headers": "?", "data": "?"} File: bin/alert_event.py Line Number: 46 I don't know how to fix this. Anyone knows how to pass this test? ... View more