Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- About luminousplumz
luminousplumz
Engager
Member since:
06-25-2019
a month ago
Community Statistics
| Posts | 2 |
| Solutions | 0 |
| Karma Given | 1 |
| Karma Received | 0 |
| Member Since | 06-25-2019 |
Activity Feed
- Karma Re: Rest api & splitting json events with fields and timestamps for scelikok. a month ago
- Posted Extracting a value from MQTT string before parsing to JSON on Getting Data In. a month ago
- Posted Rest api & splitting json events with fields and timestamps on Getting Data In. 03-26-2021 06:56 AM
Topics I've Started
a month ago
I have an requirement to extract a value from an mqtt string before i parse it to json. Initially i was using MQTT Modular input app to pull each of the topics with their own input. I found that with more than 3 inputs /topics enabled i am dropping some if not all data. So i decided to pull all the topics in a single input. This works well except i still need to be able to separate the topics for searches. I managed to get this working using multiple transforms. i changed something and now i can get it to work again. Using Transforms i can parse to json with no issues (mqtttojson) Transforms.conf [mqtttojson] REGEX = msg\=(.+)$ FORMAT = $1 DEST_KEY = _raw [mqtttopic] CLEAN_KEYS = 0 FORMAT = Topic::"$1" REGEX = tgw\/data\/0x155f\/(?<Topic>\S*?)\/ Props.conf [mqtttojson_ubnpfc_all] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true TIME_PREFIX = \"ts\": TZ = Europe/London category = Custom pulldown_type = 1 TRANSFORMS-mqtttopic = mqtttojson, mqtttopic In the example below i need the 4th topic level i.e. "TransportContextTracking". Thu Apr 24 12:42:15 GMT 2025 name=mqtt_msg_received event_id= topic=tgw/data/0x155f/TransportContextTracking/MFC/0278494 msg={"data":{"destination":{"locationAddress":"/UrbanUK/PCOTS13/Exit"},"errorCode":null,"event":"Started","loadCarrierId":"0278494","source":{"locationAddress":"/UrbanUK/PCOTS13/Pick"},"transportId":"f0409b2a-e9d4-407c-bd65-48ccea17b520","transportType":"Transport"},"dbid":8104562815,"ts":1745498528217} What am i missing ?????
... View more
Labels
- Labels:
-
field extraction
-
JSON
-
modular input
03-26-2021
06:56 AM
Hi, im really struggling to split out events from my json at the moment. currently i only get a single event with multi values in or multiple events but no field names here is an example response from a request with the props.conf setup. [ CMMS_ECP_TASKS_JSON1 ]
BREAK_ONLY_BEFORE=(\{|\[\s+{)
BREAK_ONLY_BEFORE_DATE=false
CHARSET=AUTO
LINE_BREAKER=(\}|\[\s+{)
MAX_DAYS_HENCE=180
MAX_TIMESTAMP_LOOKAHEAD=180
MUST_BREAK_AFTER=(\}|\}\s+\])
NO_BINARY_CHECK=true
SEDCMD-remove_footer=s/\]\s+\}//g
SEDCMD-remove_header=s/(\{(\s+.*){3}\[)//g
SEDCMD-remove_trailing_commas=s/\},/}/g
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S%%%
TIME_PREFIX=\"dateFrom\"\:
category=Custom
disabled=false
pulldown_type=true
description=test json {"id":16,"dateFrom":"2021-03-03T00:00:00+01:00","priority":3},{"id":29,"dateFrom":"2021-01-28T00:00:00+01:00","priority":3},{"id":183,"dateFrom":"2021-01-19T00:00:00+01:00","priority":2},{"id":184,"dateFrom":"2021-01-19T00:00:00+01:00","priority":2},{"id":197,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":216,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":217,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":218,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":219,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":220,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":221,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":222,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":223,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":224,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":225,"dateFrom":"2021-03-22T00:00:00+01:00","priority":2},{"id":226,"dateFrom":"2021-04-05T00:00:00+02:00","priority":2},{"id":227,"dateFrom":"2021-04-05T00:00:00+02:00","priority":2},{"id":228,"dateFrom":"2021-04-05T00:00:00+02:00","priority":2},{"id":229,"dateFrom":"2021-04-05T00:00:00+02:00","priority":2},{"id":230,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":231,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":232,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":233,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":234,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":235,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":236,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":237,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":238,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":239,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":240,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":241,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":242,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":243,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":244,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":245,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":246,"dateFrom":"2021-04-19T00:00:00+02:00","priority":2},{"id":247,"dateFrom":"2021-05-03T00:00:00+02:00","priority":2},{"id":248,"dateFrom":"2021-05-03T00:00:00+02:00","priority":2},{"id":249,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":250,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":251,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":252,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":253,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":254,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":255,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":256,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":257,"dateFrom":"2021-05-17T00:00:00+02:00","priority":2},{"id":258,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":259,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":260,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":261,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":262,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":263,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":264,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":265,"dateFrom":"2021-05-31T00:00:00+02:00","priority":2},{"id":266,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":267,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":268,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":269,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":270,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":271,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":272,"dateFrom":"2021-06-14T00:00:00+02:00","priority":2},{"id":310,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":311,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":312,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":313,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":314,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":315,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":316,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":317,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":318,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":319,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":320,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":321,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":322,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":323,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":324,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":325,"dateFrom":"2021-06-28T00:00:00+02:00","priority":2},{"id":326,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":327,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":328,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":329,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":330,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":331,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":332,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":333,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":334,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":335,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":336,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":337,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":338,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":339,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":340,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":341,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":342,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":343,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":344,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":345,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":346,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":347,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":348,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":349,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":350,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2},{"id":351,"dateFrom":"1753-01-01T00:00:00+01:00","priority":2}
... View more
Labels
- Labels:
-
JSON
