![foreach is clean
| timechart span=1h limit=0
sum(eval(b/pow(1024,3))) as size by st
| timechart span=1h limit=0 sum(b) by st
| foreach * [eval
"<<FIELD>>"=if("<<FIELD>>" == "_time",
_time, '<<FIELD>>' / pow( 1024 , 3 ))]
Weak Strong
Pretty & Faster Searching](https://image.slidesharecdn.com/splunkbangaloreusergroup202008-01-200801151437/85/Splunk-bangalore-user-group-2020-08-01-22-320.jpg)
![Avoid Subsearches
index=burch | eval blah=yay
| append
[ search index=simon | eval blah=duh ]
( index=burch … ) OR ( index=simon …)
| eval blah=case( index==”burch" , "yay" ,
index==”simon" ,"duh" )
Weak Strong
Faster Searching](https://image.slidesharecdn.com/splunkbangaloreusergroup202008-01-200801151437/85/Splunk-bangalore-user-group-2020-08-01-24-320.jpg)
Splunk bangalore user group 2020 08 01
- 1. Bengaluru User Group WELCOME 1st Aug 2020 स्वागत স্বাগত ಸ್ವಾಗತ स्वागत आहे స్వాగతவரவவற்பு സ്വാഗതം ਸਵਾਗਤ ਹੈ સ્વાગત છે آمدید خوش ସ୍ୱାଗତ آیا ڪري ڀلي
- 3. Housekeeping Join #splunk_bengaluru_usergroup on Slack http://splk.it/slack Use #splunk_bengaluru_usergroup for Q&A during the session Please keep your lines muted when not speaking Slides, recording & feedback form will be posted to the Events page Splunk Bengaluru User Group https://usergroups.splunk.com/bengaluru-splunk-user-group/
- 4. Search Tips Optimization & Best Practices Software Engineer (Cisco) Splunk Trust MVP (2019,2020) Kamlesh Vaghela
- 5. Search Tips Optimization & Best Practices Kamlesh Vaghela Software Engineer (Cisco)
- 6. © 2020 SPLUNK INC. Components of SPL: 1. Search Terms 2. Commands 3. Functions 4. Clauses Search Processing Language ANY question of ANY machine data SPL
- 7. © 2020 SPLUNK INC. Search Processing Language ANY question of ANY machine data SPL
- 8. © 2020 SPLUNK INC. Search Processing Language ANY question of ANY machine data SPL
- 9. © 2020 SPLUNK INC. What’s in a Bucket? SPL • Journal.gz • Events go here • Made up of from many smaller compressed slices • Raw data collected and saved into slices – 1 slice = ~128KB of uncompressed data • TSIDX files • associates each unique keyword in your data with location references to events • Bloom Filters • bloom filters narrow the set of tsidx files
- 11. © 2020 SPLUNK INC. • Retrieve only the required data • Move as little data as possible • Parallelize as much work as possible • Set appropriate time windows • Map Reduce Basic Principles Search Optimization
- 12. © 2020 SPLUNK INC. • Filter as much as possible in the initial search • Perform joins and lookups on only the required data • Perform evaluations on the minimum number of events possible • Move commands that bring data to the search head as late as possible in your search criteria • Summary Indexing • Data Model Acceleration Techniques Search Optimization
- 13. What I should Avoid? Avoid Why? Should do All time • Events are stored in time-series order • Reduce searched bucket by being specific • Use Specific time range • Narrow the time range as much as possible index=* • Events are grouped into indexes • Reduce searches buckets by specifying an index. • Specify an index in the search wildcards • Wildcards are not compatible with Bloom Filters • Wildcard matching to term in the index takes time • Varying levels of pain 1. myterm*: Not great 2. *myterm: Bad 3. *myterm*: Death • Use the OR Operator 1. myterm1 OR myterm2…
- 14. What I should Avoid? Avoid Why? Should do NOT != • Bloom Filters & indexes are designed to quickly locate terms that exist • Searching for that term which not exists takes longer • Use AND/OR operators Verbose Search Mode • Verbose search mode causes full event data to be sent to the search head, even if isn’t needed. • Use Smart Mode or Fast Mode Real-time Searches • RT searches put an increased load on search head and indexer • Use a scheduled search that occurs more frequently Joins / Sub-searches • This is intensive search command • Use the stats (preferred) Search after first pipe • Filtering search results using a second | search command in your query is inefficient • As much as possible, add all filtering criteria before the first |
- 16. table vs fields index=main | head 1000 | table A B C D E index=main |fields A B C D E | head 1000 |table A B C D E Weak Strong Faster Searches
- 17. Keep it kosher - Cosmetics at end index=_internal | rename host as "Client Machine" | stats count by "Client Machine" index=_internal | stats count by host | rename host as "Client Machine" Weak Strong Pretty Searches
- 18. Require Fields index=_internal WARN | stats count index=_internal log_level="WARN" | stats count Weak Strong Faster Searching
- 19. Be specific index=_internal "_ACCELERATE_DM_test_Test.executed_ background_job_ACCELERATE_" | timechart count index=_internal sourcetype=scheduler savedsearch_name="_ACCELERATE_DM _test_Test.executed_background_job_ACC ELERATE_" | timechart count Weak Strong Faster Searching
- 20. stats vs dedup/transaction phone=* | dedup phone| table phone| sort phone phone=*| transaction host | table host, phone phone=* | stats count by phone, host | fields - count Weak Strong Faster Searching
- 21. multi-eval | eval this=”is” | eval a=“verbose” | eval example=“of eval” | eval this=”is”, a=“verbose”, example=“of eval” Weak Strong Pretty Searching
- 22. foreach is clean | timechart span=1h limit=0 sum(eval(b/pow(1024,3))) as size by st | timechart span=1h limit=0 sum(b) by st | foreach * [eval "<<FIELD>>"=if("<<FIELD>>" == "_time", _time, '<<FIELD>>' / pow( 1024 , 3 ))] Weak Strong Pretty & Faster Searching
- 23. coalesce’s cooler than if | eval size = if(isnull(bytes) , if( isnull(b) , "N/A" , b ) ,bytes ) |eval size = coalesce( bytes , b , "N/A" ) Weak Strong Pretty Searches
- 24. Avoid Subsearches index=burch | eval blah=yay | append [ search index=simon | eval blah=duh ] ( index=burch … ) OR ( index=simon …) | eval blah=case( index==”burch" , "yay" , index==”simon" ,"duh" ) Weak Strong Faster Searching
- 25. NOT NOTs OR != index=_internal NOT log_level=INFO | stats count by log_level index=_internal log_level!=INFO | stats count by log_level index=_internal log_level IN ("ERROR","WARN","WARNING") | stats count by log_level Weak Strong Faster Searching
- 26. transaction ...| transaction host …| transaction maxspan=10m maxevents=100 … Weak Strong Search Commands
- 27. metadata index=* | stats count by host | metadata index=* type=hosts Weak Strong Search Commands
- 28. eventcount index=* | stats count by host | eventcount summarize=false index=* Weak Strong Search Commands
- 29. Event Types & Tags index=oidemo host=dmzlog.splunktel.com sourcetype=access_combined source=/opt/apache/log/access_com bined.log iphone user_agent="*iphone*” | stats count by action tag=iphone_event or eventtype=web_logs Weak Strong Search Tangent
- 30. Search Tricks
- 31. Top 5 Trending Values
- 32. Bottom 5 Trending Values
- 34. extract
- 35. mvzip, split & mvindex
- 36. addcoltotals in table header
- 37. CASE & TERM
- 38. Job Inspector & Search Logs
- 42. Thank You
- 43. © 2020 SPLUNK INC. Q&A Raise hand to be unmuted Post questions in WebEx Chat Join Slack for Q&A http://splk.it/slack
- 44. © 2020 SPLUNK INC. Challenges on Slack #splunk_bengaluru_usergroup • Challenges from July Bengaluru User Group sessions. • Challenges from current session (to be posted over the weekend).
- 45. © 2020 SPLUNK INC. Community Resources Splunk Community Resources (Both Official and Unofficial) Splunk > Clara-fication: Splunk Community: https://www.splunk.com/en_us/blog/tips- and-tricks/splunk-clara-fication-splunk-community.html
- 46. We plan to meet 1st Saturday of every month at 11:00 AM IST. Please provide feedback for : • Sessions and improvements. • Topics to be covered in future sessions. • Let us know if you are interested in presenting in User Group. Keep the comradery through Slack and Splunk Answers> What’s Next http://splk.it/slack http://community.splunk.com https://conf.splunk.com Splunk .Conf 2020 registrations are open: Oct 20th and 21st (Virtual)