AdobeStock_531270710-2

The Quixotic Expedition into the Vastness of Edge Logs, Part 1: Analyzing Numerous Cribl Edge Nodes with Cribl Search

Last edited: May 31, 2023

Cribl Search is a powerful tool that is designed to enhance your data search efficiency, irrespective of the location of your data. This blog will explore how this tool seamlessly integrates with numerous Cribl Edge Nodes in real time, simplifying the process of discovery and troubleshooting. An integral part of Cribl Search is the “teleport” feature, which enables users to access specific Edge Nodes for in-depth analysis, simply by clicking on a host field. All of this can be achieved without any need for complex ETL processes, unnecessary data movement, or time-consuming pre-processing.

Let’s dive in.

Use Case 1: Analyzing SSH Login Attempts in System Logs Across Multiple Edge Nodes

Scenario: You suspect unauthorized SSH login attempts on your Edge Nodes and want to examine the auth.log files for any suspicious activity across your entire network.

Code example
dataset="cribl_edge_system_logs" source=*auth.log sshd user | limit 1000

This query searches the cribl_edge_system_logs dataset for events containing sshd and user terms in auth.log files. It returns up to 1,000 results, enabling you to analyze login attempts and detect any unauthorized access. Once you identify an issue, you can teleport to the specific Edge Node to take appropriate action.

Use Case 2: Tracking Memory Usage Across All Edge Nodes

Scenario: You want to monitor memory usage across your vast network of Edge Nodes to optimize performance and identify potential issues.

Code example
dataset="cribl_edge_metrics" node_memory_Used_percent | summarize avg(node_memory_Used_percent) by host

In this query, we’re scanning the cribl_edge_metrics dataset specifically for data related to used memory percentage (node_memory_Used_percent). The search calculates the average percentage of memory used by each host and groups the results by host.

By monitoring average memory usage, you can identify hosts that consistently utilize a high percentage of memory – gaining invaluable insights for capacity planning, load balancing, and troubleshooting performance issues.

Use Case 3: Identifying High Network Traffic Events Across Thousands of Edge Nodes

Scenario: You want to identify high network-traffic events across your extensive network of Edge Nodes to detect potential issues and ensure optimal performance.

Code example
dataset="cribl_edge_metrics" node_network_receive_bytes_all_total>1000000 and node_network_transmit_bytes_all_total>1000000 | timestats sum(node_network_receive_packets_all_total), sum(node_network_transmit_bytes_all_total) by host
word-image-44595-3

In this query, we are searching the cribl_edge_metrics dataset for network traffic data. The search calculates the total bytes received and sent by each host (node_network_receive_bytes_all_total and node_network_transmit_bytes_all_total), and then filters the results to display only those hosts with more than 1,000,000 bytes (1 MB) of traffic.

Identifying high network-traffic events enables you to proactively address potential performance issues. With the teleport feature, you can instantly access the specific Edge Node of your choice, to investigate further and resolve any issues.

Stay tuned for the second part of this post, where we delve deeper into the investigation of unauthorized login attempts, and explore building a personal threat-intelligence solution using lookup tables. If you’d like to try out Cribl Search, you can create a Cribl.Cloud account to get started for free.

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.

More from the blog

get started

Choose how to get started

See

Cribl

See demos by use case, by yourself or with one of our team.

Try

Cribl

Get hands-on with a Sandbox or guided Cloud Trial.

Free

Cribl

Process up to 1TB/day, no license required.