Situational awareness + DataV: safe visual interaction, so play

Introduction: In the security field, the ability to "see" greatly affects the effectiveness of risk defense. Display unknown risks in a visual way, so that enterprise security teams have a "feeling of solidity" and "sense of presence". In terms of security visualization, Fu Kui, one of Alibaba Cloud's first MVPs, the head of Qianxun Location Security, came up with a "cross-border gameplay", using Alibaba Cloud's two major products: Situational Awareness and DataV, to achieve interactive security threat discovery.

In the security field, the ability to "see" greatly affects the effectiveness of risk defense. Display unknown risks in a visual way, so that enterprise security teams have a "feeling of solidity" and "sense of presence".

In terms of security visualization, Fu Kui, one of Alibaba Cloud's first MVPs, the head of Qianxun Location Security, came up with a "cross-border gameplay", using Alibaba Cloud's two major products: Situational Awareness and DataV, to achieve interactive security threat discovery.

Let's Make It True.

This article introduces how to use the DataV large screen to display situational awareness DNS session logs to realize interactive security threat discovery.

Spoiler for the final effect, definitely worth a try:

Node (source address, DNS server address, domain name to be resolved) relationship graph of all DNS logs. Click on any of the nodes, the associated points are automatically highlighted, and the non-associated nodes enter the masked state. If there are any suspicious nodes, relationships, or categories, you can see them at a glance!

Background introduction

Since the launch of the new version of the "Log" function in "Situational Awareness", team members have been thinking about how to fully exploit the value of log information, not just for fault diagnosis or incident investigation.

Among them, it is the most ideal way to realize interactive security threat discovery by graphically displaying the relationship between different network resource nodes. I have tried Graphviz, yEd, NetworkX and other methods before, but they were all abandoned due to complicated configuration and cumbersome use.

Until Li Wenyi in the MVP technology group recommended DataV to everyone, I thought: it's time to "make it true"!

product description

The following materials are quoted from the official introduction, and the description is not allowed and has nothing to do with me:

situational awareness
"Situational awareness provides a SAAS service, that is, in a large-scale cloud computing environment, it can comprehensively, quickly and accurately capture and analyze the elements that can cause changes in the network security situation, and provide a systematic security solution. Program."

DataV data visualization
"DataV aims to let more people see the charm of data visualization and help non-professional engineers easily build professional-level visualization applications through a graphical interface. DataV provides rich visualization templates to meet your conferences and exhibitions, business monitoring, and risk early warning. , geographic information analysis and other business display needs.”

demand analysis

The log function of situational awareness is very powerful, and currently supports query: all inbound layer 7 data, inbound and outbound stack layer 4 sessions, and DNS bidirectional logs. Among them, DNS logs help the security team to analyze whether the server has been invaded, has been implanted with Trojan horse virus, and there are problems such as abnormal requests.

What we are most concerned about is: which server, through which DNS server, which domain name is resolved?

In response to this requirement, if there is a visual node relationship graph to assist in the analysis, the efficiency of threat identification will be greatly improved. Before using DataV, the usual way is to use some chart tools to display, and the workload of custom development is indispensable. The emergence of DataV has greatly liberated the hands of data analysts, which can be used to drag the progress bar. With just a few mouse clicks and basic API configuration, you can instantly bring your data to life and talk. DataV comes with its own node relationship diagram and embedded native ECharts, which can fully meet such needs.

Implementation method example

You must be looking forward to what strong situational awareness and sexy DataV powerhouses look like. Expectation is not as good as action, let's make it happen together.

Export situational awareness DNS logs

Log in to the Alibaba Cloud console, go to the Security (Cloud Shield) | Situational Awareness page, and select the log new item from the submenu.

Set DNS log query conditions, which are:

Log Source: DNS
Field: qtype
Judgment condition: Inclusive (currently only inclusive)
Keyword: A
Then set the query time and click search.

The system will soon be able to return the query results, and export the current page (yes, it is the current page, 100 -_- at a time) logs to Excel through the export results in the upper right corner. If you need more data, you have to turn the pages to export -_-. Of course there are also programmatic solutions, see below.

Customize the data source API that returns JSON strings

The key to extracting data from the exported Excel file is:

Source address: src_ip
DNS Server: dst_ip
Domain name trying to resolve: qname

The source address, DNS address, and domain name are all included in the node category, and source address->DNS address, DNS address->domain name is included in the relationship category, and the output of node nodes and relationship links as JSON through HTTP is a usable API data input source.

Don't worry, as a conscience to share, how can the technical details be omitted?

In order not to affect the reading, I put the specific content related to data formatting and API in the following links.

Customize the DataV screen and specify the API data source

Enter the customization interface through Alibaba Cloud Console Big Data (Shu Plus) | DataV Data Visualization, and directly select New Visualization.

Create a big screen template, set a name you like.

Add data display components to the large screen layer, where you can choose the relationship network or the native Echarts network diagram.

Click the chart component, enter the data tab on the right, and modify the default data source type static data to API.

Fill in the API that meets the system requirements, such as: http://www.test.com/datav.json , leave the input box with the mouse, and the design area on the left will display the effect immediately.

Release the monitoring screen to see the effect

After the data debugging and preview are successful, the report can be officially released through the release button in the upper right corner. DataV is very thoughtful and provides two security protection mechanisms, password and Token, for the report.

Visit the post-release report link: http://datav.aliyun.com/share/ef9aa**a3fd8 (this is their secret not accessible ha)

interactive discovery

Well, put some data I simulated in the test environment. The following is a graph of the node (source address, DNS server address, domain name to be resolved) of all DNS logs over a period of time.

Click on any of the nodes, the associated points are automatically highlighted, and the non-associated nodes enter the masked state, which is quite cool!

Come, let's look at another set of test data. If there are any suspicious nodes, relationships, and categories, see them at a glance!

Tips: If you find any magical domain name or IP address, you can directly check the threat intelligence of Weibu, for example: https://x.threatbook.cn/domain/google.com. Is it because the bad guys hide immediately, this library is too complete...

Supplementary Instructions

The pit I stepped on for you

If you do not intend to use the server proxy method to call the API, be sure to set the API's HTTP header cross-domain parameters
If HTTPS is used to access the DataV monitor screen, the API interface must also be in the form of HTTPS
The data returned by the API interface should be in exactly the same format as the static data input box - JSON string
If you encounter some magical bug, delete the layer or component and start over (don't ask me how I know)
There is a certain difference in the data format between the relationship diagram that comes with DataV and the embedded native ECharts, please look carefully
Show you the code

The following code can help users automatically obtain DNS session logs after logging in to Situation Awareness, and generate JSON strings according to the data format adapted to the DataV relationship diagram. Temporary code for non-professional developers is ugly, don't take it too seriously.

About "Interactive"

There are two words in the title of this article that I love: "Interaction".

In addition to the interaction on the view, the user actually spends a long time on the query, export and formatting of the situational awareness log.

This is also a kind of "interaction", a relatively stupid interaction. It is hoped that in the future, Alibaba Cloud will make some upgrades in the data reference and internal interface interaction between multiple products, so that users like us can use less skills until we completely "self-abandon martial arts".