Abstract

The adoption of network monitoring techniques has culminated in a broad availability of raw data. However, giving the large-scale and complexity of long-distance networks and the massive volume of distributed data, minor progress has been achieved regarding the in-depth analysis of this collected data and inferences about the network behavior. This Work Group investigates methods to analyze the data measured by the existing monitoring tools present in the IPÊ network. The result will be a system that provides inferences to aid the process of operation, traffic engineering and network planning.

Context

Several monitoring techniques, patterns, and tools were developed in the past, and all have been maturing in a way to support the operational needs of networks. Such approaches allowed the obtaining and the storing of a massive volume of raw data. However, minor progress was obtained regarding the in-depth analysis of this collected data and in the creation of inferences about the network behavior.

Motivation

The scale and complexity of current networks make the task of analyzing large volumes of data with the existing tools hard (or even intractable). Besides, the heterogeneity of the raw data - caused by equipment with diverse measurement capabilities, variations regarding the granularity of collected information or isolated measurements - makes it difficult to standardize the analysis of the data.

Proposal

This Work Group is developing a system to analyze the data collected by the existing monitoring tools present in the IPÊ network and applying Big Data Analytics techniques to offer several relevant information to aid the operational management, traffic engineering, and network planning. The system will provide automatized correlations and categorizations, personalized user profiles based on different aspects, interactive filtering and manipulation, root-cause analysis to network events, statistics over traffic engineering efficiency and the ability to analyze the impact of changes in the network.

Project Overview.

Specific Goals

1) Mapping the information collected by the monitoring tools present in the IPÊ network, allowing the understanding of which raw data are suitable for analysis.
2) Study and Process the data to develop strategies to analyze the big volume of data and extract correlations and inferences. This step is composed of three different tasks: (a) identification of possible correlations and assumptions, (b) exploration of big data analytics techniques, and (c) analysis of tools for manipulating and storing data.
3) Enable interactive data analysis to increase the control over the data analysis, and facilitate investigations. Support will be included for personalized queries with user-defined filters and enhanced manipulation of the data visualization system.
4) Review the collected data to generate new inferences. This task is viable after having done the previous tasks. At this point, new metrics that can add value and be easily included by the infrastructure monitoring operators will be considered.