What is Blueshift?

Blueshift is a complete network visibility solution providing insight into performance and visibility across the enterprise.  Blueshift cuts through much of the technical jargon to present information about the ‘who‘, ‘what‘ and ‘where‘ of network data flows. Unlike many pure performance monitoring offerings, information is collected from a variety of data sources to build an enriched data set for reporting.

Most importantly, information is presented from a user centric viewpoint allowing for rapid identification and resolution of network performance and security issues when they occur.

 

The Challenge of Monitoring and Visibility

Put simply, there is no shortage of data. Data collection can be undertaken passively via packet capture on the wire or remote agents such as NetFlow, actively via synthetic transactions like ICMP or IP SLA or from end device logs such as syslog. Most solutions focus on just one of these techniques thereby missing other valuable sources of information. Blueshift utilises a mix of data sources, primarily DPI via packet acquisition and NetFlow but also log analysis and to a lesser extent active monitoring. Combining multiple data sources is the only way to achieve user centric visibility.

Data collection in distributed environments is challenging. Blueshift utilises a distributed architecture, processing and correlating data at remote collectors to achieve scalability  in large environments. The largest current Blueshift implementation is monitoring over 40,000 end nodes across 600 discrete subnets, although the architecture is designed to cater for even larger environments.

Maintaining collected data for reporting and analysis s also a key challenge. Fortunately database technology has improved dramatically over the last few years overcoming the inherent performance limitations of old style SQL implementations.  Modern noSQL data stores allow for rapid reporting on large datasets as well as the ability to leverage emerging machine learning and AI techniques for more comprehensive anomaly detection.

Key to effective data analysis is the ability to identify underlying hosts, applications and anomalous activity within the collected data. At the risk of using a cliché, most network and security issues rely on finding the ‘needle in a haystack’ – as the dataset increases in size, finding this needle  becomes increasingly difficult. Good data analysis tools are essential.

Visibility Foundations – Data Collection

Data collection for performance and security monitoring is an ongoing challenge. The high speed nature of current networks results in enormous amounts of data needed to be collected for both security and user experience monitoring.

Blueshift employs passive monitoring techniques using deep packet inspection and NetFlow as raw data sources. DPI is the most effective allowing for comprehensive application classification as well as additional deep dive metrics such as latency. In highly distributed environments NetFlow provides a less granular but lighter weight alternative.

Speed is also a consideration. DPI at speeds above 1Gbps have been traditional expensive often requiring complex custom ASIC or FPGA solutions. The advent of open source high speed packet processing alternatives such as Intel DPDK has enables DPI to be deployed at speeds of 10Gbps+ on standard off the shelf hardware dramatically reducing the cost of solutions.

 

Many protocols, such as DHCP and Active Directory contain device and user information which can uniquely associate an IP address with a specific device or user. Blueshift examines these protocols to enrich input data from DPI or NetFlow. Similarly many application protocols provide additional information, user agent strings from browsers using HTTP provide insight into operation system whilst proprietary apps such as Skype and Dropbox also provide valuable identification data. Dabble continue to build new decodes for protocols to enhance the traditional IP based data sets.

Like device and user identification, application classification is a critical element for both performance and security visibility. Many applications are easy to classify using a standard port to application mapping, however applications sharing HTTP, or worse still using encryption via HTTPS can be difficult to classify. Blueshift uses a comprehensive classification engine to identify such applications, employing SSL certificate inspection to identify encrypted applications.

User centric visibility relies on just more than network flow traffic. For true unified visibility we need to access alternative data sources to enrich data. Dabble are looking to integrate other data sources such as vulnerability management logs from vendors such as Qualys, firewall logs to indicate – even information from simulated phishing campaigns that may show particular vulnerabilities for specific users.  True user centric visibility relies on a disparate set of data sources which Dabble will continue to explore.

DPI and NetFlow provide the base data set for network traffic profiling however for full user centric visibility additional data sources are requires to enrich the base data.

Blueshift User Centric Monitoring

User centric visibility is the ability to view network performance and security from a user perspective. Traditional network performance monitoring used the IP address as the base identification unit, that is, analysis and reports were based around specific IP addresses or worse still complete network ranges. Whilst effective in providing insight as to network performance, the IP address approach doesn’t provide the ability to identify what users or specific devices are actually doing. For example, in an IP monitoring world it is easy to detect anomalous activity and the underlying IP address, this then triggers the search for who or what is actually using this IP address. For effective user experience monitoring and particularly the case of security monitoring, identifying traffic by not only IP address but also device name or user name becomes critical.

User and device identification require additional data sources over and above NetFlow and traditional DPI. Dabble call the integration of this additional data ’data enrichment’. Blueshift employs several types of data enrichment to facilitate user centric visibility. Specifically:

-Broadcast domain snooping via (Dabble Blueshift Bluecast) analyses broadcast and multicast traffic at remote broadcast domains to identify users and devices. Broadcast and multicast traffic provides a rich source of data for user and device analysis (more on this later). Bluecast collects and transfers this information to the central Blueshift collectors to allow for quick correlation of IP address to user/device name.

-Third party logs such as firewall, server and vulnerability management logs have traditionally been used for security and system monitoring tools. Integrating third party logs such as these enriches pure network centric data providing a more comprehensive picture of user activity. For example, when ‘clicking’ of a specific device in the Blueshift the data enriched data set can be queried to not only return network performance data but additional critical information such as current security vulnerabilities, operating system patch levels, recent logins and so on. 

-A large proportion of network traffic is externally Internet facing. User centric visibility needs to also provide the ability to identify external resources. External identification is achieved via DNS or whois lookups to identify Internet based server names and associated data such as carrier and owner. Similarly GeoIP data provides visibility of where the traffic originates. This information is of particular interest from a security perspective in identifying potentially malicious traffic from unusual geographic domains.

User centric visibility is the foundation by which the merging of performance and security monitoring can be achieved.

 

Blueshift Application Classification

Blueshift uses a combination of techniques including deep packet inspection and certificate inspection to achieve comprehensive application classification for both encrypted and non-encrypted traffic.

Application information is presented via the Blueshift context sensitive dashboard allowing fast drill downs to identify application usage for the entire network, specific subnets or even individual users and hosts.

Address Type

Identify the address or packet type of traffic traversing the network – unicast, multicast or broadcast. Packet type monitoring provides an effective indicator of potential security issues across the enterprise.

IP Type

Displaying the underlying IP protocol used such as TCP, UDP, ICNMP and so on. Ongoing collection of IP type assists in traffic profiling to help define network normal.

IP Type

Displaying the underlying IP protocol used such as TCP, UDP, ICNMP and so on. Ongoing collection of IP type assists in traffic profiling to help define network normal.

Application Type

Application type is a detailed analysis of actual applications traversing the network. Application type is available for both encrypted and non-encrypted traffic. 

Top Websites

Top website monitoring goes beyond normal network monitoring to identify where users travel across the Internet. Comprehensive website visibility is critical for monitoring of cloud based applications

Operating System Distribution

Identification of operating systems has become increasingly important in BYOD environments. OS distribution provides clear and concise indication of strange devices that may connect to the network.

Complete stack visibility is provided from IP protocol type right through to high level application statistics. Blueshift application classification identifies hundreds of applications as well as allowing for custom application definitions.

Blueshift application visibility provides quick and comprehensive real-time and historical snapshots of user experience – what they are doing, where they are going and how fast they are doing it.

Blueshift, a simple turn-key solution to satisfy all your network performance and capacity planning requirements.

Blueshift Deployment

Blueshift provides a scalable distributed architecture that allows fro monitoring from very small standalone sites through to large enterprise wide networks. Currently the largest monitored Blueshift site consists of 40,000 devices spread across over 600 subnets.

A Bluecast appliance can be installed in each broadcast domain at each sit collecting user and device identification information that is fed into the central Blueshift Server. A smaller number of Blueshift appliances can be deployed to monitor groups of sites via either DPI or NetFlow which similarly feed data to the central Blueshift server.

The Blueshift server is responsible for data correlation and storage and is based around an Elasticsearch database cluster. Elasticsearch provides a strong scalable cluster platform that can be expanded with extra nodes as required. The Blueshift server also provides the web interface for end user reporting and analysis.

 

Blueshift Reporting – Presenting the Information

Data collection, user centric monitoring, data enrichment and deep packet inspection are all critical elements in providing security and visibility solutions but are meaningless without clear and concise information presentation. Blueshift is built with an easy to use intuitive web based interface that allows seamless access to performance and security information to assess the health of the network.

A series of views provide the ability to take a high level view of overall performance with the ability to drill down to granular information right down to individual network flows in just a few clicks. The graphs are context sensitive or ‘clickable’ to quickly identify areas of interest. For example, if Facebook traffic is of interest, simply click on the Facebook slice of the application pie chart to quickly and easily identify associated information such as top talkers, conversations utilisation.

There are a number of other views that present visibility information graphically or schedule reports for IP accounting.  Put simply, Blueshift is a complete visibility window to the network.

 

Blueshift Dashboard View

The interactive Blueshift dashboard allows quick drill downs from high level enterprise wide data through to isolating individual applications flows in just a few clicks. Dashboard charts are all context sensitive to facilitate complex filters quickly and easily.

Blueshift Summary View

Network traffic can be configured to be displayed by subnet, site or groups of sites to display a quick summary of overall usage. The summary view provides a snapshot of performance across the entire enterprise quickly highlighting sites that may be experiencing performance issues.

More details can be accessed by clicking on the various links on the summary page to identify site or group specific information

 

Blueshift Topology View

The topology view provides a more graphical view of the network.  Network traffic flows can be represented via the topology view to identify the movement of application traffic across the network.

Similar to the summary view, more information is available by clicking on the various nodes in the topology view to display more detailed information.

 

Blueshift Capacity View

The ability to schedule reports to display long term capacity data is essential to identify long term trends or IP accounting.

The Blueshift Capacity view allows for long term historical reports to be generated ad hoc or scheduled daily, weekly or monthly runs.

Capacity reports are available via the web interface or downloaded as CSV files.