Behind the scenes at Black Hat USA 2023, Network Operations Center (NOC) partners watch and wait, scanning the event’s network for suspicious activity. But with over 20,000 cyber enthusiasts and professionals in attendance, protecting the network is no easy task.
Every year, the NOC partners are selected by Black Hat to provide a high security, high availability network to users and protect that network from potential attacks. This year’s partners delivering from Las Vegas included Arista, Cisco, Corelight, Lumen, NetWitness, and Palo Alto Networks.
Collaboration and integration were key to success in this dynamic environment, with leaders joining forces to share API (Automated Programming Interface) keys, documentation, and the unified goal of protecting the conference and its attendees.
While each vendor brought unique infrastructure, architecture, and solutions to the event, NetWitness specifically provided network visibility and network protection with the following objectives:
To ensure attendees didn’t practice what they learned at the conference against the Black Hat network.
To educate attendees about any potential malware or misconfigurations on their own devices.
Unlike typical enterprise environments, the Black Hat network faced unique challenges. Activities that would be considered suspicious elsewhere were commonplace here. NetWitness focused on keeping the network running smoothly for training and presentations while watching for any activities that strayed from these educational contexts.
NetWitness Architecture at the NOC
Each participating vendor and NOC partner supplied the below hardware infrastructure for the NOC to ensure Black Hat’s success.
At the center of NetWitness’s deployment was a robust architecture, made up of:
- The Admin Server: Acted as the central nervous system for management and access.
- ESA – Correlator Server: Generated alerts and correlated data in real-time.
- Packet and Log Decoders: Analyzed network traffic for signs of compromise.
- Endpoint Server: Protected the Registration Server’s integrity.
- Orchestration Server: Seamlessly automated responses and monitored files.
Complementing these were strategic cloud resources:
- NetWitness Insight: Profiled and analyzed network traffic, forming the basis for generating timely alerts.
- NetWitness Orchestrator (ThreatConnect): Streamlined processes and leveraged NetWitness’s threat intelligence capabilities.
- CISCO ThreatGrid: Offered malware analysis, further examining files traversing the network.
The NetWitness team deployed a straightforward yet powerful architecture that included SIEM (Security Information and Event Management), NDR (Network Detection and Response), and EDR (Endpoint Detection and Response) solutions. This setup was strengthened by their collaboration with other vendors, sharing insights and data to effectively differentiate between ‘known-shady’ and ‘bad-shady’ activities.
NetWitness’s Findings at the NOC
NetWitness monitored the Black Hat public Wi-Fi and Registration networks for the top 10 MITRE ATT&CK indicators of compromise, tactics, and techniques.
Ultimately, the Registration network was at higher risk of attack due to the sensitive nature of the information it contained. As NetWitness team Lead Dave Glover explains in an interview with The Register, “[Black Hat is] very different than most environments… There’s nothing to really protect outside of the registration network.” According to another expert in the NOC, the public Wi-Fi at Black Hat was as secure as possible—if an intrusion was detected, the user was sent a reminder that hacking is not permitted.
NetWitness monitored traffic in and out of the networks, focusing on the Registration network. For example, as shown in the graphic below, the sudden uptick in traffic on the Registration network indicates the beginning of the conference.
During the event, NetWitness captured 40.6 billion packets and 2.2 billion logs.
NetWitness also provided packet and log traffic to Palo Alto Wildfire for additional insight and threat analysis.
Overall, NetWitness saw 40,292 unique files. Of those files, 918 were suspicious, and 433 unique files were submitted to Cisco Threat Great for further analysis.
Additionally, of the total files analyzed and processed, 40,000 incidents were fired, resulting in more than 90 findings identified, worked, and managed by the team.
Surprisingly, about 25% of network traffic was unencrypted, and only 50% of the encrypted sessions used TLS 1.3.
Throughout the conference, NetWitness distinguished between malicious activities and misconfigurations. For instance, we identified devices already infected with malware and tracked down owners for notification. We also pinpointed misconfigurations in various applications, including those failing to encrypt sensitive data like PII or GPS coordinates. In severe data leaks or network abuse cases, attendees were invited to the NOC for a friendly heads-up through a captive portal.
Case Studies from the NOC
Tight network security at Black Hat USA 2023 meant some attendees left the conference more secure than when they arrived. If a device connected to the Black Hat network was infected with malware, the team flagged any communication between the malicious code and command-and-control servers, who then contacted its owner. NOC partners were also on the watch for unencrypted traffic, alerting whoever sent or received the data that it was not secure.
Crypto Miner
For example, during the conference, NetWitness discovered that one attendee was using a machine on the network that was already infected with malware before arriving at the conference. An example of an identified crypto miner is shown below.
With the full PCAP functionality of NetWitness, we could reconstruct sessions and scrutinize the data being transmitted. If we encountered something concerning, like a suspected infection, we took steps to locate the device owner to alert them. In instances of network misuse, we approached the responsible individual with a warning—their actions must change, or they risk being removed from the network.
Misconfigurations
We often encountered various misconfigurations, a growing issue with the shift towards cloud-based services. It wasn’t uncommon for us to find phone apps with improperly configured APIs, failing to encrypt their communications adequately. This year, for instance, we spotted messaging apps sending unsecured photo uploads, a marriage facilitation service inadvertently exposing personal information, and a child safety app revealing GPS locations. Some examples of these are below.
When dealing with these misconfigurations, we assessed the level of risk involved. For serious cases, Black Hat might invite the device owner to the NOC to discuss the issue, usually through a non-intrusive notification on their device. Sometimes, if the problem was within the app, Black Hat might contact the app developers directly.
Malicious Threats
NetWitness also identified a range of sophisticated and malicious threats. A notable example involved reconstructing an email that appeared to be a standard Amazon shipping notification from UPS. Upon further investigation, we could access the link within the email, revealing the user’s name, email address, phone number, and home address.
Additionally, NetWitness uncovered a live feed from a pet camera. This intrusion offered a real-time view into the user’s residence.
We also discovered a significant leak in a Chinese lifestyle application. This breach allowed us to see various user-uploaded content, from personal memes and selfies to images of their living spaces.
The most critical misconfiguration we detected involved a child safety location-sharing application. This app, designed for parental monitoring, was transmitting sensitive information, such as precise coordinates, direction, speed, and battery health, without proper encryption, directly to AWS servers. Utilizing the unique capabilities of NetWitness, we extracted the client key of the app, granting us further insights into all related traffic.
As a standard practice, all data gathered during the conference was securely erased at its conclusion. This process began as soon as the final session ended, with all NOC partners participating in the data wiping to ensure the privacy and confidentiality of all attendees.
Closing Thoughts
As the curtain closes on Black Hat USA 2023, the NOC’s role highlights the importance of proactive network security and education, even in the most seemingly insulated environments. By handling massive amounts of data and identifying potential threats and misconfigurations, NetWitness team members didn’t just protect attendees; they taught them how to safeguard their devices better.
In the end, the event was a showcase of security expertise and a reminder of the constant need for vigilance and education in the ever-evolving cyber landscape.