Analyzing Security and User Privacy in Network Security Protocols

Abstract:
With the ubiquity of network communications nowadays, network security protocols serve a key role in protecting users from the hostile Internet environment, safeguarding both security and privacy of the transmitted information. As a result of the complications in protocol development life cycle from design to implementation, the network security protocols, such as SSL/TLS and single sign-on (SSO) protocols, have been continually found error-prone. In this thesis, we focus on the security and privacy of the network security protocols by scrutinizing them on the design level (e.g., the protocol specifications, logics and control flows) and the implementation level (e.g., APIs, libraries, user interfaces and relevant applications).

The network security protocol designs are the basis to guarantee the desired security and privacy properties. Without a rigorous analysis, even trivial errors would easily propagate to the downstream implementations and applications. To address such problems, we propose a framework to facilitate the formal modeling and verification of the network security protocol designs. The framework covers the essential components for the analysis, including a formal model for web and network infrastructure, the attacker models and formal definitions of the security and privacy properties. To demonstrate its effectiveness and expressiveness, we have applied the framework to analyze four mainstream SSO protocols.

In spite of the secure and privacy-respecting designs, flaws introduced throughout the network security protocol implementations could still be exploited and abused by the adversaries. In particular, the man-in-the-middle (MITM) attacks and the network traffic analysis are two of the notorious threats against the implemented protocols, targeting security and privacy properties respectively.

The evil twin attack is a type of MITM attack frequently found in the Wi-Fi Protected Access (WPA) Enterprise authentication scheme, commonly due to the failed certificate validation in the SSL/TLS based authentication protocols. To facilitate effective identification of such vulnerabilities and evaluate their security impacts, we propose a systematic analysis approach based on user interface testing. We apply the approach to comprehensively scan the laptops and smart phones installed with four mainstream operating systems, and have identified prevalent insecure user configuration options, among which five are severe vulnerabilities. We further reveal that such vulnerabilities are rooted in the immature design and implementation of the WPA supplicants.

The pervasively deployed network traffic analysis have raised privacy concerns over the information leakage via the encrypted network communication channels such as HTTPS and Tor. Even though the data integrity and confidentiality is protected by encryption, the network traffic patterns still inadvertently reveal information regarding user browsing activities. To gauge such information leakage and understand its privacy implications, we propose a novel web page fingerprinting technique based on features extracted from the characteristic traffic between a browser and a content delivery network (CDN) server during the web page rendering. Through a series of in-depth analysis and evaluations, we have demonstrated that the technique can achieve high accuracy in identifying similar web pages in the same web domain, which allows the adversary to further monitor the target user's browsing behaviour.