Ul Gias, Alim, Gao, Yicheng, Sheldon, Matthew, Perusquía, José A., O’Brien, Owen and Casale, Giuliano 2023. SampleHST: Efficient On-the-Fly Selection of Distributed Traces. NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium. Miami, FL, USA 08 - 12 May 2023 IEEE . https://doi.org/10.1109/noms56928.2023.10154383
| Title | SampleHST: Efficient On-the-Fly Selection of Distributed Traces |
|---|---|
| Authors | Ul Gias, Alim, Gao, Yicheng, Sheldon, Matthew, Perusquía, José A., O’Brien, Owen and Casale, Giuliano |
| Type | Conference paper |
| Abstract | Since only a small number of traces generated from distributed tracing helps in troubleshooting, its storage requirement can be significantly reduced by biasing the selection towards anomalous traces. To aid in this scenario, we propose SampleHST, a novel approach to sample on-the-fly from a stream of traces in an unsupervised manner. SampleHST adjusts the storage quota of normal and anomalous traces depending on the size of its budget. Initially, it utilizes a forest of Half Space Trees (HSTs) for trace scoring. This is based on the distribution of the mass scores across the trees, which characterizes the probability of observing different traces. The mass distribution from HSTs is subsequently used to cluster the traces online leveraging a variant of the mean-shift algorithm. This trace-cluster association eventually drives the sampling decision. We have compared the performance of SampleHST with a recently suggested method using data from a cloud data center and demonstrated that SampleHST improves sampling performance up to by 9.5×. |
| Year | 2023 |
| Conference | NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium |
| Publisher | IEEE |
| Accepted author manuscript | File Access Level Open (open metadata and files) |
| Publication dates | |
| Published in print | 21 Jun 2023 |
| Journal | NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium |
| Book title | Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023 |
| ISBN | 978166547716 |
| Funder | Huawei Technologies |
| Digital Object Identifier (DOI) | https://doi.org/10.1109/noms56928.2023.10154383 |