Pith. sign in

REVIEW 5 major objections 6 minor 48 references

Testing Practices, Challenges, and Developer Perspectives in Open-Source IoT Platforms

T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Open-source smart-home IoT platforms are far less tested than their popularity suggests, with OpenHAB showing a test ratio of 0.04 and Home Assistant 0.42.

desk verdict Useful first snapshot of IoT testing in two platforms, but the headline 0.04 vs 0.42 ratio rests on an unverified cross-language counting rule—worth a serious referee, but the central numbers need re-verification. read the letter →

arxiv 2502.07257 v1 pith:GZP5HUFE submitted 2025-02-11 cs.SE

Daniel Rodriguez-Cardenas , Safwat Ali Khan , Prianka Mandal , Adwait Nadkarni , Kevin Moran , Denys Poshyvanyk This is my paper · ORCID
classification cs.SE
keywords IoTplatformssoftwaretestingtestratiofocalmethodsopen-sourcedevelopersurveysmarthomeunit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to measure how much testing actually happens in open-source smart-home IoT platforms and to understand from developers why testing looks the way it does. By counting functional methods and test methods in OpenHAB and Home Assistant, it reports a stark gap: OpenHAB has about 4,000 test methods against roughly 76,000 functional methods, a test ratio of 0.04, while Home Assistant has about 32,000 test methods against roughly 40,000 functional methods, a test ratio of 0.42. A survey of 80 active IoT developers adds the human side: automated unit testing dominates, manual and integration testing are rare, and developers name device compatibility, firmware updates, and low-power behavior as the hardest things to test. If the numbers hold, the paper establishes that two of the most popular open-source IoT platforms are substantially under-tested, and it points to the areas where testing effort is missing.

What carries the argument

The load-bearing mechanism is the focal method (fm) and focal test method (ftm) distinction, borrowed from the Methods2Test dataset. A focal method is a public method with defined parameters and a body; a focal test method is a method inside a test file, identified by naming conventions, folder structure, and AST parsing, that is meant to test all or part of an fm. The paper's test ratio is defined as ftm divided by (ftm plus fm), with 0.5 as the threshold for a good ratio. This counting procedure converts how much testing exists into a single comparable number across platforms and add-ons, and the paper pairs it with manual labeling of test purpose and with survey responses.

What would settle it

Run the existing test suites of a random sample of add-ons from each platform under a coverage tool such as JaCoCo for Java or pytest-cov for Python, and compare measured line coverage with the paper's test ratio for the same add-ons; if OpenHAB add-ons show high coverage despite a ratio near 0.04, or Home Assistant add-ons show low coverage despite 0.42, the ratio is not a valid proxy. A complementary check is to manually map a sample of focal test methods to their claimed focal methods and count how many have no corresponding executable method.

Watch

Extended reading notes

Core claim

The central claim is that open-source IoT platforms suffer from a significant dearth of testing, and the paper quantifies this for the first time using the focal-method approach. For OpenHAB, only about 4,000 focal test methods exist for roughly 76,000 functional methods, a test ratio of 0.04; for Home Assistant the ratio is 0.42, with about 32,000 focal test methods for roughly 40,000 functional methods. Only 3 of 406 OpenHAB add-ons reach a test ratio above 0.5, while 65% of Home Assistant add-ons fall below that threshold. The survey of 80 developers explains the pattern: developers prefer automated unit testing for early defect detection, rarely do manual or integration testing, and face IoT-specific obstacles such as cross-device compatibility, firmware-over-the-air updates, and low-power mode debugging. Together the two studies yield ten findings about what gets tested, what does not, and what developers say blocks better testing.

Load-bearing premise

The quantitative argument assumes that the ratio of counted test methods to counted functional methods measures how much functionality is actually tested, which requires every counted test method to exercise at least one functional method, a link the paper does not verify by executing tests.

Editorial extensions

If this is right

  • OpenHAB has near-zero automated test presence by this measure: roughly 4 test methods for every 100 functional methods, and most add-ons have no tests at all.
  • Home Assistant is better tested but still leaves 65% of its add-ons below the 0.5 test-ratio line, so under-testing is not confined to one project.
  • Testing effort concentrates on unit-level automated checks, while integration, end-to-end, usability, and acceptance testing are largely absent from the repositories.
  • Developers' stated hardest problems, cross-platform compatibility, firmware updates, and low-power behavior, are precisely the areas where repository evidence shows the least testing.
  • The results give researchers concrete targets: IoT-specific testing tools, simulators for real-world scenarios, and better support for compatibility and update testing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the 0.42 ratio for Home Assistant may overstate true coverage because one parametrized test method can exercise many functional methods, while the 0.04 ratio for OpenHAB may miss tests that live outside the naming and annotation conventions the pipeline recognizes.
  • Beyond the paper: a cheap follow-up would be to calibrate the test ratio against execution-based coverage on a sample of add-ons; if the correlation is strong, the ratio becomes a maintenance-free coverage proxy for large IoT ecosystems.
  • Beyond the paper: the study covers two Java- and Python-heavy platforms, leaving open whether other IoT stacks, such as firmware C code, mobile companion apps, and cloud backends, show similar ratios with the same method.
  • Beyond the paper: if under-testing is as severe as reported, the safety and security consequences for smart homes deserve attention, since untested integrations are a plausible route for faulty automation and device compromise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper reports a mixed-method study of testing practices in two open-source IoT platforms, OpenHAB and HomeAssistant. The authors mine the repositories, define a 'test ratio' based on focal methods (fm) and focal test methods (ftm), and report markedly different ratios (0.04 for OpenHAB and 0.42 for HomeAssistant). They also survey 80 IoT developers and combine the quantitative and qualitative results into ten findings (F1–F10) about testing priorities, preferences, and challenges.

Significance. If the empirical comparison is valid, the paper provides one of the first systematic characterizations of testing in open-source smart-home platforms, and the survey component gives useful insight into developer perceptions. The study is accompanied by a replication package, which is a strength. However, the headline quantitative claim—that OpenHAB has a 'significant dearth of testing' relative to HomeAssistant—rests entirely on a cross-language comparison of ftm counts that is not specified or validated for Python, and the manual validation procedure is not representative. These issues are load-bearing for the central finding, but they are fixable within the manuscript's scope.

major comments (5)
  1. [§IV-A3 and §IV-B] The ftm identification rule is defined only for Java: 'method within a test class with the @Test annotation.' HomeAssistant is predominantly Python, and the paper never states the equivalent Python rule. The headline comparison in F1 (0.04 vs. 0.42) depends entirely on counting ftms comparably across Java and Python. If Python functions in test files are counted by filename matching or by including all functions in a test file, the HomeAssistant numerator would be inflated by fixtures, helpers, setup/teardown routines, and non-test utilities, while the OpenHAB numerator is restricted to @Test-annotated methods. Please specify the exact Python detection rule and validate it with a per-function manual inspection or a stratified random sample.
  2. [§IV-A4] The manual validation sample is not representative of the automated counts. For OpenHAB, the authors selected the top 36 add-ons ranked by the highest count of ftm and classes, and for HomeAssistant they selected the top 43 filenames that appear most frequently. This is a test-heavy, frequency-biased sample, so it cannot validate the overall automated ftm counts. The claimed 'confidence level (z-score) above 95%' is not substantiated: no population size, sample-size formula, margin of error, or per-stratum sampling details are provided, and no inter-rater agreement statistic is reported. Please provide a reproducible sampling plan and quantitative agreement metrics.
  3. [§IV-B, Table I] There is an unexplained internal inconsistency in the file counts. Table I reports 7,071 source files and 5,597 test files for OpenHAB but only 4,585 ftm, giving fewer than one ftm per test file on average, while HomeAssistant has 7,307 test files and 32,735 ftm, about 4.5 ftm per test file. The paper does not explain why OpenHAB test files contain so few @Test methods, and the large number of test files appears to contradict the very low test ratio. Please clarify what counts as a 'test file' and reconcile this discrepancy, because it directly affects the comparison in F1.
  4. [§IX and §IV-B/F1] The conclusion contains numerical claims that are inconsistent with the results in Section IV-B. The conclusion says 'the majority of add-ons and integration apps of both platforms fall[] short of the 50% test coverage threshold' and 'only 5% add-ons contains any test methods for OpenHAB,' while F1 reports that 3 of 406 OpenHAB add-ons (0.7%) surpass the 0.5 threshold and HomeAssistant has 327 of 937 add-ons (35%) above it. The '5%' figure is unexplained and appears to contradict the other numbers. Please correct or clarify these statements so that the abstract, findings, and conclusion are mutually consistent.
  5. [§IV-B and §IX] The paper uses 'test ratio' and 'test coverage' interchangeably (the conclusion refers to a '50% test coverage threshold'). These are not equivalent: ftm/(ftm+fm) is a proportion of test methods, not a measure of how much functionality is exercised. The paper itself acknowledges in Section IV-B that the 0.5 threshold 'doesn't guarantee each fm has a corresponding ftm.' Please use consistent terminology and explicitly state that the test ratio is a proxy for testing activity, not a coverage metric.
minor comments (6)
  1. [§II] The focal method definition says 'a method within a functional block that has defined parameters.' Please clarify whether fm is restricted to public methods, and state how the definition is applied to Python functions, which do not have Java-style class contexts.
  2. [§IV-A1] The text explains that HomeAssistant keeps test code in a dedicated test directory while OpenHAB keeps tests alongside project folders. Please discuss how this organizational difference affects the automated identification of test files and whether it biases the comparison.
  3. [§V-A] The survey recruited participants from community forums and offered a $10 gift card. Please report the number of invitees or the response rate and discuss possible self-selection bias in the 80 valid responses.
  4. [Table II] The meaning of 'Maybe' in the usability and regression rows is unclear. The text says 'we categorize usability and regression testing as Maybe, indicating a potential implementation.' Please define the criterion for 'Maybe' and explain why it differs from the 'ND' entries.
  5. [§VII] The threats-to-validity section says 'We calculate the z-score on our test file sampling and manual inspection to achieve the 95% confidence,' but no calculation is shown. Please move the detailed sampling and confidence computation to Section IV or an appendix.
  6. [Abstract and throughout] There is a typo in the abstract: 'HomeAssitant' should be 'HomeAssistant.' Also, 'add-ons' is used inconsistently as both singular and plural; please standardize.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circular reasoning: the test-ratio metric is an operational definition applied to source code, and the survey is independent self-report data.

full rationale

The paper's central quantitative claim (F1, 0.04 vs 0.42 test ratios) is computed by applying a stated counting pipeline (Section IV-A: regex folder/file matching, tree-sitter AST parsing, and the @Test annotation for Java focal test methods) to external repositories. The ratio ftm/(ftm+fm) is a definition, not a fitted parameter, and no equation is derived from a target conclusion. The 0.5 threshold is an arbitrary design choice openly labeled as not guaranteeing a one-to-one fm-ftm correspondence, so it does not force a specific result. The survey (Section V) is independent self-report data gathered from 80 developers and coded thematically; it is not constructed from the mining results. The paper's own threats-to-validity section acknowledges traceability and cross-language counting concerns, but acknowledging a limitation is not circular reasoning. No load-bearing step reduces to its own inputs or to an unverified self-citation: the only cited definitional basis (Tufano et al. Methods2Test for focal methods) is external prior work that the paper applies, not a premise equivalent to its conclusion.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central quantitative claim depends mainly on the identification of test files and test methods and on the validity of the test-ratio metric. The survey findings depend on self-report. No invented entities are postulated.

free parameters (1)
  • test ratio threshold = 0.5
    Chosen by the authors to classify add-ons as adequately tested or not. It directly determines Finding F1 (3 of 406 OpenHAB add-ons above threshold; 327 of 937 HomeAssistant add-ons above). A different threshold would change the counts, though OpenHAB's overall 0.04 average remains low under any reasonable threshold.
assumptions (4)
  • domain assumption Test files can be identified by folder and naming conventions (src/test, FooTest.java, x_test.py).
    The automated pipeline uses regex-based folder and file matching to separate test code from production code; false negatives or positives would bias the test ratios (Section IV-A2).
  • domain assumption A focal test method is any method with @Test annotation in Java or any function in a Python test file.
    The paper defines ftm with the Java @Test criterion (Section IV-A3) but applies the same logic to Python without specifying an equivalent rule, leaving ambiguity in the HomeAssistant counts.
  • ad hoc to paper The ratio ftm/(ftm+fm) is a meaningful proxy for the degree of testing.
    This metric is introduced by the authors and is not validated against execution-based coverage; the paper itself cautions that the threshold does not guarantee each fm has a corresponding ftm (Section IV-B).
  • domain assumption Survey respondents truthfully and accurately report their testing practices.
    The survey results rely on self-report from self-selected participants recruited from community forums, with no verification against actual behavior (Section V-A1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Testing Practices, Challenges, and Developer Perspectives in Open-Source IoT Platforms." pith.science (2026). https://pith.science/paper/GZP5HUFE

@misc{pith2026250207257,
  author       = {Pith},
  title        = {Pith review of: Testing Practices, Challenges, and Developer Perspectives in Open-Source IoT Platforms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GZP5HUFE}},
  note         = {Machine review of arXiv:2502.07257}
}
abstract

As the popularity of Internet of Things (IoT) platforms grows, users gain unprecedented control over their homes, health monitoring, and daily task automation. However, the testing of software for these platforms poses significant challenges due to their diverse composition, e.g., common smart home platforms are often composed of varied types of devices that use a diverse array of communication protocols, connections to mobile apps, cloud services, as well as integration among various platforms. This paper is the first to uncover both the practices and perceptions behind testing in IoT platforms, particularly open-source smart home platforms. Our study is composed of two key components. First, we mine and empirically analyze the code and integrations of two highly popular and well-maintained open-source IoT platforms, OpenHab and HomeAssitant. Our analysis involves the identification of functional and related test methods based on the focal method approach. We find that OpenHab has only 0.04 test ratio ($\approx 4K$ focal test methods from $\approx 76K$ functional methods) in Java files, while HomeAssitant exhibits higher test ratio of $0.42$, which reveals a significant dearth of testing. Second, to understand the developers' perspective on testing in IoT, and to explain our empirical observations, we survey 80 open-source developers actively engaged in IoT platform development. Our analysis of survey responses reveals a significant focus on automated (unit) testing, and a lack of manual testing, which supports our empirical observations, as well as testing challenges specific to IoT. Together, our empirical analysis and survey yield 10 key findings that uncover the current state of testing in IoT platforms, and reveal key perceptions and challenges. These findings provide valuable guidance to the research community in navigating the complexities of effectively testing IoT platforms.

Figures

Figures reproduced from arXiv: 2502.07257 by the authors.

Figure 1
Figure 1. IoT platform layers and common tests per layer [6] [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. HomeAssistant and OpenHAB platform analysis steps. maintain test codes for each add-ons within their project folders. OpenHAB’s repositories exhibit a variety of code languages, with Java being the most prevalent, accompanied by instances of Python, JavaScript, and Kotlin (see Table I). We noticed that OpenHAB lacks a standardized naming convention for Java classes, with names such as “Test”, “Stub”, or “Mock”, addi… view at source ↗
Figure 2
Figure 2. Focal method (fm) and focal test method (ftm) example. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Left: Test ratio per add-ons. Right: Top ten tested components for HomeAssistant and OpenHAB ‘authentication’, and ‘status’ reflect the platform’s strong emphasis on rule-based automation, security, and system organization. The components common to both HomeAssistant a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 41 canonical work pages

  1. [1]

    Number of internet of things (iot) connected devices worldwide from 2019 to 2023, with forecasts from 2022 to 2030

    Anonymous, “Number of internet of things (iot) connected devices worldwide from 2019 to 2023, with forecasts from 2022 to 2030.” [Online]. Available: https://www.statista.com/statistics/1183457/ iot-connected-devices-worldwide/

    arXiv 2019

  2. [2]

    A Study of Data Store-based Home Automation,

    K. Kafle, K. Moran, S. Manandhar, A. Nadkarni, and D. Poshyvanyk, “A Study of Data Store-based Home Automation,” in Proceedings of the Ninth ACM Conference on Data and Application Security and Privacy. Richardson Texas USA: ACM, Mar. 2019, pp. 73–84. [Online]. Available: https://dl.acm.org/doi/10.1145/3292006.3300031

    work page arXiv 2019

  3. [3]

    Helion: Enabling a Natural Perspective of Home Automation

    S. Manandhar, K. Moran, K. Kafle, R. Tang, D. Poshyvanyk, and A. Nadkarni, “Helion: Enabling a Natural Perspective of Home Automation,” Jun. 2019, arXiv:1907.00124 [cs]. [Online]. Available: http://arxiv.org/abs/1907.00124

    work page Pith review arXiv 2019

  4. [4]

    Technogym connected gym equipment

    TechnoGym, “Technogym connected gym equipment.” [Online]. Available: https://www.technogym.com/en-US/business/

    work page

  5. [5]

    A decade review on smart cities: Paradigms, challenges and opportunities,

    T. Singh, A. Solanki, S. K. Sharma, A. Nayyar, and A. Paul, “A decade review on smart cities: Paradigms, challenges and opportunities,” IEEE Access, vol. 10, pp. 68 319–68 364, 2022

    work page 2022

  6. [6]

    IoT Bugs and Development Challenges,

    A. Makhshari and A. Mesbah, “IoT Bugs and Development Challenges,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), May 2021, pp. 460–472, iSSN: 1558-1225. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9402092

    work page arXiv 2021

  7. [7]

    On the challenges novice programmers experience in developing iot systems: A survey,

    F. Corno, L. De Russis, and J. P. S ˜A¡enz, “On the challenges novice programmers experience in developing iot systems: A survey,” Journal of Systems and Software , vol. 157, p. 110389, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0164121219301566

    work page 2019

  8. [8]

    How is open source software development different in popular iot projects?

    F. Corno, L. De Russis, and J. P. S ´aenz, “How is open source software development different in popular iot projects?” IEEE Access, vol. 8, pp. 28 337–28 348, 2020

    work page 2020

Show all 48 references
  1. [9]

    openHAB

    Anonymous, “openHAB.” [Online]. Available: https://www.openhab.org/

  2. [10]

    Home Assistant

    H. Assistant, “Home Assistant.” [Online]. Available: https://www. home-assistant.io/

  3. [11]

    Methods2Test: A dataset of focal methods mapped to test cases,

    M. Tufano, S. K. Deng, N. Sundaresan, and A. Svyatkovskiy, “Methods2Test: A dataset of focal methods mapped to test cases,” in Proceedings of the 19th International Conference on Mining Software Repositories, May 2022, pp. 299–303, arXiv:2203.12776 [cs]. [Online]. Available: h...

    2022 arXiv

  4. [12]

    Nest App

    Anonymous, “Nest App.” [Online]. Available: https://nest.com/app/

  5. [13]

    Iot testing study online appendix

    Daniel Rodriguez-Cardenas, “Iot testing study online appendix.” [Online]. Available: https://github.com/WM-SEMERU/iot-mining-helion

  6. [14]

    Enabling Mutation Testing for Android Apps,

    M. Linares-V ´asquez, G. Bavota, M. Tufano, K. Moran, M. Di Penta, C. Vendome, C. Bernal-C ´ardenas, and D. Poshyvanyk, “Enabling Mutation Testing for Android Apps,” in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering , Aug. 2017, pp. 233–244, ...

    2017 arXiv

  7. [15]

    Challenges of testing complex internet of things (iot) devices and systems,

    A. K. Gomez and S. Bajaj, “Challenges of testing complex internet of things (iot) devices and systems,” in 2019 11th International Conference on Knowledge and Systems Engineering (KSE) , 2019, pp. 1–4

    2019

  8. [16]

    Testing IoT systems using a hybrid simulation based testing approach,

    S. Bosmans, S. Mercelis, J. Denil, and P. Hellinckx, “Testing IoT systems using a hybrid simulation based testing approach,” Computing, vol. 101, no. 7, pp. 857–872, Jul. 2019. [Online]. Available: https://doi.org/10.1007/s00607-018-0650-5

    2019 doi

  9. [17]

    Tree-sitter Introduction

    Anonymous, “Tree-sitter Introduction.” [Online]. Available: https: //tree-sitter.github.io/tree-sitter/

  10. [18]

    Amazon Alexa

    Amazon, “Amazon Alexa.” [Online]. Available: https://www.developer. amazon.com/en-US/alexa/

  11. [19]

    Homekit

    Anonymous, “Homekit.” [Online]. Available: https://www.apple.com/ home-app/

  12. [20]

    Braun and V

    V . Braun and V . Clarke,Thematic Analysis: A Practical Guide . SAGE Publications, 2021. [Online]. Available: https://books.google.com/books? id=eMArEAAAQBAJ

    2021

  13. [21]

    “False negative - that one is going to kill you

    A. Ami, K. Moran, D. Poshyvanyk, and A. Nadkarni, ““False negative - that one is going to kill you” - Understanding Industry Perspectives of Static Analysis based Security Testing,” in 2024 IEEE Symposium on Security and Privacy (SP) . Los Alamitos, CA, USA: IEEE Computer Soci...

    2024

  14. [22]

    Xuat-copilot: Multi-agent collaborative system for automated user acceptance testing with large language model,

    Z. Wang, W. Wang, Z. Li, L. Wang, C. Yi, X. Xu, L. Cao, H. Su, S. Chen, and J. Zhou, “Xuat-copilot: Multi-agent collaborative system for automated user acceptance testing with large language model,” vol. abs/2401.02705, 2024. [Online]. Available: https: //doi.org/10.48550/arXi...

    2024 doi arXiv

  15. [23]

    Chapter 3 efficient and effective exploratory testing of large-scale software systems,

    T. M˚artensson, D. St ˚ahl, A. Martini, and J. Bosch, “Chapter 3 efficient and effective exploratory testing of large-scale software systems,” in Accelerating Digital Transformation: 10 Years of Software Center . Springer, 2022, pp. 51–81

    2022

  16. [24]

    Modelling android applications through static analysis and systematic exploratory testing,

    J. Doyle, T. Laurent, and A. Ventresque, “Modelling android applications through static analysis and systematic exploratory testing,” in 10th International Conference on Dependable Systems and Their Applications, DSA 2023, Tokyo, Japan, August 10-11, 2023 . IEEE, 2023, pp. 94–...

    2023

  17. [25]

    On effectiveness and efficiency of gamified exploratory GUI testing,

    R. Coppola, T. Fulcini, L. Ardito, M. Torchiano, and E. Al ´egroth, “On effectiveness and efficiency of gamified exploratory GUI testing,” in TOSEM, vol. 50, no. 2, 2024, pp. 322–337. [Online]. Available: https://doi.org/10.1109/TSE.2023.3348036

    2024

  18. [26]

    Apache jmeter,

    E. H. Halili, “Apache jmeter,” 2008

    2008

  19. [27]

    (2023) gatling: Modern Load Testing as Code

    Gatling. (2023) gatling: Modern Load Testing as Code. [Online]. Available: https://github.com/gatling/gatling

    2023

  20. [28]

    (2023) taurus: Automation-friendly framework for Continuous Testing

    Blazemeter. (2023) taurus: Automation-friendly framework for Continuous Testing. [Online]. Available: https://github.com/Blazemeter/ taurus

    2023

  21. [29]

    (2023) General Data Protection Regulation (GDPR) – Official Legal Text

    European Union. (2023) General Data Protection Regulation (GDPR) – Official Legal Text. [Online]. Available: https://gdpr-info.eu/

    2023

  22. [30]

    California Privacy Rights Act of 2020 (“CPRA

    C. S. Legislature, “California Privacy Rights Act of 2020 (“CPRA”),” https://leginfo.legislature.ca.gov/faces/codes displayText. xhtml?division=3.&part=4.&lawCode=CIV&title=1.81.5, 2020

    2020

  23. [31]

    Software testing: Survey of the industry practices,

    T. Hynninen, J. Kasurinen, A. Knutas, and O. Taipale, “Software testing: Survey of the industry practices,” in 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2018, pp. 1449–1454

    2018

  24. [32]

    Practitioners’ Views on Good Software Testing Practices,

    P. S. Kochhar, X. Xia, and D. Lo, “Practitioners’ Views on Good Software Testing Practices,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). Montreal, QC, Canada: IEEE, May 2019, pp. 61–70. [Online]. Availab...

    2019

  25. [33]

    How do developers test android applications?

    M. Linares-V´asquez, C. Bernal-Cardenas, K. Moran, and D. Poshyvanyk, “How do developers test android applications?” in 2017 IEEE Interna- tional Conference on Software Maintenance and Evolution (ICSME) , 2017, pp. 613–622

    2017

  26. [34]

    Looking from the mirror: Evaluating iot device security through mobile companion apps,

    X. Wang, Y . Sun, S. Nanda, and X. Wang, “Looking from the mirror: Evaluating iot device security through mobile companion apps,” in USENIX Security Symposium , 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:199523951

    2019

  27. [35]

    Internet of things as system of systems: A review of methodologies, frameworks, platforms, and tools,

    G. Fortino, C. Savaglio, G. Spezzano, and M. Zhou, “Internet of things as system of systems: A review of methodologies, frameworks, platforms, and tools,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 1, pp. 223–236, 2021

    2021

  28. [36]

    Testing techniques in iot-based systems,

    N. Medhat, S. Moussa, N. Badr, and M. F. Tolba, “Testing techniques in iot-based systems,” in 2019 Ninth International Conference on Intelligent Computing and Information Systems (ICICIS) , 2019, pp. 394–401

    2019

  29. [37]

    Trace2TAP: Synthesizing Trigger-Action Programs from Traces of Behavior,

    L. Zhang, W. He, O. Morkved, V . Zhao, M. L. Littman, S. Lu, and B. Ur, “Trace2TAP: Synthesizing Trigger-Action Programs from Traces of Behavior,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 4, no. 3, pp. 1–26, Sep. 2020. [Online]. Available: https://dl.acm.org...

    2020 doi

  30. [38]

    How Users Interpret Bugs in Trigger-Action Programming,

    W. Brackenbury, A. Deora, J. Ritchey, J. Vallee, W. He, G. Wang, M. L. Littman, and B. Ur, “How Users Interpret Bugs in Trigger-Action Programming,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . Glasgow Scotland Uk: ACM, May 2019, pp. 1–12. ...

    2019

  31. [39]

    What are iot systems for real? an experts’ survey on software engineering aspects,

    G. Reggio, M. Leotta, M. Cerioli, R. Spalazzese, and F. Alkhabbas, “What are iot systems for real? an experts’ survey on software engineering aspects,” Internet of Things, vol. 12, p. 100313, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2542660...

    2020

  32. [40]

    Interoperability and integration testing methods for iot systems: A systematic mapping study,

    M. Bures, M. Klima, V . Rechtberger, X. Bellekens, C. Tachtatzis, R. Atkinson, and B. S. Ahmed, “Interoperability and integration testing methods for iot systems: A systematic mapping study,” in Software Engineering and Formal Methods , F. de Boer and A. Cerone, Eds. Cham: Spr...

    2020

  33. [41]

    Iot bugs and development challenges,

    A. Makhshari and A. Mesbah, “Iot bugs and development challenges,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 460–472

    2021

  34. [42]

    Learning-based methods for cyber attacks detection in iot systems: A survey on methods, analysis, and future prospects,

    U. Inayat, M. F. Zia, S. Mahmood, H. M. Khalid, and M. Benbouzid, “Learning-based methods for cyber attacks detection in iot systems: A survey on methods, analysis, and future prospects,” Electronics, vol. 11, 11 no. 9, 2022. [Online]. Available: https://www.mdpi.com/2079-9292...

    2022

  35. [43]

    Detecting vulnerability on iot device firmware: A survey,

    X. Feng, X. Zhu, Q.-L. Han, W. Zhou, S. Wen, and Y . Xiang, “Detecting vulnerability on iot device firmware: A survey,” IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 1, pp. 25–41, 2023

    2023

  36. [44]

    Understanding IoT Security from a Market-Scale Perspective,

    X. Jin, S. Manandhar, K. Kafle, Z. Lin, and A. Nadkarni, “Understanding IoT Security from a Market-Scale Perspective,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. Los Angeles CA USA: ACM, Nov. 2022, pp. 1615–1629. [Online]. Availab...

    2022

  37. [45]

    State-of- the-art survey of artificial intelligent techniques for iot security,

    T. A. Ahanger, A. Aljumah, and M. Atiquzzaman, “State-of- the-art survey of artificial intelligent techniques for iot security,” Computer Networks , vol. 206, p. 108771, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S138912862200007X

    2022

  38. [46]

    Why Crypto-detectors Fail: A Systematic Evaluation of Cryptographic Misuse Detection Techniques,

    A. S. Ami, N. Cooper, K. Kafle, K. Moran, D. Poshyvanyk, and A. Nadkarni, “Why Crypto-detectors Fail: A Systematic Evaluation of Cryptographic Misuse Detection Techniques,” in 2022 IEEE Symposium on Security and Privacy (SP), May 2022, pp. 614–631, arXiv:2107.07065 [cs]. [Onli...

    2022

  39. [47]

    Survey of Testing Methods and Testbed Development Concerning Internet of Things,

    S. Zhu, S. Yang, X. Gou, Y . Xu, T. Zhang, and Y . Wan, “Survey of Testing Methods and Testbed Development Concerning Internet of Things,” Wireless Personal Communications, vol. 123, no. 1, pp. 165–194, Mar

  40. [2022]

    Available: https://doi.org/10.1007/s11277-021-09124-5 12

    [Online]. Available: https://doi.org/10.1007/s11277-021-09124-5 12

    doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.