IoT testing and quality assurance indicthreads
Download as PPT, PDF8 likes3,130 views
The document discusses testing for Internet of Things (IoT) software. It begins with an introduction to IoT and describes emerging IoT applications and the typical IoT technology stack. It then discusses challenges in testing IoT software and how the role of quality assurance is changing. The document outlines various areas of IoT testing including connectivity, security, performance, functionality and more. It provides examples of test cases for each area. Finally, it proposes a strategy for effective IoT software testing that emphasizes automation, virtualization, robust backends, and testing at the design stage.
Ad
More Related Content
What's hot (20)
Similar to IoT testing and quality assurance indicthreads (20)
Ad
More from IndicThreads (20)
Ad
Recently uploaded (20)
IoT testing and quality assurance indicthreads
- 1. IoT Testing and Quality Assurance Girish Kolapkar SAS Research and Development (India)
- 2. Agenda • Introduction to Internet of Things (IoT) • Industries approaching the opportunity of IoT • Emerging IoT Applications • Typical IoT Technology Stack • Challenges testing software’s supporting IoT • Software QA’s Role Redefined • IoT Testing Areas • IoT Testing Types • Strategy for effective IoT software testing • Q&A
- 3. Introduction • The Internet of Things (IoT) is the network of physical objects or “things” embedded with electronics, software, sensors, and network connectivity, which enables these objects to collect and exchange data. • Software industry is blending the physical and logical cyber worlds with the Internet of Things. • IoT isn’t merely a buzzword. It’s where we’re headed. • Offers potential for businesses worldwide to unlock value and benefits from devices, systems and services communicating with each other, using the internet as a backbone.
- 4. Industries approaching the opportunity of IoT Internet Of Things Smart Cities and Homes Connected Customer RetailHealthcare Energy Connected Car/ Transportation Communications Manufacturing
- 5. Emerging IoT Applications • Information & Analysis • Tracking Behaviour • Examples- SCM, Trucks with sensors for predictive maintenance • Enhanced Situational Awareness • Examples- Weather Warning Systems • Sensor-driven Decision Analytics • Examples- Oil and gas company optimizing oilfield production
- 6. Emerging IoT Applications (Contd.) • Automation & Control • Process Optimization • Examples-Manufacturing Assembly Line • Optimized Resource Consumption • Examples-Energy Distribution Networks • Complex Autonomous Systems • Examples- Collision Avoidance Systems
- 7. Typical IoT Technology Stack Data Streams at the EdgeData Streams at the Edge ThingDeviceSensorActuator Controller, Agent, Short HaulLong haul Communication InfrastructureInfrastructure Network, Compute ,Data Storage Application PlatformApplication Platform Event Management, Intelligent Filter Transform, Steaming Model Execution ApplicationApplication WebsitesMobile AppsIndustry Specific appliances & consoles, AlertsReports Decision Dashboards Device Hub Gateway, Device Management API DesignBuild
- 8. Challenges testing software’s supporting IoT Big Data-Edges generates astronomical volume of data, speed & variety
- 9. Challenges testing software’s supporting IoT (Contd.) • Complex mix of modern IT & Web services with embedded systems • Security • Expensive test environment setup • Too Many- networks, protocols, gateways, device hubs
- 10. Challenges testing software’s supporting IoT (Contd.) • Multi-user Applications & distributed environment deployment • Applications require decision support in real-time, batch, hybrid modes • Debugging requires understanding of multiple technologies • Difficult to establish point of failures • Limited User Interfaceconsole
- 11. Software QA’s Role Redefined • More focus on user centric testing • Just not find bugs,prevent them • Quality advocates, influencing both development and operational processes • Software testers who do not have experience testing hardware or embedded systems may need to develop some new skills • Testers with domain knowledge
- 12. IoT Testing Areas • Connectivity • Security • Performance • Functionality • Compatibility • Localization • User Experience
- 13. IoT Testing Areas-Connectivity • Seamless connectivity • Data restore when no connectivity • Sample Test cases • Verify that all the gadgets involved in the Internet of things testing are able to register to the network. • Verify that application ensure data security and storage so that whenever the connection is restored, data is back to shape as it was.
- 14. IoT Testing Areas-Security • More possibilities and avenues for cyber attacks • More threats with financial data. E.g. RFID based highway toll collection systems • IoT ecosystem provides multiple entry point for attacks • Lost or stolen devices • Sample Test cases • Verify no unauthorized access to device or information. • Verify data on compromised IoT devices can be remote wiped out.
- 15. IoT Testing Areas-Performance • Device- to-Device communication • Device to Server communication • Server To Server communication • Network bandwidth, latency, packet loss • Sample Test cases • Verify response time against benchmarked time with defined connectivity conditions.
- 16. IoT Testing Areas-Functionality • WebMobile applications • Functionalbusiness requirements • Access control • Data Storage • IdentityRoles management • Sample Test cases • Verify medical monitoring system returning correct outputs.
- 17. IoT Testing Areas-Compatibility • Device-to-device communication protocols are not compatible, forcing the user to select a family of devices that employ a common protocol • Lack of standardization leads to incompatibility issues • Compatibility needs to be validated for combinations of devices, protocols, operating systems • Sample Test cases • Verify IoT software supports defined set of decives.
- 18. IoT Testing Areas-User Experience • Refers to a person's total experience using a particular product, system or service • Familiar • Clear & Concise • Feedback • Consistent • Attractive • Efficient • Forgiving
- 19. IoT Testing Types • Compatibility Testing • Functional Testing • Performance Testing • Security Testing • Exploratory Testing • Load Testing
- 20. Strategy for effective IoT software testing • Automation rules-Develop automation systems, through code, that can ensure quality standards at each level and which assures - • Response Time • Data Validation • Real Time Data Accuracy • Secure Access control • Early detection of issues at design time • Needs more detailed Acceptance Criteria
- 21. • Robust Backend – NoSQL ? • Minimal manual end to end testing • Virtualization • Simulators for various industries • Remote Testing • Mock up the API Strategy for effective IoT software testing (Contd.)
- 22. • Elevate a test approach rather than simply executing it • Analyse Test Results and Test effectiveness • Measure quality of Test • Timely review testing tool set as per user feedback, remote debugging and quick turnaround Strategy for effective IoT software testing (Contd.)
- 23. • Generate Realistic Test data • Data Generator Tool • Test Analytics • Understand signals in data to derive new patterns based test cases Strategy for effective IoT software testing (Contd.)
- 24. • Effective Test Data Management Tool • Data on Demand • Data Integration • Data Security • Data validation Strategy for effective IoT software testing (Contd.)
- 25. • Test suite of source adapters • Predefined adapters e.g. Hadoop (+YARN), RabbitMQ • Test for publish(read) only- e.g. Sniffer,Twitter, SYSLOG, HTTP RESTful Interfaces • Test for subscribe(write) only- e.g. SOAP,SMTP • Customizable publishsubscribe API’s Example- Test Event Stream Processing Engine (Contd.)
- 26. • Test visual interface which definerefine event stream models Example- Test Event Stream Processing Engine (Contd.)
- 27. • Deploy & Test On Workstation • E.g. Test for iPhone sensor logs • Deploy & Test on Edge Gateway • E.g. Test for physical raspberry-pi Example- Test Event Stream Processing Engine (Contd.)
- 28. • Connect iPhone and Laptop to wireless network Example- Test Event Stream Processing Engine (Contd.)
- 29. • Step 1: Install and check SensorLog app on iPhone, get ip address. • Step2: Start ESP server and put ESP model in test mode • Step3:Run SensorLog and start streaming data through socket • Step4: Run adapter script to read iPhone sensor data • Step5: Open visualization to see streaming data • Choose line graph to show sensor data and moving average • Shake your iPhone to see graph changes Example- Test Event Stream Processing Engine (Contd.)
- 30. QUESTIONS?
Editor's Notes
- #13: Functionality : Does your apps work in the hands of your users? User Experience: Are your apps as intuitive to your users as you think? Localization : Are your apps culturally appropriate?
- #20: What is the OWASP Internet of Things Project? The OWASP Internet of Things Project provides information on: IoT Attack Surface Areas IoT Vulnerabilities Firmware Analysis ICS/SCADA Software Weaknesses Community Information
- #22: http://www.contiki-os.org/start.html#start-cooja