Nov 19, 2018

Top 4 considerations for IoT Testing

The Internet of Things (IoT) is the new buzzword with predictions ranging from “it’s just another connected network” to “this is the new sliced bread of the generation”. Every new idea causes change; RADICAL ideas have a greater impact and REVOLUTIONARY ideas disrupt the course of humanity. IoT, as a revolutionary idea is coming off age and holds the promises to disrupt the Industrial Ecosystem.IT is already being touted as the next Industrial Revolution. The below graphics are evidence enough.

Before we start thinking about what it means to the Testing industry, let us agree on the definition, to be on the same page!

“The network of physical objects that contain embedded technology to communicate and sense or interact with their internal states or the external environment.”

With this definition and from what we see in reality, there are two different types of IoT;

  • The Industrial IoT
  • The Consumer IoT

The Industrial IoT covers areas that organizations could use for improving the efficiency and/or effectiveness of producing/delivering their products/services, which includes:

  • Smart Factories
  • Utility Grids
  • Intelligent Machines/Robots
  • Smart Cities
  • Waiter-less Japanese restaurants [hmm, no tips?]

The Consumer face of IoT focuses on providing a richer experience to individuals and families. Some examples include:

  • Smart Appliances (Whirlpool integrating Amazon Dash into its Washing machines or Parrot’s plant-watering sensors)
  • Self-driving/autonomous cars
  • Smart utility meters
  • Wearables

Many of them are already here, but this is only the beginning. As more and more devices get connected, sophisticated software’s developed to collect, process and store data, better algorithms designed to make faster decisions, thereby, showcasing a substantial increase in the use of IoT in our daily lives.

All this means that many physical things now will become digital and bring with it, its own set of challenges. Failed connections, loss of power, hardware failures, software errors and a lot more, could cause a myriad of issues – minor inconveniences to large-scale loss; like –

  • Security threats leading to privacy and financial loss
  • Slow performance due to overworked networks and latency
  • Low fault-tolerance and high-maintenance devices
  • New interfaces
  • Completeness of sensor and other recorded data

Some breaches that could occur include:

  • Target (retail giant in the US) data breach through malicious hacking of PoS systems
  • Easily control a connected car
  • Home Automation devices get hacked – lights are flipped or IP cameras are turned on without the owner’s knowledge

What has this got to do with Testing?

Testing the IoT system means that all the skills that are now applied individually to hardware and software systems must be brought together under one roof, to develop and deliver an IoT test solution. This could be a challenge as not all organizations would have the processes, the skills and the tools to get the job done effectively. Following are some considerations that need to be kept in mind while designing IoT test cases.

Following are some considerations that need to be kept in mind while designing IoT test cases –

  • Connected devices – which means hardware, firmware and software are equal points of failure
  • Multiple protocols – different devices are connected using a variety of protocols, with each interconnection having the potential to disrupt the system
  • Device Fragmentation – this includes OS, architecture and other forms of differences between the connected devices
  • Power considerations – many field systems have low power specifications. Simply connecting them could cause them to drain power rapidly, so software must be written with this in mind

What should a typical IoT Test Strategy cover?

    • Embedded System Testing – Testing the Device as a whitebox
    • Web Testing – Testing the device as another application on the web
    • Security Testing – Authentication, Privacy & Control Levels (Autonomy vs Controlled)
    • UI Testing – User Interface, remote administration, and usability
    • Architecture based Testing – Interfaces, OS and Device fragmentations
    • Exploratory Testing – Increased use of SBET and Testing Tours
    • Interoperability Testing – Supported protocols, encryption, and data transfer
  • Automation and Tools – Multi-layered automation, use of Tools for specific areas

It is recommended that companies looking to Test IoT systems build a deep understanding of Embedded system/hardware, Hardware and Software APIs, Multi-protocol Testing and also a high-level of “White-hat hacking” capabilities.

Going by current trends, however, what may be possible is the emergence of an ecosystem of organizations with complementary skills that partner together to develop and test complex IoT systems.

It is only a matter of time and we will see mature organizations delivering complex IoT testing solutions.