H ang around embedded software engineers long enough and the words design for test or test-driven development will become commonplace. This is because in a world where functionality is ever-increasing in complexity, you need to be able to both verify and validate your device’s functionality such that it matches the requirements. And, while these design practices are well understood in digital systems made up of microcontrollers (MCUs) or system-on-chip devices (SoC) with functional safety driving extremely distributed systems, they are just as applicable in low level mixed signal devices.
Gone are the days where the main MCU of the embedded module is trusted to do everything; in industrial and automotive systems where safety is critical, there now exist other devices to help test the main microcontroller to aid in the safety integrity level (SIL) of the device. These functions vary in complexity and range from helping the MCU toggle pins, to ensure stuck at faults are mitigated, to helping verify complex question and answer watchdog, and voltage monitoring functionality.
Often the next ‘smartest’ device in the system is either another low-level MCU or, in the quest to simplify a bill of material (BOM), a power management device (a PMIC) with dedicated safety functions. And since these devices don’t have flash memory, and are traditionally analog in nature, it makes the validation of these devices somewhat challenging for engineers who traditionally focused mostly on transient load responses.
To aid in helping an engineer develop a test philosophy for such a device, we will focus on a distributed system made up of both MCU and PMIC. The article will use this system to demonstrate validation concepts that system designers have been employing for quite some time, the simplest of which is an open loop test philosophy and its strengths and weaknesses. The closed or in-the-loop based test philosophy will address those weaknesses and demonstrate how one can easily expand upon their test setup by including an MCU to model the system or system device. Taking these two test philosophies together will shed light on how exactly a design-for-test philosophy can be adopted for a traditional analog-based device such as a PMIC.
Figure 1: Example of open loop (top) and closed loop (bottom) test setups
In both these examples, we’ll introduce languages and methods needed to be able to easily implement and address. We’ll also introduce simple yet effective constructs in C that can be implemented to help with test flow and modularity. In the end, the goal is to help you come up with design patterns that not only focus the validation of basic functionality of the application specific integrated circuit (ASIC) but help emulate the system integration tests during validation to address complexity.
In order to best approach how to create a scalable test architecture for an embedded ASIC, an example system needs to be defined. After definition, we then address requirements that are applicable to both the MCU and the PMIC individually, and the requirements that are applicable to both devices which ultimately make up the core of the system.
This system is outlined in Figure 2, made up of a high-end MCU, a power management device, along with an external sensor that will monitor throttle position.
Figure 2: Our example throttle by wire system, with some simple interfaces
The MCU is responsible for sampling the throttle position sensor and then managing the air, fuel, and spark of the engine to ensure acceleration is kept constant and smooth; this is defined as a simplified, but typical, throttle by wire system. The power management device needs to deliver monitored power to both the MCU and the throttle position sensor, assists in monitoring the MCU and system voltages, and alerts the system when it is unable to do so reliably.
Each active device in this example, the MCU and PMIC, needs to go through two types of testing prior to starting their task. The requisite testing includes:
Taking these two points into account, we’ll modify Figure 2 by identifying the critical interfaces that will be used to help validate not only the PMIC but any system requirements that the PMIC must perform to help test the MCU.
Figure 3: Our example throttle by wire system, with critical interfaces
In summary, our example system functionality includes requirements such as the ability to:
At first glance, this seems like quite the task, as the PMIC is normally focused with only one goal, that is, designing a high-performance regulation loop with a focus on power integrity and not on the validation of embedded functions. Because of this, things like loads and specialized power supplies are the norm for these engineers, but not digital devices such as digital to analog converters (DACs) or MCUs.
When it comes to testing these embedded functions, the focus must be on developing a test setup that allows the PMIC team to ensure that these functions work properly by exercising their pass-fail criteria. Because of the variety of functionality to test, validation can be done in one of the following two ways with low-cost devices that are easy to implement and program:
We’ll start first by explaining some methods for working with the PMIC device in an open loop fashion, where there’s no plant device that the PMIC will work with.
A simple open loop test setup is a valuable tool in a validation engineer’s toolbox, since it allows them to validate communication through an external interface, such as a:
Overall, the goal of an open loop setup is to be able to communicate not only to the device under test (DUT, in this case, the PMIC), but to trigger measurement devices to automate as much of the test sequencing as possible. An example setup is found in Figure 4.
Figure 4: A common open loop bench setup for evaluating a PMIC
Traditionally higher-end test setups will utilize Labview, or Matlab, but Python (a rather new but very powerful scripting language), and lab equipment capable of receiving digital commands over a standardized interface is perfectly usable. Regardless of how you set this up, creating an environment that allows a user to communicate and control various devices in sequence is crucial in an open loop setup for automation and repeatability.
In order to demonstrate how to evaluate the effectiveness of our open loop test setup and its ability to interface, we’ll examine two system test cases denoted by our example system; they are:
This test case focuses on the ability of the PMIC to monitor a voltage and issue a fault reaction within a certain time. The example requires the device to react properly under a swept VIN and load to ensure the accuracy of the converter. To execute this test case, the open loop setup makes use of the following hardware:
The sample system is outlined in Figure 5.
Figure 5: A bench setup used for evaluating ADC accuracy and fault response
While this may seem like a simple setup, that power that comes from this is using them together to run and rerun a test setup using our Python scripting language. With just a few commands in Python, you can program this test setup to:
With this setup, we can easily swap in and out different versions of the PMIC or a new board to perform regression testing and to make sure that the design functions from revision to revision.
As is the case with many PMIC devices that sit in systems that adhere to functional safety standards, most DUTs have internal sensors to monitor various reference points inside the device, and a dedicated GPIO to alert the system of an error in one of those sensors. In this case, we assemble the following equipment:
The setup is found in Figure 6.
Figure 6: A bench setup used for evaluating fault reaction and response
In both of these simplified cases, the validation engineer gains the ability to create an open loop regression test suite that allows them to test new devices and boards with a common test setup. A computer can script these sequences, take screen shots, and log data simultaneously. It is relatively simple to set up, provided your lab equipment includes addressable devices, your validation group opts for a suitable software license, or opts to control these devices via Python.
However, where this setup falls short is in the modeling an actual embedded system. The assessed functionality is limited to just the DUT and the actions of the DUT in response to directed stimuli and does not include the interaction of the DUT with the accompanying MCU in response to those stimuli. For that, we turn to a closed loop setup or a system model.
Traditionally, validation engineers of ASIC type devices focus on simply validating the device in an open loop fashion. For example, for a simple regulator, they are most likely concerned with:
Both of these examples employ a simple oscilloscope and power supply, with an engineer performing a manual evaluation of the setup. However, in a mixed signal design that is highly integrated with a microcontroller, this approach is limited. To address this, we turn to a model of the system with the MCU in it. There is a wide variety of ways to accomplish this, including:
In the rest of this article, we’ll consider a closed loop PIL approach which is typical of a system validation that is now becoming more commonplace at the device level.
A MCU offers the example test system a lot of freedom in how evaluate. Among the many advantages are:
Quite possibly the most attractive part of an MCU is that evaluation boards that allow access to the functionality do not necessarily need to be the end target hardware and thus can be much less expensive and easier to program.
In our example throttle-by-wire system, the addition of the microcontroller allows us to introduce more complex test scenarios. Two examples of these more complex scenarios are:
Figure 7: Modeling our system using a PIL method
Now that we have the microcontroller and a concept of what a closed loop evaluation system can offer us, we need to discuss some strategies that go into creating firmware to facilitate a state driven test environment.
Now that we’ve defined what a closed loop validation system is our MCU firmware needs to be written so that it takes full advantage of the environment. This means that the validation engineer needs to implement design patterns for testing that influence:
Taking these into account, we’ll now discuss some embedded C-level constructs that can be used in order to control execution and to address our modularity requirement. Together, these will give us a great amount of flexibility in debugging and regression testing.
First, we introduce the concept of a state driven test environment through the use of a common design pattern found in digital and embedded systems, the state machine. A state machine is a design pattern that allows the designer to organize functions and behavior by defining and tightly controlling the states of a system. It is commonly found in the negotiation of ethernet handshaking, or the internals of a CPU.
An example, found in Figure 8, is a state diagram that outlines the startup and initialization tests between the two devices in our example system, the PMIC and the MCU.
Figure 8: A state diagram depicting initialization tests ran on the MCU in a closed loop model
The diagram conceptually organizes the execution pattern into individual functions, with pass-fail criteria that allow us to control the flow of execution. This concept is extremely powerful in our validation environment because of how the system’s functionality is distributed between the PMIC and MCU. Otherwise, it would be difficult to understand where the PMIC (or any other ASIC, without a debugging environment) is in its internal processes.
This design pattern is borrowed from a universal verification methodology (UVM). With this context, we can define a finite state machine as a computational model used to simulate sequential logic in a ‘stateful’ means. It allows abstraction of a complex series of events to a series of states to control execution flow.
The implementation is usually done in C and follows a design pattern similar to that in Figure 8. An example of its instantiation is shown in Figure 9.
Figure 9: An example design pattern of a state machine in C
The state machine is in control of the order of execution and driving a test to a specific operating point, an approach best suited for forcing the PMIC/MCU interaction through a series of defined steps with each state transition gated by a test assertion.
For example, for the system to move from the initialization state to a communications interface state, the combination has to match the expected behavior. If it does, this successfully satisfies the ‘pass’ case. Otherwise, the test would signify a failure and the ‘fail’ case would be able to be validated
Another example, shown in Figure 10, demonstrates how the state machine can control the order of execution to a finite end.
Figure 10: A state diagram depicting the startup sequencing of a PIL based system
The MCU needs to command the PMIC to dynamically change voltages (this is commonly referred to as DVFS), require the PMIC to detect a fault which would then reset the system, and then allow it to recover from a faulted state.
By implementing this concept as a state machine, depicted in Figure 11, the system can easily validate both the ‘pass’ case, in which the system recovers, as well as the ‘fail’ case in which the system goes to an error state.
Figure 11: A state diagram depicting the startup sequencing of a PIL based system
However powerful state machines are in controlling execution, the validation of these requirements often require the need for a pass and fail case to be tested. And while an individual can copy their firmware, if they adopt modularity in the design of their test cases, they would easily be able to reuse work need to address both the concepts of modularity and program control.
As we alluded to in the previous section, our validation setup will often need to validate both the pass and fail paths. And while there are several different ways to address this, one overlooked function in C is compile time build options.
In production environments, build options (sometimes referred to as compile switches) are a powerful tool to create modularity in firmware for building various embedded targets or multiple applications. Inside a disciplined organization, multiple people using the same compile switch can reconfigure an entire application by compiling large sections of data in and out instead of creating, tracking, and supporting new firmware variants.
However, in a state driven test environment, the goal is not to be memory efficient but to be able to support a wide variety of test cases in a single design pattern to take advantage of the software architecture choice. To demonstrate how powerful these test cases can be, we present two test cases in which our embedded system model needs to perform a test of:
In the case of the watchdog, the state diagram is depicted in Figure 12. We can easily reuse our main, non-finite state machine to step the system to the watchdog interface validation test, and then use compile switch to examine the simple window watchdog case, followed by the challenge-response watchdog case.
Figure 12: Pseudocode for a WDT test with conditional compilation
Additionally, we could create a pass/fail compile time switch that would validate the ‘pass’ test case and the ‘fail’ test case inside of each function.
In the case of the external pin, we simply create three functions:
In each of these toggleable situations, demonstrated in Figure 13, the reaction to the fault is organically used, and the compile switch is used to create a fault.
Figure 13: A state diagram depicting the startup sequencing of a PIL based system
However, this C construct is not without risk. The main dangers of overuse of these build switches is their complexity and the ability to document them, especially when they appear in various sections throughout the validation application. Overuse and poor discipline can create a complex set of decisions that are difficult to maintain, let alone pass to another validation engineer. To combat these, we suggest to:
Validation of traditional, analog-based devices is becoming much more complex with the advent of highly integrated systems. Digital functions like watchdog timers, pin checking, and ADCs are finding their way into mixed signal devices and their functionality needs to be exercised as rigorously as the analog control loop. Using an open loop test setup, an engineer can get a jump on the validation using some simple scripting tools or more expensive, off-the-shelf ones. But quickly, they may find limitations in that approach, depending upon how deep their validation plan takes them.
By implementing a state driven test concept in a closed loop system with a model of the target embedded device, you can achieve higher validation coverage and create a method to help identify and fix those hard-to-catch bugs before release!
Christopher Semanson works at Renesas Electronics America Inc. as a Staff Power Systems Applications Engineer in Durham, NC supporting the design of PMICs and other power generation semiconductors in automotive applications in accordance with ISO 26262. He has five years previous experience in EMC Education at the University of Michigan, teaching EMC and Electronics with Mark Steffka. Semanson has a bachelor’s degree in Electrical and Computer Engineering and a master’s degree in Electrical Engineering from the University of Michigan Dearborn. He can be reached at christopher.semanson@renesas.com.
Your email address will not be published.
Save my name, email, and website in this browser for the next time I comment.
In Compliance is a leading source of news, information, education, and inspiration for electrical and electronics engineering professionals.
Aerospace Automotive Communications Consumer Electronics Education Energy & Power Industrial Information Technology Medical Military & Defense