Why Sweat the Small Stuff in Functional Test?

7 Best Practices for Mid-Volume / High-Mix Production Test

Author: Patrick Kelly, Cal-Bay Systems, Inc.

Executive Summary

The total cost of ownership for a functional tester includes many things beyond just the cost of instrumentation, fixtures, and software design.
The small stuff adds up quickly & includes things such as: Archiving & Reporting on Test Data Tester Diagnostics Configuration Management Scalability to Future UUTs Maintenance, Back-up Plans, Etc. Companies often rush through important architectural planning and design steps, which can dramatically increase the total cost and schedule, mainly due to the small stuff. This white paper reviews the key cost drivers involved in building a high-mix functional tester, allowing managers to more accurately access the total costs of alternative automation plans.

(Digital rendering of a typical FlexATE)

C A L B A Y. C O M

Best Practice #1: Handle the Big Stuff

An article about small stuff is only meaningful in the context of understanding what big stuff means.
The big stuff in high-mix functional test includes the following: 1. Choice of Instrumentation 2. Choice of Software Tools and Software Architecture 3. Fixture Architecture
1.1 Instrument Selection
Choosing the right instruments is often the first critical step. Obviously, you cant automate a test process that cant be successfully tested manually. Determining the correct instruments, which consistently and accurately measure your UUT (Unit under Test), is the first of the big stuff. An important best practice for instrument selection is to choose a COTS (commercial off-the-shelf) instrument from a sustainable vendor. If the instrument vendor goes away, then you have a major obsolescence problem, which is why most ATEs are built with instruments from National Instruments, HP (Agilent), Tektronix, Keithley, etc. Cal-Bay has extensive experience integrating these COTS instruments into ATEs (either 19 rack systems or the FlexATE platform). On many projects in the early stages, there are unknown measurement requirements. When this happens, CalBay will propose a specification phase to ensure that the instrumentation selected will accurately measure all aspects needed by the client. Since the instrument selection can dramatically impact cost, the final price can only be determined AFTER the specification phase is finished.

C A L B A Y. C O M

Best Practice #1: Handle the Big Stuff

1.2 Software Tool Selection and Software Architecture
Cal-Bay builds most ATEs with NI LabVIEW and TestStand (COTS products). We use the specification phase to properly understand all requirements and select the best software architecture for your exact long-term needs. For example, during a spec phase, we would document such things as: 1. 2. 3. 4. 5. 6. 7. 8. Multiple Use Cases (different perspectives of how the software will be used) Test Executive Selection Programming Language Selection for Test Steps Test Parameter and Configuration Management Test Data / Results Storage & Analysis Report Requirements for Engineers & Managers Initial User Manual with Mock-Up Screen Shots Selection of Toolkits and Third-Party Software Components

Scalable ATE Architecture

Instruments

Fixture

Computer

UUT

o ATE S

ftwar

hite e Arc

cture

Cal-Bay has standardized on our iVVivi platform, which offers an Enterprise-optimized work-flow process for the creation of ATE software. We have standardized on IntraStage for Test Data Management, Test Results Handling, and Report Generation. More information on these tools is available in our brochure Improving Your Product Quality with a Smarter Test Software Process and our flash video Solid Software Design for Test.

DIAdem

Scalable Plug-in Architecture Standards Documentation Training Templates Examples

C A L B A Y. C O M

Best Practice #1: Handle the Big Stuff

1.3 Fixture Architecture
Fixture architecture will depend on the selection of a mass interconnect, which is Best Practice #6. Special signal or test requirements include: 1. 2. 3. 4. 5. 6. 7. 8. 9. RF signals, which may require an enclosure, and often require special cabling, attenuation, and filters Light source requirements for testing sensors, which may require a darkened enclosure High voltage signals (> 24 V), which must be handled, documented, and maintained carefully Low voltage signals (< 10 mV), which may require signal conditioning High speed DIO, which may require special cabling Pneumatics Part handling Barcode readers Temperature chambers

10. HASS chambers 11. Etc. Cal-Bay has built systems to solve all of these unique requirements, and the pictures below are some examples:

Bed of Nails" tester & pneumatics

Example of fixturing & cabling to an outside chamber

HASS tester where the fixture handles 16-units at a time

C A L B A Y. C O M

Best Practice #2: Design Diagnostics So You Can Test Your Tester
Companies who work in a high-mix production environment must consider the costs of tearing down, setting up, or debugging an ATE. Ideally, you want to test the ATE daily to ensure that all instrumentation and wiring is set up properly. Having a self-test diagnostic fixture (STDF) is a great way to achieve this.
An STDF is a combination of a hardware fixture and software. The hardware routes together inputs and outputs and contains active electronics when necessary. The software sequences through many stimuli and measures the responses to determine if all instruments in the system are running and wired properly. An STDF eliminates the need to have golden UUTs and offers advantages such as: Determine False Failures AND False Passes (where a Golden UUT can only catch False Failures) Tests the equipment more thoroughly, since an STDF is not limited to the UUTs specifications STDFs are sometimes less expensive than the Golden UUT (if your product costs are high) STDFs are simpler electronically, so they retain golden status longer and are easier to debug One STDF for multiple testers vs. multiple golden UUTs (one for each type of tester) Since there is only one STDF, the documentation and test sequence is simplified

Building an STDF from scratch can be very expensive, which is why most companies dont use them. Luckily, Cal-Bay has extensive experience and existing intellectual property with STDFs using PXI instruments. Involving Cal-Bay in the specification phase of the project allows us to recommend certain instrumentation that will lower your total NRE. The final benefit of the STDF is that Cal-Bay can use it to test and validate your personalized ATE before we ship it to you. This improves supply chain quality and lowers the costs of receiving and validating duplicate ATE systems.

C A L B A Y. C O M

Best Practice #3: Manage Your Test Data with COTS Solutions
A typical functional test system will gather gigabytes of data. Yet, the management and reporting of that data is typically an afterthought. Traditionally, data is written out in un-standardized formats and only available on the ATEs Computer.
This creates the following issues: 1. Disparate and non-standard data files, spread throughout the organization 2. Manual and complex processes for gathering all of the data necessary for creating reports, which wastes engineers time 3. Data is at-risk of loss, due to hard drive failures or inadvertent file overwrites 4. Managers cant easily get visibility into yields, trends, or SPC information 5. Marketing and Sales personnel cant easily get reports to customers or prospects about product specifications, pass / fail information, or product characteristics Cal-Bay has chosen to standardize on IntraStage as a COTS solution for Test Data Management. By including IntraStage in our system architecture, we can save our test results in LabVIEW into a format that IntraStage can easily import and share with your ERP / MES.

Analytics Software Web-Based | DIAdem | Excel | Matlab | Etc.

Manager Eng 1 Eng ManagerEng 1 Eng Client Client Client Manager Client Manager Eng 1 2 Web Dashboard Dashboard Dashboard Dashb Web Dashboard Web Dashboard Web Dashboard DashboardDashboard

CENTRAL SERVER Central Ser Central Server Central Server IntraStage & NI Data Finder

IntraStage & NI Data FinderIntraStage & NI Da IntraStage & NI Data Finder

ATE 1

ATE 1 R/D ATE 2 ATEs

ATE 1ATE 3 ATE 2

ATE 2 3 ATE

Each ATE Contains: Computer Instruments Fixture iVVivi SW Standard Architecture

LabVIEW and TestStand DIAdem Test Data Analytics IntraStage Client

The IntraStage server would reside on your network and pull data from the test systems (perhaps every five minutes) into a robust, centralized database. This data can be tied into ERP / MES systems and served up via web-based reports to engineers and managers who need insight into the quality of the products or the test data results.

INTRASTAGE

Mondays meeting COVERED!

Last weeks design and manufacturing test results

C A L B A Y. C O M

Best Practice #4: Centrally Control Your Test Configurations

Just as it is best to centralize test data into a robust database, test configurations should also be managed from a central location.
In a high-mix environment, you are testing different products and subassemblies all the time. Each UUT will have a set of test parameters and test steps that may change as new revisions are designed or problems are detected from the field. In a company where multiple ATEs are deployed, you would like to have all of the test configurations automatically pushed to each of the ATEs (rather than having to manually update the software on each tester). Cal-Bay proposes to leverage the same IntraStage server for the storage of test configurations. We would then train your personnel on a process for loading updated configurations into IntraStage such that they can be downloaded and used by the LabVIEW software at each FlexATE. This process is a best practice used by Cal-Bay, and it would be personalized for your unique requirements, including your specific release and validation needs. Using a central server allows for FlexATE systems located anywhere (including at suppliers or contract manufacturers) to be automatically updated with the latest test configurations. This same server will also receive the test data from those remote stations, allowing for a consolidated view of all test data and test configurations.
Client Web Manager Dashboard Client Eng 1 Web Dashboard Manager Eng 2 Dashboard Dashboard Eng 1 Dashboard Eng 2 Dashboard

Central Server IntraStage & NI Data Finder

Central Server IntraStage & NI Data Finder

Phase 5
ATE 1 ATE 2

Supplier 3 ATE

ATE 1

ATE 2

ATE 3

Client Web

Client Manager Eng 1 Eng 1 Eng 2 Eng 2 Manager Web Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard

Client Web

Client Manager Eng 1 Eng 1 Eng 2 Eng 2 Manager Web Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Central Server Central Server

RMA

R&D

Repair ATEs

Central Server Central Server IntraStage & IntraStage & NI Data Finder NI Data Finder

Client Web

Client Manager Eng NI Eng 1 Manager & IntraStage &1 NI Data R&DIntraStage DashboardFinder Data Finder Eng 2 Web ATEs Dashboard Dashboard Dashboard Dashboard Central Server Central Server IntraStage & IntraStage & NI Data Finder NI Data Finder ATE 1 ATE 1 2 ATE ATE 2 3 ATE ATE 3

Eng 2 Dashboard

ATE 1

ATE 1 2 ATE

ATE 2 3 ATE

ATE 3

ATE 1

ATE 1 2 ATE

ATE 2 3 ATE

ATE 3

Manufacturing
Client Web Client Manager Eng 1 Eng 1 Eng 2 Eng 2 Manager Field Dashboard Dashboard Web Dashboard Dashboard Dashboard Dashboard Central Server Central Server IntraStage & IntraStage & NI Data Finder NI Data Finder

Client Web

Eng 2 Client Manager Manager Client ManagerClient 1 Eng 2 Eng 1 Eng 2 Eng 1 Eng 2 Eng 1 Eng 2 Client Manager Eng 1 Eng Manager Web Dashboard Dashboard Dashboard Dashboard Dashboard Web Web Web Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Central Server Central Server Central Server Central Server Central Server Manager &1 Data NI &IntraStage & NI ClientIntraStage & IntraStage Manager Eng 1 Eng 2 Finder Eng 2 Finder Eng 2Eng 1 Eng 2 Manager Eng 1 Finder ManagerClient NI DataClient NI Data Finder Data Eng IntraStage & NI Data Finder Client IntraStageManager Eng 1

Client Web

Web Web Web Dashboard Dashboard Dashboard Dashboard Dashboard Dashboard Web Dashboard Dashboard Dashboard Dashboard Dashboard Production ATEs Dashboard Dashboard Dashboard Dashboard Central Server Central Server Central Server Central Server Central Server NI NI IntraStage & IntraStage &IntraStage & IntraStage & NI Data FinderData Finder NI Data Finder Data Finder Data Finder IntraStage & NI ATE 1 ATE 1 2 ATE ATE 1 ATE ATE 2 3ATE 1 ATE 3 ATE 2 3 1 ATE ATE 2 ATE ATE ATE 3 2 ATE 3

Eng 2

ERP
ATE 1 ATE 1 2 ATE ATE 2 3 ATE ATE 3

MES
ATE 1 ATE 1 2 ATE ATE 1 2 ATE ATE 2 ATE 1 ATE 2 3 ATEATE 3 1 ATE 3 ATE ATE ATE 2 3 ATE 3

C A L B A Y. C O M

Best Practice #5: Plan for System Maintenance

Given the rushed schedules of most projects, it is understandable that many test teams neglect proactive things like test data management, configuration management, documentation, and long-term maintenance.

The first step to proper maintenance is to document the ATE sufficiently. Because a successful ATE will be around for over five years, it is typical for system upgrades to be managed by personnel different from those who designed the ATE. Since the recipient of good documentation may be someone who doesnt even work for the company today, it is easy to cut corners; however, the organization suffers in the long-term when this planning and documentation isnt done.

Cal-Bay can work with you to create a proper Maintenance Plan, addressing issues such as: What is the comprehensive list of all system-related documentation? How to handle system failures, such as hard-drive crashes, equipment failure, or self-test diagnostic fails? How will the system scale for new UUTs, revisions, test steps, or measurement needs? What is the process for software back-ups, configuration management, and software updates? How will the system and its instruments be calibrated? How will duplicate systems be built? How will those duplicates be validated such that the measurements are reproducible across duplicates? How can the system be used by other divisions, outside suppliers or contract manufacturers?

C A L B A Y. C O M

Best Practice #6: Design Your Fixtures to Use Mass Interconnects

Electronic products and subassemblies evolve quickly. You shouldnt design your ATE assuming the set of inputs and outputs are fixed forever.
Consider the following scenarios: 1. You have to design a suite of ATEs for a new product scheduled to release in six months, but the product design is still in limbo. Your tester must be able to handle lastminute changes that come from R/D or Marketing. Your company innovates, with new product releases every year. You need a tester that will work for last years model, this years, and the future releases. Youd like to re-use the same capital equipment to test the final assembled product as well as many of the subassemblies.

2.

3.

In high-mix test environments such as these, companies should use a Mass Interconnect to allow for the re-use of capital equipment across multiple UUTs. Mass Interconnects also allow for future extensions due to their modularity and high signal count capacity. Several companies make mass interconnects, including Virginia Panels (www.vpc.com), Everett Charles (www.ectinfo.com), and Mac Panel (www.macpanel.com). Cal-Bay works with all of these vendors and would determine which to use based on your unique system requirements.

C A L B A Y. C O M

Best Practice #7: Buy Your ATE From a Single Vendor With a Single Part Number
Most companies recognize the added cost of managing and buying from multiple vendors and are therefore looking to simplify the supply chain. Luckily, in the field of high-mix functional test, Cal-Bay can offer several advantages: 1. We personalize the design of your ATE (whether a 19 rack or the FlexATE) and give it a unique part number stored and maintained in our Product Lifecycle Management (PLM) system. 2. You can then order duplicates with a single part number, where Cal-Bay: a. Handles all procurement from companies like Agilent, National Instruments, IntraStage, Dell, Virginia Panels, etc. and integrates everything for you b. Uses your specific self-test diagnostic fixture to test the FlexATE before it ships c. Ships the FlexATE on a crate to any location, ready to test UUTs out-of-the-box d. Can provide on-site support and training of the personalized system 3. Cal-Bay can also create part numbers for other needs, such as: a. Spare parts (of subcomponents) b. Self-Test Diagnostic Fixtures c. Custom Power Supplies (or other electrical designs) d. New personalities of the ATE or FlexATE

Buy your entire ATE from Cal-Bay as a turn-key system with one part number!
C A L B A Y. C O M

Additional Reference Material

About Cal-Bay Systems
Cal-Bay Systems is a recognized leader for test and measurement solutions when product quality is critical. Our global offices serve medical device and aerospace electronics manufacturers with functional test and DVT solutions for R/D and production. We serve power generation companies with Vibration Monitoring and Analysis.

Online Flash Demos (video and audio presentations):

Cal-Bay Services Overview FlexATE Overview Solid Software Design for Test IntraStage Overview 4 min 10 min 17 min 15 min http://calbay.com/flashdemos/1.html http://calbay.com/flashdemos/FlexATE.html http://calbay.com/flashdemos/sw_design.html http://calbay.com/flashdemos/intrastage.html

Articles, White Papers and Brochures:

Improving your Product Quality with a Smarter Test Software Process: http://www.calbay.com/pdf/Smarter_Test_Software_Process.pdf FlexATE Brochure: http://www.calbay.com/pdf/FlexATE%20brochure.pdf Article: Three Trends That Could Make Your Test Data a Big Problem http://zone.ni.com/devzone/cda/pub/p/id/552 The 10 Things to Consider When Choosing an Outsourced Vendor: http://labviewexpert.com/resources/whitepapers/10thingstoconsider.asp FlexATE Envelope Drawing & Specifications: http://calbay.com/pdf/11590-2001-ed_rev1a__env_dwg_flexate.pdf IntraStage Brochure: http://www.calbay.com/pdf/INTRASTAGE_Brochure.pdf Virginia Panel Mass Interconnect used with Cal-Bay FlexATE: http://www.vpc.com/products/applications/application_articles.cfm?ArticleID=5

C A L B A Y. C O M