Factory Acceptance Testing (FAT)/Vendor commentary 2
From LabAutopedia
A LabAutopedia invited article
Commentary on FAT by technology providers
Authored by: Bryan Greenway, Treis Technology
Information provided to The LabMan for the September 2009 blog.
- What is FAT testing and how is it unique in the process of developing an automated system.
What a FAT should be: a formal review/demonstration of how each specification requirement has been addressed. A handy tool for doing this is a Verification Cross Reference Matrix (VCRM). The VCRM is a simple table containing each requirement along with its associated description and method of verification. A sample table heading is given below:
| Requirement # | Type of Requirement | Specification Paragraph # | Name | Requirement Text | Verification Method (Test, demonstration, analysis, design, inspection) | Verification Comments |
The VCRM is then used as a tool early in the Spec negotiation process with the vendor, clearly outlines how “success” is defined, and how each requirement will be measured. It also provides the core for the FAT agenda; by simply stepping through each row (requirement) in the table, an organized approach is facilitated. A healthy safeguard for the buyer is to require that the VCRM be used as a reporting tool during the project…what requirements are currently met? The customer should require documentation that a complete, mock FAT has been successfully run at least twice before even scheduling that real FAT. The vendor should make sure that the customer is plugged into the development process all the way through. Engage them in regular reviews, and try not to hide anything from them. If this is done properly, the customer becomes well informed, mitigating the chance for late surprises, as well as becoming a calming source of information inside his company. Try to get customer representation to see mock FATs. Also, make sure the vendor understands the expectations and pressures on the customer side.
For vendors and customers: Expect surprises that neither anticipated. Work together to find a solution. Remember, both sides have skin in the game. Also remember that for most of automated systems, it is a uniquely function device…i.e. this is the first time it has been done. Not everything can be anticipated, so there will be discoveries along the way.
What a FAT typically turns in to:
Most FATs that I’ve been involved with uncover surprises. These “surprises” happen for a variety of reasons:
- A customer turns up with a set of unexpected conditions under which he/she wants to “see what happens”. Many times they’ve done this intentionally to “see how thoroughly the system has been tested”. Make all attempts to discuss (early in the process) the reasonable conditions under which the system will be expected to operate.
- A last minute fire drill where there are first-time attempts at required functionality during the FAT (this may seem unbelievable, but it is VERY common)
- An abbreviated FAT (abbreviated because too many requirements are failed right away) that turns into a drawn out meeting around “what do we do now?” discussion….do we adjust the requirements we’re having trouble with, give us more time to try to finish it (sometimes asking for more money), ship it anyway and then try to finish it up on site (BAD IDEA!!), or cancel the project all together.
- One side or the other taking a hard, many times emotion-driven, stance, making it very difficult for a collaborative process to move forward. Common sources of disagreement include handling of unexpected or error conditions, error recovery, and means for measuring system performance against a specification. Be sure to spend time discussing these areas well before the FAT.
- Who should attend a FAT?
For the vendor: have a very small group to lead the FAT, minimally including the sales lead, the project management lead, and the technical lead…but remember to have EVERYONE that was involved on call. You don’t want too many voice around the system during the FAT; it creates a sense of NON-CONFIDENCE. Having them on respond quickly when needed creates a sense of COMPETENCE and ORGANIZATION.
For the customer: make sure that your team consists of an automation “champion” (usually an engineer or a automation-saavy scientist), an expert on the process the system is to perform (usually a lead scientist), and an IT specialist (to manage integration of the system into the corporate net). This team should have the authority to negotiate the specification.
- In your experience, is there a single critical factor that leads to a successful FAT ?
Two things, that are very closely related:
- Early, honest, and thorough communication
- Management of expectations on both sides
- What specific categories of things should be evaluated during FAT
Not everything in the system specification should be addressed at the FAT. Many categories should have been address well before the FAT, such as electrical power requirements, required floor space, facility connections (air, water, gases, etc.), human factors, environmental factors, and shipping method. Others may not be checked until installation, such as IT integration…although this should be thoroughly discuss early in the project, it may not be demonstrated until delivery. The primary categories tested at the FAT include: functional/operational requirements, logistical requirements, environmental requirements, and safety requirements. Also, anything seen as a risk, should be tested at FAT, if possible. An incomplete checklist of requirement categories is given below (NEEDS MORE WORK!).
- What should be the product of a FAT?
Intangible product:
Confidence. Confidence that the system to be delivered meets expectations. This confidence is built from a long process consisting of several major milestones, one of which is the FAT; another being the successful installation and execution of “real” science on the system.
Tangible product:
The certification of a formal agreement. In an ideal world, this agreement would be a VCRM table with a SUCCESS checkmark for each row (requirement), and a plan for delivery and installation. More commonly, it will consist of a signed agreement detailing what passed, what failed, and the remediation plan for each failure/deficiency. If the remediation plan cannot be fully addressed at the FAT, then a deadline for presenting this plan to the customer should be set. If another FAT is required, this should be part of the remediation plan. In the worst case, the remediation plan may include how the system will be corrected on site, after installation at the customer facility.
Also, assuming all goes well, some initial training can take place at this time.
- How much of the FAT protocol should be repeated once the system is on-site (SAT)?
This is a time to be disciplined. Repeat the FAT. Stay on script. Expect surprises. Keep the group as small as possible.
Vendors:
Take proper technical support personnel on site. Hopefully you won’t have any significant problems, but better to be prepared.
Don’t get in the “driver’s seat” too much. Let the customer run the system.
Customers:
Don’t make this a big show yet. Keep the crowd small; handling any surprises with a small team will keep the proper level of information flow inside the company to a manageable level. You can have a big demo later, once you have confidence.
Perform detailed training on the system, including maintenance, trouble shooting, general do’s and don’ts.
Specification Checklist
Mechanical
- System footprint
- System weight
- Facilities requirements
- DI water (pressure, flow rate, consumption)
- Compressed air (pressure, flow rate, consumption, dryness)
- Other gases (nitrogen, oxygen, etc.)
- Facilities connection locations and disconnects
- Cable management
Electrical
- Power Requirements (Voltage(s), Current load)
Functional/Operational
- Throughput
- Functional coverage (does it perform all the functions needed?)
Logistical
- Material Flow
- Equipment access
- Robot teaching
- Equipment maintenance
- Restocking of consumables
- Access to and refilling of containers
Environmental
- Thermal impact
- Noise levels
Human Factors
- Workstation comfort
- Required bending and lifting situations
Safety
- Machine Safety
- ANSI B11.19,Performation Criteria for Safeguarding
- ANSI/ RIA R15.06, Safety Requirements for Industrial Robots and Robot systems
- ISO 14119 (EN 1088) Safety of Machinery- Interlocking Devices Associated with Guards
- Chemical and biological safety
- Laser, Radiation, and electromagnetic energy shielding
Information Technology
- Computer systems
- Network interfacing
- Software
Shipping
- Crate sizes
- Method of shipment
Installation
- Who provides the manpower for installation
- Who is responsible for disposing of shipping/packing materials?
- Considering all of the equipment sizes, check the access to each room. Walk the path from the loading dock to the lab.
Intangibles
- Management of complexity and customization. KEEP IT SIMPLE.
- Quality components/subsystems with an available network of support and parts
- Risk management
- Use of mature technology
- Identify components that must operate at or near their performance limits
Support Contract
Preventive Maintenance
| Click [+] for other articles on | The Market Place for Laboratory Automation | The Market Place |
