Software Vendor Assessments Part 3: Conducting the Assessment

January 28, 2014 by

What to Assess?

This breakdown targets the essential factors that require attention when conducting vendor assessments for software vendors producing systems designed to work within a regulated environment.

The Facility Tour

Whether the vendor occupies an entire building or one floor in an office building, getting a quick tour of the premises tells you a lot about how the vendor operates.  Always make the data centre a part of your tour to see how data and documentation are being secured and stored.  Be sure to observe cleanliness and habitation practices as well, as the overall look, feel, and organizational culture will be reflective of how well they can deliver on what what they say they do.

The Quality Manual

Earlier we discussed that the purpose of the vendor audit is to ensure quality is built into the product.  There is no better way to start measuring this than by looking into what the vendor perceives as quality.  That is where the quality manual comes in. This can go by many names such as quality plan, quality objectives, quality directives, etc.  It is basically a document that outlines the vendor’s dedication to quality and outlines at a high level, all the actions and processes they have in place  to ensure they maintain an adequate level of quality within their software products.   A good idea is brush up on your company’s own quality manual and some other ones found on the web to familiarize yourself with the contents and key elements required.  This will give you an idea of what to look for when you are assessing the vendor.  At minimum, the quality manual should contain high level directives, and define overall company quality objectives, standards, SDLC, Change Control, Defect Management, Support, Data control, Security, Training, and Roles and Responsibilities.

Standard Operating Procedures (SOPs), Policies & Instructions

Defining SOP’s to govern processes is a necessary and practical approach to deliver accountability to employees and ensure consistency in execution.  A vendor with non-existent, or improperly defined, SOPs can be somewhat of a wild card. They will  generally have a lot less control over their processes and product quality because they have no clear definition of what the standard is, or what methodology employees should follow to arrive at the same results.  The SOPs required by vendors will vary by software and requirements, but at minimum, the following should be in place:

1.       Computer, Documentation & Facility Security

Are all computers for employees restricted for access? Is facility access controlled? If not, are documentation and sever/data rooms controlled for access?

2.       Backup, Restore, and Disaster Recovery

Does the company conduct regular (nightly) backups, and have restore procedures in place? Are there UPS and failover measures in place? Are backups preserved offsite? Is there a disaster recovery plan?

3.       Change Control

How does the vendor manage released software? Do they provide detailed impact assessments and properly documented release notes for changes. How are patches managed? What documentation is provided?  How are the changes deployed?

4.       Defect Tracking & Management

How does the vendor track and manage defects/bugs reported in the system during and after development? Is there a clear cycle of delegation and approval that ensures proper analysis and testing?  How do the bug fixes make their way into a release?

5.       Version Control

Is code properly version controlled during development to avoid the wrong or outdated code being deployed to clients when new patches are released or hotfixes are deployed?

6.       Support System

Does the vendor properly track, report, mitigate and respond promptly to support calls received by clients?

7.       21 CFR Part 11

If the systems store data as electronic records and/or employ electronic signatures, do they comply with 21 CFR Part 11 requirements? Is there an interpretation document that shows adherence to the ruling?

8.       SDLC

Is there a document Software Development Life Cycle methodology being used at the company? If so, are the methodologies, phases, testing and documentation requirements described clearly and followed?

9.       Source Code

Does the vendor have standards in place to govern code writing during software development? Are source code reviews conducted periodically to ensure that the established quality coding standards are followed?

These are just some of the critical elements to review when assessing your vendor. This list might grow or be more rigorous depending once again on the type of software you are having built or are purchasing from the vendor.  It is essential to put all this into a checklist and take it with you to write down your findings against each item.  Make sure to put down references to documentation as well.  This checklist and documentation will then go on file at your own location as documented evidence that you did your due diligence when auditing your vendor as per your own company standards and policies.

 Stay tuned for the last post in this blog series that will contain a wrap up on the discussion including how to handle subcontractors of your vendors, scoring a vendor audit, and following up on non-conformities.

Did you miss the first posts in the series? Read Part 1 here and Part 2 here.


Leave a Reply