It’s common practice for IT teams to use a reference list of NFRs (non-functional requirements) as a quality assurance gate to ensure systems are healthy when they are released into production. They form a checklist of operational expectations that must be ticked off at the service transition phase to confirm the system is (i) easy to install, maintain, and troubleshoot; (ii) it will perform and is scalable; and (iii) it is secure, reliable and robust.
Injecting that list of NFRs into the design phase of your project will ensure early consideration of those operational characteristics, making it easier to build them into your solution from the very outset. It will be a much better product as a result.
But what do you do about those NFRs when buying packaged software solutions from vendors? Do you include them in your vendor selection and evaluation process?
I suspect most people research package solutions entirely based on the business process functionality and pay little attention to the non-functional requirements. It’s very easy to assume the vendor will have covered all those operational aspects, especially when you’re buying a cloud-hosted SaaS solution.
But that isn’t always a safe assumption. I’ve seen all sorts of operational overheads come about because of package software implementations. So much so, I drew up a list of package selection criteria and encouraged colleagues to include them in their requirements when evaluating potential suppliers.
Here are a few examples of the issues that can cause you pain.
Operating system compatibility
Is the vendor truly committed to maintaining their product so that it is compatible with new versions of the underlying operating system as they become available?
If not, you are heading towards a deadly embrace when the system software goes out of support. If you don’t upgrade the operating system (whether it’s Windows, Unix, or whatever) you’re exposed to far greater risk of system failure and malicious attack. If you do upgrade, the application vendor won’t support you because they haven’t certified their product on the new level of operating system.
So – stay and be damned because you’re not supported by one party. Or upgrade and be damned because you’re not supported by the other party.
Remember, this was a factor in the NHS ransomware incident in 2018. Specialist medical equipment being controlled by old versions of software running on old versions of Windows.
The vendor may offer their software on multiple platforms, but they will have a preferred platform – the one they do their primary development on. If you’re on one of their lesser platforms, you may not be in a healthy position.
I remember we insisted a vendor port their application onto IBM’s AIX software and p-series hardware, because that was our strategic Unix platform. However, the vendor’s primary development platform was Sun Solaris (as it was at the time). They had loads of customers on that OS, and only us on IBM’s AIX. Their functional upgrades and fixes were always available on Solaris first and it would take ages for those functions to make it into the AIX version. I was working for a large corporate and we clearly thought our size would force the vendor’s hand, but we were wrong.
When it comes to upgrading a package version, the vendor will provide scripts to install the new release and make alterations to the database schema. Don’t assume they will all work perfectly. Some are very poorly constructed and haven’t been through thorough testing. If you don’t test them out yourself ahead of your implementation date, you could be heading for a car crash. A change that runs on well past your allocated window because the scripts are so slow. Or, worse still, the outcome could be a corrupted database.
Performance and scalability
Is the vendor able to good quality performance and capacity information to demonstrate how the application behaves at high transaction load? It’s not always easy for you to test and it can be difficult to get out of a performance/capacity hole quickly. If you run into a logical scalability issue, you can’t solve it by throwing money and hardware at it.
Operability and troubleshooting
If you run into a problem, the last thing you want is error messages that tell you nothing and logs that contain precious little symptom information. How easy is it to identify specific workload and functional behaviour inside the application? I remember running a task force to get to the bottom of a very curious problem with a vendor application, gathering and analysing data to find that one particular user function was causing all other transactions to stall.
These things will not be visible on day 1. They will creep up on you and inflict pain when you least expect it. And it could take some time to resolve. So, my recommendation is make sure you include the non-functional requirements in your selection and evaluation criteria.
If you want help with non-functional requirements, here are three things you can do:
1. Get in touch and let’s have a discussion.
2. Buy a copy of Avoiding IT Meltdown which contains lots plenty of practical hints and tips about managing your technology estate
3. Ask us to undertake a friendly audit of one or more of your IT operational capabilities
You can contact us at:
firstname.lastname@example.org 07470 002019
email@example.com 0114 398 4344