The Right Processes to Select Great ICT Systems
Sourced from a white paper co-authored by Daniel Muggeridge and Cameron Bedford, as a parallel effort to the presentation Daniel delivered to the Australian Community Philanthropy annual conference in September 2014.
Selecting a system for your key organisational area is a significant decision, and should not be taken lightly – be it for customers, clients, donors or contacts.
Appropriate information management systems are the “arteries” of a healthy and humming organisation. These systems connect your people, processes and activities to the heart of the organisation. Get it right, and you’ll set your team up for success. Let the arteries grow clogged (let poor information management systems persist) and every task and exercise becomes an inefficient and difficult effort. Growth and vigour are almost impossible without good systems.
Why should organisations pursue this? Not-for-profits and smaller organisations are always resource- and time-poor. However they are often better placed to trade time than money. The opportunity cost of wasted effort isn’t always easily visible, but the team are likely already working extended hours, and volunteers often have a high learning curve to come up to speed. Appropriate systems will reduce manual or error-prone effort and will become a reference point for powerful and accurate information. With good supporting processes this can help realise a significant reduction in people’s time and effort – which can then be redirected to higher value tasks more often. In short, your whole organisation will become more effective.
A thorough focus on a few key aspects, if executed at least moderately well, will deliver a superior outcome to just “good intent” – even when good intent is executed exceptionally. A strong foundation will be prepared with attention to:
- Sufficient process and appropriate specialist partner assistance
- Sufficient documentation of needs (requirements)
- Strong Vendor Request for Proposal process
- Who decides?
A good process is a proven method to achieve better results. Where a team has a skill gap – often in skills knowledge or applied tacit experience – a process can be a great compensator.
It is fantastic if your team has relevant skills, knowledge and experience to draw on for selecting IT systems. It is definitely worthwhile leveraging this, but do be aware of the opportunity cost of that resource not doing their normal job while contributing to the selection activities. It is also necessary to watch for a common trap that subject matter expertise in your organisation’s business needs does not adequately compensate for a lack of expertise in the requirements and tender processes.
If your team lacks these capabilities, we recommend engaging contract or consulting help. It might cost more in the short term, but will help deliver a cheaper and better outcome overall.
There are a myriad of supporting tools that will help in documenting needs and tracking progress. We have found that a spreadsheet or structured Word document can go a long way.
Knowing your key constraints will be important; including schedule, your team’s availability to work on the project, and how much money is available – not only to purchase and implement a product, but also to run the product selection activity and to maintain an effective solution after go-live.
Sufficient Documentation of Needs (Requirements)
A problem well stated is a problem half solved – and this applies to business problems and product selection as well as to anything else.
A requirements document provide vendors all they need to know about your needs of an IT system, and should contain:
- General context about your organisation
- Process diagrams detailing information flow and decision-points in general operations
- Specific, numbered, prioritised lists of short, actionable statements that together describe what your organisation needs from a new IT system
Documenting requirements can be done in any variant you like so long as it captures:
- who the user is in each requirements scenario
- the key elements each user need to be enabled to do – these must be statements of need, not statements of solution – no technologies or systems should be mentioned here
- anything the system does in the background when the user does the action
- the measures of proof that it worked – really important because these become the test acceptance criteria
- Writing acceptance criteria is a bit of a skill. It moves from statement like, “the user can change the status” to something slightly more specific, “authorised users can change the status to accept or reject”. In this there are test conditions about security, and test conditions about the functionality.
When capturing requirements and defining a scope, start with high ideals and aspirations – what do you ideally want the system to do, in the absence of implementation constraints. Continue the process with pragmatism to trade-off the must haves from the nice to haves; and finish with resolve to ensure completeness of scope and user flows. The 80/20 rule is a good one to follow in balancing the level of detail but the scope of the problem needs to be sufficiently defined and documented.
Documenting your needs well ensures there is a shared understanding and effective communication, and:
- Facilitates choice of best-fit system (the most obvious outcome)
- Ensures selected option is implemented optimally by having a clear definition of what that successful implementation will look like.
- Presents an excellent opportunity to review organisational process and priorities and corresponding alignment to the organisation’s strategic objectives and measures.
How to document:
It is worth reading the Curse of Knowledge tapping demo, which cleverly emphasises the scale of misperceptions when a communicator doesn’t fully appreciate the size of the gap between their own and their audience’s context on the topic at hand. The demo plays this out with the simple device of one volunteer tapping the rhythm to a song and the audience guessing the result. The difference between the expected and actual success rate of guessing the tapped song is typically quite astounding; and informative for our work in documenting requirements! The point of this demo can be powerfully applied to demonstrate the need and the difficulty of providing sufficient context and detail for vendors to respond well to RFP requirements.
Your requirements will not all be the same priority; and sometimes robust discussion is necessary to sort this through. Requirements should be tagged as “critical”, “important”, “nice to have”, or a similar three-category system. There’s always pressure to say “all requirements are critical”, however that becomes meaningless and unhelpful. Resist the urge and ensure that no more than 30% of your requirements are marked “critical”, and aim at 40-50% in the “important” category.
As you work these details through, always capture specifics rather than generalities, even as examples. More detail is almost always better than less. There is eventually a diminishing return and wasted effort on refining requirements for greater detailed; but a far more likely risk is that authors imagine the detail to be “thorough and complete” and yet potential vendors will respond with a long list of questions they consider the requirements have glossed over. When researching and writing organisational requirements, always ask “why?”… and then “why?” again to help break up one statement into more detailed component statements.
Strong Vendor Request for Proposal process
You may be lucky enough to have sufficient skills and time in house to run a Request for Proposal (RFP) – great, you can skip this section. However if you’re one of the many organisations wrestling with how to deliver change, then you are likely to need external help to lead the vendor selection process and subsequent system implementation.
To this end, an RFP can be a very helpful activity to select a partner based on understanding clearly what you know that you know, and also being clear about what you know you don’t know.
We encourage a wide enough net be cast to a set of defined questions. This quickly helps you work out who is probably going to do a good job and who definitely won’t; it is better to learn this early and without significant effort.
How to run an RFP:
- Define and publish selection criteria.
- Issue questions with a response template – so it’s easier for you to process the responses.
- Publish a date by which vendors must provide responses.
- Score responses based on how well they meet you criteria.
- Scoring need not be an art form, but a defined set of “fully meets”; “partially meets and can uplift”, “partially meets”, “doesn’t give sufficient confidence”, “not answered / no way” each with an appropriate numerical score, for which a spreadsheet will be very helpful.
- Each assessor should score independently, and in absence of knowing the proposal’s dollar costs.
- Each assessor’s scores should be aggregated with an average but it is also important to consider degree of variation in the scores – we like to use a stock graph type which allows for three values at each point, a low, average and high. For larger groups it is also helpful to understand the most common response and the overall number of responses.
- A short-list of vendors can then be asked to provide a tailored demonstration of their offering. This adds life to the process and helps really understand a vendor’s offering. Seeing products in action often provides far greater insight and more mature recommendations far more than does a static RFP response document.
- Definitely conduct detailed discussions with vendor referees, including a selection of referees found through personal networking rather than offered by the vendor, if possible. The vendor will of course have provided only clients with happy implementation stories. It is most helpful to also talk to companies who have moved away from the vendors being considered, as that is more likely to uncover challenges and issues important to your own evaluation (particularly in understanding some of the unexpected constraints and short comings).
- In addition costings should be factored in and ensuring “apples to apples“ for both project costs and operational ongoing costs, then rolled up into a recommendation for the decision makers.
Expectation management is nearly everything (think the metaphor of innovation is 99% inspiration and 1% perspiration); this is likely to be a difficult journey and ensuring that people know what is happening, are kept abreast of updates, proactively resolve issues and avoid sticking their heads in the sand so that the “elephants in the room” are acknowledged and address is more than important; the success of the project will depend on it.
Good process is a worthy servant and a terrible master. It is critical to have and to follow a well thought-out process for strategic tasks such as IT system selection. The process provides a map and compass for navigating an often-difficult journey. As with any journey, though, it is best travelled with an open eye for surprises around each corner – and a willingness to adapt where there is value outside the strict limits of agreed plans.
Use the cloud – or have good reason not to. With the commoditisation of cloud-based services, unless you have data sovereignty, PCI DSS, privacy laws or some other compliance driving a different approach, your default position should be to leverage a cloud provider. Even then, with care these can be managed as risks or avoided. Be they Amazon, Google, Microsoft, RackSpace or a smaller play, the automation and control they give is often more than sufficient.
There is no single one-size-fits all answer for selecting IT systems. Your organisation’s unique focus should have a strong influence on the choice and implementation of systems and if in doubt, leverage the expertise of a trusted provider. Having said that, your needs are not unique, and there will be plenty of other organisations who share your basic needs and are in this same space. It is likely there is something out there that will do 80% of what you need relatively easily. The 20% of differentiation from competitors (or collaborators) is the hard bit to reach. Moral of the story here is don’t re-invent the wheel!
Determining who decides is a strategic decision itself, and must have relevant stakeholder buy-in (Board, Senior Management Team etc) with minuted approval.
The recommendation should also have been socialised with the influencers and specifically the leader. Basically the recommendation should have the support of the CEO else it’s doomed to fail and shouldn’t be tabled.
If there are others involved who will have accountability for the working system once installed, their undying support of the leadership team decision as the member ACCOUNTABLE for its implementation is also paramount. Absence of this support is a deal-breaker as they will not fight for it when in trouble and it will be perceived by them as an annoyance.
It is vital that a decision is made, and reaching that decision can require a concerted effort to push through the risk of analysis paralysis. Failure to make a timely decision can be more costly than making a decision with adequate but less-than-perfect information. A key technique to avoid this analysis paralysis is to explicitly validate that any time-consuming questions of your vendors actually have a real claim on potentially changing your final vendor selection. If it can’t change your choice of vendor, then take the question offline and answer it after the process completes.
A thorough BRD and successful RFP is just the beginning; next your organisation will need:
- to assess the implication on the business processes, staff training and current technical systems in the form of a design for the solution;
- to manage the implementation effectively;
- involve an end user representative or working group in validating the design, but also the system in addition to your functional testing;
- to manage stakeholder expectations and change journey exceptionally;
- and establish AND maintain a healthy vendor relationship
Not-quite final note
It is expensive to short-circuit due process or attempt in-house without appropriately-experienced guidance and leadership. There are multiple examples of this in the industry and in practise but a project done poorly (despite the best intent) will cost more (tangible and intangible) and take longer than a project executed well with skilled resources and rigorous focus on the goal.
Good carpenters know to measure twice and cut once, to avoid mistakes and waste.
Get help; this might even just be in the form of a “health check” to ensure that you know what you know, and have your eyes opened to some problems and challenges you didn’t know. You may also benefit from help to sell the message that you need to invest in infrastructure which may appear to be a black art, but is really very structured and logical when broken down in a systematic way, to have an emphasis on what organisational services and outcomes are enabled or hindered.
What if I’m too small?
Please don’t think that 100% of your funding has to go out the door just because you’re small. Processes can be scaled up and down and detail can be scaled up and down depending on the risk being managed. There is also value in establishing infrastructure and application capabilities given the right mix of needs. This will however introduce a need for operational support ongoing that must be managed by skilled staff or a support partner.
Where to go for more information on your vendors of choice?
- Vendors will provide a reasonable amount of information on their websites – but remember they’re trying to flog you something
- Some organisations provide not-for-profit pricing which is significantly cheaper than their normal price, so if your potential vendors supply for-profit organisations then it is well worth asking pricing questions
- Find a comparison review of products if you can – but beware the need to share an email or phone number to get the “free” report – they will ring you!
- Talk to similar organisations in your network
To re-iterate the key issues to harness your organisation’s innate value:
- Get your process right
- Document your requirements
- Vendor Request for Proposal
- Who decides is a strategic decision
- Remember your minimum factor defines capacity
- It is expensive to short-cut process. Carpenters measure twice and cut once