This blog is inspired by Dave Briggs blog Buying new software won’t save you money and the resultant twitter conversation regarding case studies and their merits/value. For me it also relates back to Dave’s earlier blog There are no digital silver bullets . The twitter conversation was largely about most case studies not being worth the paper they are written on and savings being stated being highly dubious and of unclear provenance. There was a question stated a number of times around why do suppliers do that?
I am quite unique and lucky to have experienced this from both sides of the equation and now from a kind of middle ground with a foot in each camp. The reason suppliers create these case studies is because almost always this is one of the first things we are asked for i.e. a case study demonstrating what someone has saved directly by using the software. I completely agree with all that Dave says about this normally being done to early, with benefits not realised and actually there being a lot more complexity to delivering savings. I would also say there have been an awful lot of projects in Local Government over the last 10 years where a project is run to deliver savings from technology change and actually the results may be that savings are achieved (headcount reduced) but this has little or nothing to do with the technology change (even though it may be reported as such). To change the supplier landscape and the incredibly slow movement from legacy to new technology we need to look at the way we procure solutions, manage them and fail faster.
I would also say there have been an awful lot of projects in Local Government over the last 10 years where a project is run to deliver savings from technology change and actually the results may be that savings are achieved (headcount reduced) but this has little or nothing to do with the technology change (even though it may be reported as such)
We are lucky that for our solution full procurements are not normally required however we do regularly get asked for the killer case studies with the silver bullet business case. We are lucky that our customers will happily talk to prospective customers and discuss the benefits our software brings and there is a simple business case built in our software (i.e. the software you should all be using to improve your services and identify benefits of change!).
Lets turn the Titanic!
It is a huge frustration of mine how long legacy solutions continue in the marketplace and also how slow we are in adopting new technology(across the majority of LA’s), this is fundamentally rooted in the way we buy and deliver solutions. There are a number of factors affecting successful procurement(here are a few) and from my background firstly I must say the following
- Requirements/Problem definition. I have worked on many digital (silver bullet) projects and also with Councils where these projects are being executed understanding the detailed requirements or the problems the solution is to solve is done after procurement. Projects where the answer has been to write a strategy and buy a digital tool to solve most of our problems. These projects normally take a while to get the strategy through, business case sign off, budget identified and signed off, procurement done (maybe multiple times) and then agreement to spend. Finally after many months and even years a new ‘solution’ arrives and the hard work starts throughout the whole of this purchasing process normally expectations of the organisation of this new solution are being amplified either directly or indirectly by a number of project stakeholders. Now we identify early adopters and get into the hard work of understanding the problem and testing potential solutions which is almost always the point where an organisation realises that that the solution purchased is not delivering the solutions required. I say an ‘organisation realises’ but this knowledge is not always widely shared due in no small part to cognitive dissonance related to the new solution procured(more on this later).
- Business Case. The business case for these solutions is often at a high level and broadly equates to enough benefits to justify the expenditure normally from a reduction of customer facing staff or a combination of that and reduction of costs through existing software removal. This type of business case is often reported at a high level within a Council. These reported savings can make a great source of potentially dubious case study figures if reported in public meetings. If requirements and problem definition is done well you should be able to justify a business case with a far higher level of confidence.
- Beware of ICT confirmation bias. There are many different models of ICT delivery across LA’s and ICT provide a number of different roles and responsibilities when it comes to new solutions. I always believe that ICT should not lead a procurement of digital solutions as they do not normally understand the outcomes required by the business (as detailed before this could be true of other departments involved in the procurement). ICT provide a vital role in any procurement of solutions. I have experienced many times ICT confirmation bias. The best example of this is with traditional Revenues and Benefits portals which traditionally were very hard to implement and have very low take plus a lack of customer or business value being derived from the solutions. Due to the time and complexity of implementation of these solution ICT can have a very rose tinted view of their value. Many a time new solutions are discussed to deliver value and ICT can try to lead you back to delivering/procuring another module of the existing suppliers portfolio even though there is experience of existing and past poor outcomes. There is elements of optimism bias coming in here as well that despite evidence of performance to the contrary with existing solutions this module will be fine.
- Silo nature of LA’s, no value stream owner. Due to the silo’ed nature of Local Authorities there is often no owner for your digital end to end value streams. Often the projects are run by one service and the value streams cross multiple business units for each digital service. This causes friction and issues when it comes to change and also challenges where solutions are bought and multiple stakeholders would prefer different options. Addressing these issues can often be the cause of raised expectations of the potential of the solution as a heavy selling approach has to be followed to gain stakeholder buy in.
- ‘Legacy’ procurement approaches. There are many facets to this element and almost it could justify a blog of its own.
- I have many examples of traditional waterfall procurements with large RFIs, long lead in times, longer contracts and large values. These types of contract favour traditional ICT suppliers with large bid teams and extensive examples of solutions already delivered. These solutions also negatively affect your ability to be agile and adapt to changes in requirements and capabilities over time.
- No supplier engagement days. There seems to be a reticence in LA’s and LA procurement teams generally to have supplier days with some notable exceptions. Supplier days obviously have to be run fairly to ensure a transparent procurement process which is not subject to challenge but a supplier day provides 2 valuable things firstly your ability to explain clearly your objectives and outcomes for this procurement and secondly your ability to see more of the products in question and their fit.
- Make sure your existing customer visits are done and that you have a clear understanding of the requirements/problems you want to solve with the solutions. Ask the existing customers about how it fits your organisation, the resources, time and skills requirements what it does well and what it doesn’t. Don’t just take supplier referred customers look around and find others or get a long list from suppliers.
- No real test of solutions as part of procurement. Again as above procurement needs to be fair and transparent. Define an alpha for suppliers to build based on requirements you have. Score this as part of the assessment of suppliers and spend some time going through this and even amending it with the suppliers during the assessment. Many procurements I have seen and been involved in allow suppliers to attend and do their tried and tested sales demo happy path. Take them off piste and really try to understand if the product fits your requirements suppliers who do not want to do this probably cannot deliver your requirements.
- Failure to procure for an agile approach. Agile is undoubtedly the best way to deliver digital solutions, procure for agile project delivery but also plan for scale. By this I mean most suppliers will charge a relatively larger licence for smaller implementations but equally will happily provide a scale price as part of that.
- Failure to use existing frameworks such as GCloud and Digital Outcomes. These frameworks are used at scale by a small number of LA’s but broadly LA procurement teams can be sceptical of these approaches to procurement (Turkeys voting for Christmas springs to mind). These frameworks deliver rapid procurement methods supportive of both agile and waterfall delivery methods. Have a look and encourage procurement colleagues to give it a try.
- From an SME perspective try to create realistic timelines for your procurements. Tenders I have been involved in almost always state things like must be able to attend to present on x date and must be ready to start with x people on site on y date (always way too early not allowing any time for slippage). Resource is not normally sitting around at short notice and on 90%+ 1 or more of those dates slip. This results in lost income and also will be a reason for a supplier to no bid.
- Finally be careful what you put in your procurement documents as must have criteria. Digital Marketplace procurements seem to be more susceptible to this factor. We do work in partnership with other SME to bid for tenders and many of the tender documents I get involved in have criteria in them that either directly or indirectly adversely affect SMEs or innovative new products. I would like to see the same sort of scrutiny that goes into job descriptions being considered for procurement documents. A couple of recent examples include ‘must be willing to employ and train (LA area) apprentices’ and ‘must be able to provide 5 references where a similar project has been undertaken’ both of these I am sure have positive motives behind them but both have caused no bids due to the unintended consequences of the statements. Have a look at what you have put and consider it from a supplier perspective i.e. can a non local supplier sign up to train an apprentice in the LA area when a 3-4 month piece of work only is being procured? More people continuing to procure ‘legacy’ not ‘innovative’ solutions slows the progress of the marketplace causing issues for customers and suppliers.
Ok we have bought a solution its all plain sailing from here surely?
The final couple of challenges we have left and these contribute massively to the pace of change and innovation in the sector.
- Permission to fail, controlled failure or fail fast. Legacy approaches to procuring solutions make this harder than if following agile delivery methods. We need to create a learning culture where the perception of failure is changed. Failure is a negative word but the reality of the fail fast approach of agile is that instead of ‘failing’ on a 3 year £500k project you ‘fail fast’ and learn from the experience in the early discovery or alpha phase meaning that not only have you spent less money, but you have save huge amounts of time and have learnt and are able to apply that in the next solution. This is a new mindset but vital to delivering successful change. I have worked on many failing ICT Digital initiatives and predominantly these continue even when evidence and acquired knowledge clearly show a different approach would be more optimal. Continuing these projects when evidence suggests you should not also impacts on the pace of change and innovation in the sector. More people using legacy longer = less drive and funding to deliver solutions in the marketplace that deliver the value we (LA’s) want.
- Similar to the above is cognitive dissonance. With these large legacy procurements and the time and effort for the lead officers to make them happen it can make it very difficult for the lead officers to see any flaws in the approach. Think Tony Blair and the War in Iraq, no matter how much evidence to the contrary the resolve of Tony that the war was correct will never change. It is a major issue currently as many officers have gone out on a limb to back digital and have too much skin in the game to change their view. This also can be a cause of some of the really out there case study savings where they double down to try to continue to keep up the smoke screen. I have rarely seen this resolved generally it is resolvable if the lead moves on and fresh eyes can be applied to the approach. The fact this cognitive dissonance happens is at least in part due to the legacy waterfall procurements of solutions so this would improve with time and a change of approach.
Top Tips for actionable improvements
So how could we do this better? (these are my views and I welcome your input)
- Stop looking for silver bullet case studies/solutions (actually try to help suppliers by supporting factual case studies and helping get them through your coms teams). Remember it is unlikely that you are in the same position as any other LA so their savings do not necessarily represent your potential ones.
- Use agile techniques to manage, procure, implement and iterate solutions
- Build a multi disciplinary team to implement new services
- Don’t forget ongoing iteration beyond the project resources will need to be in place post project to deliver continuous improvement
- Use common standards and GDS mature solutions by default
- Understand your requirements before buying solutions
- Create realistic business cases and timelines
- Fail fast and learn and share this knowledge with wider LA community
- Double check the way you articulate your requirements are you excluding people who may add value
- Have supplier days
- Test your potential suppliers and take them off piste
- Buy small and plan to scale
- Fail fast and learn