Providing Affordable IT Management to SME's How do I evaluate service changes where a test or development environment does not exist?
The simple answer is "I cant" and therefore the appropriate response should be to rename this subject "How do I create a test or development environment as cheaply as possible".
Implementing any change, large or small without testing it first carries a level of risk. Most likely we all do it and have taken a calculated decision not to test certain changes. Virus definitions and Microsoft updates are probably two of the most common changes that get deployed to private users, SME's and larger corporate's where a level of risk management has been applied to the testing and evaluation phase. In most cases these work fine, applying in the background and never bothering the user but in very rare occasions they can have devastating effects. In a later chapter of this topic I will try and give a view point on this but now back to the original question.
As I said, my initial response to this question was that you cant test changes without a test of development environment but in all honesty you can. Option 1 is always to test on live. Risky but still an option. In this topic I would also like to propose two other others. The second is "ring fencing" a small part of your live estate as UAT (or user acceptance testing), finally considering the third option which is how a live estate with no Dev or Test could be modified to accommodate one.
For the basis of this blog, I am going to have in the back of my mind two types of change. The first is a PC based application that resides on the client (PC) only and the second is a server based application that the clients connect to.
So our first option considers a test direct on live! Risky, yes! and if you are carrying out some form of Change Management, it is likely to be scrutinised quite strongly. The key thing about deploying onto live is awareness and planning. In reality, you have got to plan for the worse (that is something goes wrong and you are going to have to work through a recovery plan) so picking a time to do the change that does not impact your business is critical. The trade off here is if you do it out of hours, you will have limited support people available (even contracted support or local PC support companies) but doing it in hours impacts the efficiency of your business. You will probably need to consider moving any key critical tasks forward (eg payroll runs, supplier orders etc) to create a larger "change window", also have in the back of your mind your VBF's or "Vital Business Functions". These are the tasks you MUST performance to keep your business operational even if you had limited system resources available. The evaluation phase of this approach is very much about risk management and contingency planning and very little about the technical aspects of the change. You may also want to use a popular search engine to research the change (especially around common changes such as Microsoft patches etc) to see if any articles exist around know problems. This sort of approach does sit nicely with both of our scenarios above (the PC and the server change) but once again I have got to close this paragraph by emphasising that this is more about risk management than testing!
So our second option was around creating a UAT group and testing the change on them first. Out of the three options, this is probably the most common approach (certainly around desktop changes) and it is fairly straight forward to carryout. Obviously one assumption is that all of the PC's in your business are the same, running the same OS and AV and have a very similar application stack. If this is not the case then a restricted UAT group will not really give you much benefit. It will certainly give you an indication but if the estates is fairly diverse you are building risk into the change as each PC and build MAY responded differently to the change. Once you have your UAT group you need to find a way of applying the change across a sub-set and not to the whole estate. This could be as simple as follows:
For Microsoft changes set your UAT group to download and apply automatically and your no UAT group get the updates to apply manually on your request. For AV definitions, get the UAT group to obtains definitions on a daily basis but the rest of the estate to only apply on a Friday. Obviously in both cases you would only apply to your live group once you are happy that the UAT group have been OK.
The third option considered creating a UAT and test environment and for this I would introduce the possibility of imaging software or Raid disk configurations. Basically you need to get you estate (whether that is a PC or a server) into a situation whereby it can co-exist as a live use PC/Server or a test.dev piece of hardware. In either case you need to restore the hardware to its pre-change state once it has been used as a test/dev machine. RAID disk configurations should allow you to create hot swappable hard drives that in effect "rebuild" themselves. Prior to the change being made you you remove a disk and allow the RAID to rebuild a spare set of disks. These are then removed and put to one side. Once the testing has been completed, the RADI disks are re-inserted and the hardware returns to its pre change state. Disk (or hard drive) imaging is a different approach whereby an "image" of the disk is taken on a secondary storage media (whether that is a recovery partition on the hard drive or onto a DVD. In this case, once the change is completed the hardware is restored from the image.
This topic is not aimed at giving you every type of option available, but more about introducing the concept of dev and test environments and giving a few pointers about how to move away from a "we don't have one" mentality into one that fits with you business model
2nd March 2010
Click here for details of our FREE business healthcheck and join the rest of the companies using IT Service Management.