Last week at Applied Trust we held our first annual “Preventative Maintenance Day” (PMD), aimed at preempting software and hardware failures and performing those little tasks that are important, but that are often placed on the back burner in favor of more pressing issues.
To minimize operational impact, the business day was bordered by two planned downtime periods in which the tasks requiring that systems be offline would occur, in addition to several other maintenance periods that didn’t require infrastructure outages during business hours. Each maintenance period had at least one small team consisting of several people assigned to perform -planned tasks.
I had the pleasure of being on a 5.30am shift, and prepared our generator for annual maintenance, primarily by removing the enclosure panels that protect the generator from curious alley-stumblers. Note to self: bring some gloves next time. My PMD team also established new logs for generator, HVAC, and UPS maintenance going forward, to ensure that we have ongoing records of all the monthly, quarterly, and annual tasks that should be performed.
One might ask how it would be possible to find so many tasks that it seemed necessary to deem it necessary to have a large portion of a day dedicated to it. A little thought, however, opens a huge array of possibilities; here is a selection from our master PM plan:
- UPS battery replacement
- Cabling clean-up and labeling
- Patch application
- System power-down/power-up
- Water-detection sensor installation
- Workstation patch compliance and AV verification
- First-aid kit replenishment
- Fridge cleanup
This is by no means a complete list, but it illustrates the breadth of tasks that were performed. It’s also important to note that it was not just pure IT infrastructure maintenance, but maintenance on all things office related as well.
Looking back, our first PMD was a huge success, and really was invaluable. We even encountered problems booting one server due to some diagnostic tools that had recently been installed that were much easier to resolve in a scheduled outage window with multiple staff on-site, than at some random hour of the day, or perhaps night, when a poor soul is paged to address an unplanned outage. When was the last time you restarted your systems for maintenance-related activities (hopefully relatively recently if you’re following a defined patch management plan)? Are you confident they would start up again if you restarted them tomorrow?
My advice to those reading this post: plan your first annual Preventative Maintenance Day! If you are already doing this, then kudos to you. If not, then preempt a future headache and 2am emergency pages by getting one scheduled. Here are some tips to point you in the right direction.
- Plan – Designate a PMD lead (a person or small group) responsible for task approval, scheduling, and delegation. This person or group should create the master PMD schedule, and maintain overall ownership of PMD.
- Assess – Aggregate a collection of known “back burner” issues as well as pressing maintenance issues. Seek input from other staff with responsibility for systems, services, and facilities, and finalize a comprehensive list of tasks. Prioritize these tasks – there’s a good chance you’ll be surprised how many things make the list. Avoid the temptation to try to accomplish too much. Also, be sure to consider dependencies between systems, and not overlap activities that require systems or devices in a period where those systems or devices will be offline.
- Delegate – One person can’t do everything! Our PMD utilized small teams, usually 3 people, defined by the PMD lead. Each team also had a lead, responsible for task review, assignment and acquisition of materials/parts necessary. Based on the assessment phase, tasks should be broken up among teams, with those tasks requiring outages being assigned to non-business hour time slots. Consider rating tasks for teams in terms of objectives, e.g., primary, secondary, and tertiary. This ensures that critical tasks are tackled first, and other tasks can be accomplished if time permits.
- Execute – Make PMD an effective yet fun event. There’s no shame in one task being “acquire tasty donuts, bagels, coffee, and juice.”
- Document – Make sure teams record what they did, so the master task list can be updated, and what didn’t get done can be noted. These other tasks can then be scheduled for some later date. Capturing that data also gives a good indication of whether task assignment was overly aggressive or not.
Again, the value of this event cannot be overstated. This event was borne out of Applied Trust’s own improvement of its internal DR/BC planning. If you’d like assistance with the planning, review or execution of your own DR, BC or PMD planning, or simply have experiences of your own to share, we’d love to hear from you!
2 Responses
July 20th, 2009 at 12:33 pm
“our first PMD was a huge success”
Is it hard to overstate your satisfaction?
January 1st, 2010 at 10:57 am
[...] Commit to a coordinated Annual Preventative Maintenance day. Applied Trust is naming July 13, 2010 as IT Preventative Maintenance day. Please join us [...]
Leave a Comment