CIT adopts a new patch management schedule
Effective October 2009 CIT adopted a new patch management schedule for campus desktop and laptop Windows computers. CIT will push security patches once per month, usually on the 3rd Wednesday.** In addition, CIT will deliver required software program updates once per month, on the 1st Wednesday of the month.
What has changed and why is it changing?
The previous patch strategy was to deploy security patches within 7 days of release by a vendor. The number of vendors releasing security patches has steadily grown and the frequency of their patches has increased to a point that we were finding it difficult to balance the need to keep campus workstations secure while maintaining a tolerable disruption level to our users.
As the nature of computer threats has continued to change and evolve it has become critical that CIT maintain security patches on workstations to the most current levels and that we have the capability to respond quickly to incidents and outbreaks.
In August 2009 CIT was able to begin using a new patch management system from Novell which allows us to cleanly “bundle” multiple security patches into one delivered package to workstations. Whether the patches come from Adobe, Apple, Microsoft, Mozilla, Symantec, or others we can bundle them together. If a system reboot is required it can be done once for the entire bundle rather than for each patch. This will allow us to maintain a higher patch level while limiting the number of disruptions to users.
What is the difference between security patches and program updates?
Security patches are issued by application vendors to address specific security-related concerns with their software. They are issued because a security flaw has been identified in their product. Once identified these security flaws quickly become known by the hacker community and exploits of them can occur as quickly as the same day they are discovered (so called “zero day exploits”).
Program updates are issued by application vendors to address “bugs” in their software which impact the functionality or stability of the program but are not related to security concerns. CIT is interested in deploying these when they address specific issues on Calvin workstations.
Why are security patches and program updates done on different days?
Security patches and program updates are managed by different systems in CIT. This means that we cannot bundle the patches and updates together. If we were to deploy these on the same day it would mean users would be interrupted twice on the same day and possibly be required to reboot their system twice on that day. CIT feels this is too much of a disruption on one day.
Can I opt-out of a patch or update?
You cannot opt-out of a security patch or update. To the degree that our system allows we will give you options to either defer installation of a patch for a period of time (3-5 days) or we will allow you to defer the system reboot for up to 24 hours after installation of the patch.
Why do I have to reboot?
Some patches and updates require a reboot and some do not. This is determined by the software vendor and by the current status of your machine. Some users may be prompted for a reboot and others might not be prompted for the same patch.
Once a patch is installed and before a reboot the machine is in what is called a “dirty state.” The patch has been delivered but it is not effective until the reboot occurs. This “dirty state” is a time when the machine can be less stable and unable to accept additional patches or updates. In some cases the machine may become unresponsive and/or not able to be managed by CIT’s management tools.
Patch management best practices recommend that you reboot as soon as possible when prompted. We recognize that it is not always practical to immediately reboot so we allow up to 24 hours for the reboot in case you have processes running that cannot be interrupted immediately.
Can I delay rebooting my computer?
Whenever possible CIT will provide you with the option to either defer installation of the patch for a period of time (3-5 days) or defer the reboot for 24 hours. This will depend on the patch bundle being delivered, the requirements programmed by the vendor, and the capabilities of our patch system tools.
Each bundle will be different and CIT will always look for the best balance between minimizing your disruptions while still achieving an effective delivery of the patches.
What about labs and Smart Classrooms?
Computer labs will not receive monthly patches or updates during the semester unless there is a critical need to do so. These labs have DeepFreeze installed on them which brings the computer back to its original state on every reboot, even if something malicious happens to it. DeepFreeze will also wipe out any patch we push to the machine during the semester so it does not make sense to deploy patches to these machines during the semester. They are updated at semester breaks.
Smart Classrooms are patched by CIT during non-classroom hours. We do this so that patching and updates do not interrupt class sessions.
Do I need to receive CIT delivered patches even if I have “Automatic Updates” turned on?
CIT disables the Windows Automatic Updates feature when we set up your computer. Every year there are patches and updates which we chose not to deploy because they cause other problems on our systems. So we want to test the patches before deploying them.
In addition, Microsoft Automatic Updates only covers Microsoft patches. Our patch management system covers security patches from 17 different vendors. It is more comprehensive in coverage than the Microsoft system.
Where can I find out about planned patches and updates?
CIT maintains the CIT Scheduled Maintenance blog at http://www.calvin.edu/weblogs/maintenance which you may refer to for a list of current and upcoming patches and updates. This blog includes a specific list of patches included in each bundle. If you use an RSS reader you may subscribe to this blog to be notified of new postings.
CIT will also provide notices through CIT-Alert emails initially, especially for large patch bundles. Once the monthly schedule becomes more routine we will provide CIT-Alerts only as needed.
Will this affect requests for new software installs?
No, this does not affect requests for new software installs or requested software updates. CIT will continue to fulfill these requests on an “as requested” basis. Often for our standard software that is within a couple of business days for your office computer, depending on availability of the software.
Computer labs are updated prior to the fall semester and again if needed before Interim and second semester. Please make your request for software in computer labs several weeks before the beginning of the semester.
Does this impact Mac OS computers?
CIT uses a different patch management tool for Macintosh computers than for Windows computers. We have adopted this same patch deployment schedule on the Mac platform so that campus users know when to expect patches no matter what platform they are using.
** It technically is not the 3rd Wednesday of each month. It is actually the 8th day after the 2nd Tuesday of the month. Huh? OK, here’s the full scoop: Microsoft releases its security patches on the 2nd Tuesday of each month (a.k.a. “Patch Tuesday”.) Our patch management subscription provides them to us by Thursday or Friday. We need a couple of days to review the patches, create the bundle, and run some initial tests on our test machines. By the Monday following Patch Tuesday the patch bundle has been created and is deployed to all machines in CIT as a pilot group. We run them for two days in CIT to verify the patches and the delivery mechanism are working as planned before releasing them to the entire campus. That brings us to the 8th day after the second Tuesday, which is usually the 3rd Wednesday of the month.
Here’s the catch: If the first day of the month is on a Wednesday (e.g. September 2010) then the 3rd Wednesday comes the day after the 2nd Tuesday. This leaves us no time for testing. So in those months we will deliver the patches on the 4th Wednesday of the month.
Yes, this is more information than you care to know, but it does give you a glimpse into the complexities we have to deal with on these seemingly routine matters.
