There are many peculiarities that must be taken into account when considering the safety of industrial systems and SCADA systems. One especially relevant is patching or updating the systems or software that they support. When through a security assessment of this type of system you get to the question: “how do you carry out maintenance of systems to patch known vulnerabilities?” We can find very different answers. Some examples:
Option 1: Poker face
We do not apply security patches. It is not necessary since our industrial network is completely isolated, we rely on the ‘air GAP’ to protect our systems and most vendors don’t publish security updates. On the other hand, sometimes the software upgrade also involves hardware changes, so budgetary constraints don’t permit such updates.
This answer or other similar ones are quite common. I do not think it is a crazy strategy to follow to not apply security patches when these conditions are met:
- A risk analysis was performed to clearly understand what the threats that may affect the non-patched systems and what impact such threats could have. Note that I do not mean to make a superficial risk analysis, but I mean analyzing risks in-depth. That is, know exactly what vulnerabilities are not patched up, how it could be exploited by an attacker and what compensatory measures are implemented to mitigate the risk of not patching it. When considering the threats one should pay particular attention to the perimeter of industrial systems, points of interaction with traditional networks and access points that are easily accessible by visitors or the general public.
- Once this risk analysis is done, if the problems, costs or difficulties that result from applying the patches are greater than the risk of non-patching, it make sense not to apply the patch.
- This decision should be carried out in an informed and conscious way by the risk owner.
- The risk level should be reviewed regularly.
On the other hand, it is clear that we must put pressure on vendors to implement vulnerability management processes for their products and this point should be a key criteria in the selection of these technologies.
Option 2: The quiet man
Well, it depends on the vendor, the device and the technician is responsible for the update. We don’t have actually documented, but we use different methods such as direct download of patches from the manufacturer’s website (who don’t publish a signed hash of the file for verification after download, or when they do, we don’t check it anyway). Sometimes, to save time, we even download the patches from our home where bandwidth is higher than in the office. We recorded it in our USB and connect to the network of industrial systems that is completely isolated from the IT network. You know, the famous ‘air gap’. Other times, a partner or the vendor comes with their USB or with laptop and connects it directly to our industrial network to apply updates or perform any other maintenance task.
In these cases, as you can imagine, the problem is that the ‘ isolation ‘ ceases to be such as when the media device is connected to the isolated network. With these practices our systems are exposed to so many threats. Some of them could be:
- Malware that can cause performance problems or even a denial of service on these systems.
- Advanced Malware can even allow remote control or data theft. Although a priori this seems impossible in an isolated network, today we can find numerous proofs of concept on how they could perform these attacks dodging the ‘Air GAP ‘.
- Fraudulent updates or patches downloaded from internet whose system changes can be different than expected.
- Connections from third parties to our network using their own laptops which may have a lower level of our security. Also, if we do not control what activities performed in our systems can be a source of threat to be considered. Do not forget that it is very likely that our partners also work for our direct competition, so it is a source of risk to consider.
Option 3: The outstanding
In our organization we have documented and secure processes for carrying out the update of all our industrial systems. We have different systems to be informed when any new vulnerability is discovered which could affect our systems. A comparative analysis of the risks between upgrade or leave the systems unpatched is performed, so the risk owner can set the criteria used for deciding whether to patch and in which term it should be done. Once we decided that a patch should be applied, we obtain it from a secure source verifying its integrity and authenticity, we deploy it in our test environments to verify that the update will not compromise the functionality or security of systems and, only after this, and under our strict control and supervision, the update is deployed in production within the time limit set by the risk owner.
Well, if you obtain this answer, you can’t do anything else but congratulate the client. However, I never found this answer yet.
Of course I’m simplifying the possibilities and using hyperboles in this article, but my goal is to make you think about the fact that managing critical vulnerabilities are a key aspect to consider in security evaluations, especially when you are evaluating Industrial or SCADA systems. If you don’t update your systems, you will be accumulating vulnerabilities but if you update it in the wrong way, the update process itself can be a (big) risk source. Therefore, to plan, to document and to establish a continuous improvement process over the vulnerability management process should be in the agenda of any CSO who intends to improve the security of his organization.
Kudelski Security Research 