CIP Secure Development Process 

Revision History
Revision No	Date	Change description	Author	Reviewed by
001	2021-03-14	Draft secure development process	Dinesh Kumar	TBR
002	2021-08-12	Added about CVE tracking process in CIP kernel and CIP Core	Dinesh Kumar	TBR
003	2021-09-03	Added reference for File Integrity document	Dinesh Kumar	TBR
004	2022-08-01	Updated SM-11 with additional information	Dinesh Kumar	TBR
005	2023-10-12	Update SM-3,SM-7 and SM-11 details	Sai Ashrith	TBR
006	2023-11-10	Update process related details for SM-3	Sai Ashrith	TBR
007	2024-03-25	Updated SG requirement details	Sai Ashrith	Dinesh Kumar
008	2024-03-25	Updated SVV-5	Tsukasa Yobo	TBR
009	2024-04-05	Update SVV-1, 2 & 4	Tsukasa Yobo	TBR
010	2024-05-22	Updated based on BV feedback to remove TODOs from SG requirements	Dinesh Kumar	SWG members
011	2024-05-27	Updated SM-4 requirement process based on BV feedback	Dinesh Kumar	SWG members, BV
012	2024-07-05	Updated SG-5 based on BV feedback on 26th May	Dinesh Kumar	TBR

1. Overview 

This document is based on IEC-62443-4-1 (Edition 1.0 2018-01) secure development process requirements.The Objective is to adhere IEC-62443-4-1 secure development process requirements in CIP development as much as possible.

Adherence to these secure development practices will give an edge to CIP over other distributions at the same time it will reduce IEC-62443-4-x development effort for CIP member companies for making products based on CIP.

2. [SM-1] Secure Development Process 

The development process details of CIP are provided in this document.

3. [SM-2] Identification of Responsibilities 

CIP has defined roles and responsibilities for the members who are responsible for CIP development. This RACI(Responsible, Accountable, Consulted and informed ) is reviewed and updated yearly once or whenever there is change in responsibilities

Detailed RACI MATRIX is available at CIP RACI matrix page.

4. [SM-3] CIP Software version 

The process flow while defining applicable software version is mentioned below:

CIP SWG members had a lot of internal meetings to decide the Debian and Kernel version applicable for IEC-62443-4-1 & 2 certification.
Each SWG member comes up with a proposal to define the applicable software ( Debian & CIP Kernel ) for the assessment.
After finalizing the decision based on majority vote, SWG members prepared a proposal to produce infront of TSC WG members.
SWG members include pros and cons in their proposal and finally TSC WG agreed upon Debian 12 (Bookworm) as the version for the base layer of CIP-Core component and 6.1.x version for the CIP Kernel for IEC-62443-4-1 and 4-2 certification.

5 [SM-4] CIP Developer Security Expertise 

CIP has multiple working groups. when a new member joins any CIP working group, its respective CIP member who is responsible to provide basic training and about CIP. Over the years, CIP members have developed various documents and material which is helpful to provide a quick overview of CIP to any new member.

When new member joins specific working group, it’s working group chair responsibility to share following information with the new member.

Details of how working group operates like meeting schedule etc.
Details of Gitlab repositories used for development or testing
Providing required permissions to access repositories or contributions

As part of SWG above mentioned process is followed to on board new member. If a member is supposed to work for specific roles he/she is provided with required privileges.

SWG members maintain a template [Security Expertise document](https://drive.google.com/file/d/1SnUCRPeoZa3uJCwuapvF5tQjP1SLKaFl/view?usp=drive_link) where following information need to be provided by each member to understand the security expertise, past experience and suitability to assign a specific roles.

Note Security expertise document is a private document and is only accessible to CIP SWG members.

6 [SM-5] Process Scoping 

Following three documents can be used to list met and unmet requirements by CIP.

exida gap assessment report
Secure development process document which is current document
Application and hardware guidelines document which describes about IEC-62443-4-2

7. [SM-6] File Integrity 

Following document explains about CIP File Integrity and how user can verify integrity of CIP deliverables.

About CIP File Integrity

8. [SM-7] Development Environment Security 

8.1 Development Security 

Development environment security is achieved by providing restricted privileges to developers as well as all developers use certificate based authentication used by Gitlab.

8.2 Production Time Security 

This requirement is not applicable to CIP.

8.3 Delivery Time Security 

CIP does not deliver any products to its customers. Instead the meta-data as well as the Kernel can be downloaded by the CIP users using Git cloning mechanism. Security during this phase is provided by Git security protocols, file integrity checks when the user tries to get the product from the respective Git repositories. During release, the commit hash is provided as an identifier to help the users get access to precise release made by the CIP members without any integrity issues.

Details of CIP development environment security is provided in CIP Development Environment Security document.

9. [SM-8] Private Key Protection 

CIP maintains document which explains about CIP Private key management. Please refer following document for more details.

CIP Private Key Management

10. [SM-9] Security Risk analysis for externally provided components 

Since all components in CIP are developed externally, hence doing risk assessment for all components is not feasible. CIP team decides which components may pose risk to CIP platform. If a component is suspected to have known vulnerabilities then risk assessment is carried out by following below mentioned steps.

Evaluate the open security issues/CVEs
Categorize open CVEs from CIP perspective as low/medium/high
If open CVEs fall in medium and high categories, do a threat modeling of the component and decide the mitigation
In addition CIP users should be notified for the vulnerable component via email notification

CIP Security, Kernel, Core, Software update working groups are responsible for carrying out such risk analysis for externally provided components. Based on working group recommendation and risk analysis, CIP TSC members make final decision by following voting process whether the component should be included in CIP.

In general any component which is not taken from Debian repository will undergo this process.

11. [SM-10] Custom Developed Components from third party 

This requirement is not applicable to CIP. This is applicable to end products.

12. [SM-11] Security Issues Assessment 

SM-11 requirement expects process followed for security issue handling during following phases.

Requirements
Secure by design
Implementation
Verification/Validation
Defect Management

As in CIP primary work is integration of components hence, CIP does not have any process to address security issues during all the above phases.

CIP completely relies on Debian upstream and mainline linux kernel for security issue fixes and bug tracking. CIP follows upstream first policy and all security issue fixes are first submitted to upstream. Even CIP member companies are advised to directly report issues and submit fixes to upstream projects.

However, CIP uses open source vulnerability scanner and open source databases for security issues and identifying and sharing open CVE details with CIP users.

CIP users should check the CVE list and decide which CVEs may impact their product security.

In order to meet this requirement, CIP users are advised to take following actions.

Regularly review CVE list shared in CIP-DEV ML
If any CVE is critical for the product, do a risk assessment or wait for the fix to be available
Overall, ensuring no critical security issues which may compromise product security leak to end users.

CIP uses open source vulnerability scanner and open source data bases for security issues. Refer this CIP CVE handling document which explains how CIP maintains CVE related information regarding the CIP-Core components using tools like debsecan and Kernel related CVEs from sources like upstream CVE scanner repositories and the process involved in sharing these results to the CIP users.

In addition, CIP specific minimal issues are tracked at following two places.

13. [SM-12] Documented Checklist Review 

CIP SWG maintains Security checklist which is used to verify all security-related processes for IEC-62443-4-1 secure development process compliance is followed for CIP releases.

Current Security Checklist is available at IEC-62443-4-1 Security Checklist.

14. [SM-13] Define Review frequency 

All development process artifacts should be reviewed once in a year. The review should cover following items and review comments and observations should be documented based on IEC-62443-4-1 requirements of review evidence. 1. Issues in current development process. 2. Any critical issues reported by CIP members 3. Actions to be taken to improve on points #1 and #2

15. [SR-1, SR-3, SR-4] Product Security Context 

CIP generic security context has been defined in Security Requirement document. The Security Context would be revised as and when new deployment scenarios and requirements are found.

CIP Security Requirements

16. [SR-2] Threat Model 

CIP generic Threat Model has been created which defines the condition of Threat Model Review and update frequency.

Threat Model document is available at CIP Threat Model document

17. [SR-5] Security Requirements Review and Approval 

Security requirements should be reviewed and approved when created. All review comments should be documented. During review following members should be invited.

Architects/developers (those who will implement the requirements)
Testers (those who will validate that the requirements have been met)
Customer advocate (such as sales, marketing, product management or customer support) and Security Adviser

18. [SD-1] Secure Design Principles 

The design shows how the system’s devices and subsystems are connected, and how external actors are connected to the system.
The design shows all protocols used by all external actors to communicate

with the system.
Trust boundaries are documented.
The design document should be updated whenever the design changes

The above mentioned details are documented here.

19. [SD-2] Defense in depth design 

This requirement is not applicable to CIP. This should be met by end product owners. Defense in depth design should be created by end product owners as it depends upon end products design and what kind of security layers would be part of the defense layers.

20. [SD-3, SD-4] Security design review 

Create evidence for security design reviews
Issues identified during security design reviews are tracked using Gitlab
Create Traceability matrix for security requirements to security design
Create Traceability matrix from threat mitigation to security design
Create Security guidelines for user
Include security design best practices used in debian as well as review Design best practices being developed by OpenSSF if suitable include in CIP

Details regarding security design review and best practices in CIP are documented here based on above checklist.

21. [SI-1] Security implementation review 

CIP project only reuses open source components primarily from Debian upstream. As part of CIP project no new component development happens. As a result there is no consideration for secure design and it’s implementation.

However, secure design related aspects are taken care in Debian and individual package upstream. Debian distribution is one of the most secure Linux distribution and has it’s own security bug tracking system and numerous best practices which are followed by Debian developers. One can refer Best practices for security review and design.

Hence CIP inherits security from Debian. From SI-1 requirement perspective, please following points

Identification of Security requirements

CIP does not address this requirement and recommends CIP users to identify security requirements which are not addressed by the implementation of required to meet certain compliance based on product use cases.

Identification of Secure Coding Standards

If CIP users need to develop new components or modules, it’s their responsibility to identify suitable secure coding standards and avoid using any obsolete or insecure functions. CIP does not address this requirement

Static Code Analysis (SCA)

CIP does not do static code analysis as it directly uses binaries for packages from Debian repositories. However, some of the Debian packages upstream does static code analysis and CIP relies on it.

Review of the implementation

Not Applicable to CIP.

Examination of threats

Not Applicable to CIP.

22. [SI-2] Secure Coding Standards 

As CIP does not develop any new component hence no specific Secure Coding Standards are followed. However, CIP has a brief document which explains about few examples of upstream packages where Secure coding standards are followed.

Refer CIP Security Coding guidelines document

23. [SVV-1] Security requirement testing 

CIP Security Working Group and Testing Working Group performs security testing by executing IEC layer tests. These tests primarily have test cases to verify security features supported by CIP Security image. Further sections explain about the details.

Functional Testing of Security Requirements 

Functional testing of Security Requirements is carried out by IEC layer tests. CIP IEC layer testing repository contains the tests which are executed during IEC layer testing.

Each IEC layer test has three phases.

PreTest

Make security configurations like configure password policy, create test users etc and other security configurations based on security requirements. CIP currently has defined some security configuration which is used as default and used for testing CIP security images.

Sample security configuration can be found at CR-1.1 Security configuration

RunTest

As part of this phase test is executed by using the security configuration provided by the user. Once execution is over, test result is written in a text file which can be referred once execution is over. Current IEC layer tests can be executed manually or using isar-cip-core CI.

Results of isar-cip-core CI for IEC layer tests are published in SQUAD repository.

PostTests

Once execution of individual test is over, any temporary test data is deleted e.g. any test user created for IEC layer test will be deleted once test execution is over.

Performance and scalability Testing 

CIP relies on upstream testing for Performance and scalability testing and no additional tests are executed for measuring any performance parameters or scalability.

CIP end users are expected to perform this testing based on the actual use cases.

Boundary and Edge condition Testing 

CIP relies on upstream testing for boundary and performance testing. However, CIP kernel is tested.

24. [SVV-2] Threat Mitigation testing 

As CIP primarily focuses on integration by reusing open source components. Hence It does not frequently work on design changes or including new components. Upstream components are tested in respective projects for various types of security issues.

However, CIP has created Threat Model to uncover security issues by doing attack surface analysis. The threat model was created considering various data flow scenarios which are regularly executed during various CIP development activities.

During CIP threat modelling identified threats were mitigated by including security packages which adds capabilities to mitigate the threats. These tests are part of IEC layer tests and are executed on regular basis.

CIP recommends users to do Threat Modelling for respective use cases and based on IEC-62443-4-2 requirements and should plan for threat mitigation testing.

25. [SVV-3] Vulnerability testing 

About Vulnerability testing 

Vulnerability testing is a systematic process of identifying, evaluating, and addressing weaknesses, flaws, or vulnerabilities in systems, networks, applications, or other digital assets. Its purpose is to reduce the possibility of cyber criminals breaching your IT defenses and gaining unauthorized access to sensitive systems and data.

It involves scanning, probing, and analyzing systems and applications to uncover potential vulnerabilities, such as coding errors, configuration flaws, or outdated software components.

Why is Vulnerability testing important 

Comprehensive understanding of the attack surface
Adapting to evolving threats
Reducing attack vectors
Enhanced security measures
Continuous improvement
Risk management

General Steps for Vulnerability Testing 

Following are the general steps for performing vulnerability testing. However, steps may vary based on the type of product and goals.

Conducting risk identification and analysis
Developing vulnerability scanning policies and procedures
Identifying the type of vulnerability scan
Configuring the scan
Performing the scan
Evaluating risks
Interpreting the scan results
Creating a remediation and mitigation plan

Tools provided by Debian for Vulnerability Testing 

Debian provides several tools for vulnerability testing, based on the use cases CIP users should select right tool, here is the the list of some of the most common tools.

nessus
nmap
doscan
TIGER
tripwire
lynis
checksecurity
logcheck

CIP Vulnerability testing 

As the key goal of CIP is to maintain OSBL (Open Source Base Layer) by reusing Open Source Components hence CIP does not perform any regular vulnerability testing. CIP totally relies on upstream and Debian Security vulnerability testing.

CIP users are advised to follow Debian-Security-tracker,`Debian-Security-announcement mailing list <https://lists.debian.org/debian-security-announce/>`__ and Debian-Decurity-advisories which will help to improve overall security posture of the CIP based products.

In addition, CIP users can develop their own Vulnerability testing processes and strategy based on the product requirements.

> During CIP IEC-62443-4-1 & IEC-62443-4-2 assessment Vulnerability Testing for CIP Security images was carried out by a third party agency.

As part of CIP testing, SVV-3 c is covered. CIP Kernel Working Group regulalry scans CIP kernel sources for CVEs. CIP CVE scanner is used to scan for CVE findings. CIP Kernel Working Group shares weekly CVE information with CIP users. CIP CVE handling document outlines detailed process followed for CVE handling.

Whereas as part of CIP testing, SVV-3 a, SVV-3 b, SVV-3 d, SVV-3 e, are not covered as CIP primarily focuses on integration and long term maintenance by reusing Debian binaries. CIP users are advised to do further analysis for SVV-3 sub requirements and plan for testing.

26. [SVV-4] Penetration testing 

CIP does not perform Penetration testing considering it’s key goal of reusing Open Source Components and providing SLTS (Super Long Term Support) for CIP Core packages and Linux Kernel.

However, CIP users can refer a generic Penetration Testing document prepared by CIP developers which will help to plan for Penetration testing for specific products.

> During CIP IEC-62443-4-1 & IEC-62443-4-2 assessment Penetration Testing for CIP Security images was carried out by a third party agency.

27. [SVV-5] Independence of testers 

Routine testing will be performed by the CIP Testing Working Group, which is “independent” of the upstream project developers and CIP kernel maintainers.

28. [DM-1 to DM-5] Receiving notifications of security issues 

Security issues are tracked using CVE scanner tools for both CIP Kernel and CIP Core.

CIP CVE scanner runs periodically to fetch fixes for CVEs and apply in CIP repos.

Further details can be found about CIP Core CVE scanner at CIP Core CVE scanner

CIP Kernel CVE Scanner

CIP Kernel CVE checking is done weekly and reports are published in cip-dev mailing list.) Currently there is no specific policy to stop releasing because of missing patches as long as stable kernels are released

Release policy is reported at every E-TSC. The latest one is as follows CIP Kernel CVE fixes release policy

Further details of CIP Kernel CVE scanner can be found at CIP Kernel CVE scanner

Notification of CVE fixes are sent by email to CIP users.

CIP does not maintain it’s own bug tracking system. Refer this document to see the upstream methods to handle the CVE cycle.

29. [DM-6] Periodic review of security defect management practice 

Review current defect management practices and processes once in a year and make required changes as needed.

30. [SUM-1] Security Update Qualification 

As CIP Core and CIP Kernel follow different mechanism for security updates, following section explains how this requirement is met in CIP Core and CIP Kernel.

Security updates address the intended security vulnerability.

CIP Core 

CIP Core does not develop or modify any upstream components. It just reuses Debian packages to build CIP reference images.

CIP Core does not release any security updates instead CIP users are advised to apply security updates directly on the device based on the security analysis of security issues.

CIP Kernel 

CIP Kernel working group constantly monitors security issues data in upstream and applicable fixes for security is applied in CIP kernel. After applying security fixes, CIP Kernel testing is carried out to ensure there are no side effects introduced by the security issue fixes.

SM-11 related section of this document explains how CIP kernel working group members share security issues fixes to CIP kernel.

CIP Kernel test reports are regularly published at Kernel CI page

So the actual verification of fixes is carried out by Debian and other security teams in upstream.

31. [SUM-2, SUM-3] Security update documentation 

a. The product version number where security patch applies 

CIP Core

CIP Core does not release any security fixes instead users can directly apply security fixes from upstream. The version number cmpatibility is handled by apt package manager.

CIP Kernel

CIP Kernel working group releases updates for security issue fixes via CIP-DEV mailing list on regular basis. The email has details of specific CIP kernel version for the fixes. The updates email also has CVE IDs and users can find all relevant details about security issue fixes.

One example of security issues fixes updates can be found at CIP-DEV

32. [SUM-4] Security update delivery 

CIP Core 

Security updates for CIP Core packages are directly downloaded from signed URLs and apt package manager does verification of binaries before installing on the system. Hence this requirement is automatically met in CIP Core packages updates. However, CIP itself does not take any actions explicitly.

CIP Kernel 

Security issue fixes email shared by CIP Kernel working group members have security issue fixes details with commit it. Each commit is signed commit by respective kernel developer with their private keys. This ensures the authenticity of each commit and CIP users can verify each security fix sources.

The required process to verify patches from Github or Gitlab is common and well known hence not adding it here.

33. [SUM-5] Timely delivery of security patches 

As stated above security fixes for CIP Core are not released by CIP team instead users are advised to directly apply security fixes from upstream by directly updating individual packages.

Whereas, CIP Kernel working group regularly releases security issues fixes and CIP users get the updates via CIP-DEV mailing list. Following information is provided to address this requirement.

a) the potential impact of the vulnerability

CVE IDs are global IDs and CIP users can get details of potential impact of the vulnerability. Based on the relevance users can choose to apply or ignore the updates.

b) public knowledge of the vulnerability

It’s a public information and can be found using CVE IDs. CVE IDs are shared along with CVE release email by CIP Kernel members.

c) whether published exploits exist for the vulnerability

This information is not shared by CIP.

d) the volume of deployed products that are affected

This information is not shared by CIP.

e) the availability of an effective mitigation in lieu of the patch

This information is not shared by CIP.

34. [SG-1, SG-2] Product defense in depth 

SG-1 and SG-2 requirements are not applicable to CIP as CIP users may develop various types of products using CIP platform (e.g. host device,network device), therefore this requirement should be filled by CIP end product not by CIP platform.

35. [SG-3] Security Hardening guidelines 

Every user created by a CIP user/administrator in their product has certain access privileges based on the default security context of the system. acl utility provided in CIP can be used by the admin to alter these access controls for certain user to protect some confidential data or process.
Instructions to follow while integrating product’s application programming interfaces/protocols with user applications should be done by the end product’s owner.
Instructions to apply and maintain the product’s defense in depth strategy should be documented by the end product’s owner.
In CIP there are some pre-defined policies regarding security capabilities like password strength, locking an account if multiple incorrect login attempts are detected, immediate intimation if audit logging failures are detected in the system, remote login policies etc. These settings are already pre-configured here but based on product owner’s use-case they can be modified to meet their requirement.

Contribution of these security capabilities provided by CIP to the product’s defense in depth strategy should be judged by the product owner.

But in a general scenario, let us assume an attacker trying to perform a login to the running system with some random passwords. If he does this for a certain number of times which is configurable, the account gets locked. Here, the password strength and number of incorrect login attempts set by the user will acts as a firewall. If this is crossed by that attacker, then access controls provided to the files that user wants to protect can act as a second firewall. If this is also crossed by the attacker, then the intrusion detection system provided by aide shall act as an alarm to the user in taking immediate course of action.

These configurable values can be set/changed even in the running system the administrator.

More than 20 Debian security packages are installed to provide a certain set of security capabilities to the end product. These tools can be used for administration, monitoring, incident handling by the user.
CIP’s two main components CIP-Core and CIP-Kernel are being maintained regularly. So it is the product user’s responsibility to actively follow latest security fixes provided to CIP Kernel and Debian packages installed in the system. The user has to update their system to bring in the latest fixes. Refer REQ-CIP-HARD-007 & REQ-CIP-HARD-008 in hardening document for more information.
Any vulnerabilities in CIP Kernel shall be provided on a weekly basis. It is the end user’s responsibility to subscribe and get the latest information on Kernel CVE’s. CIP provides a tool which can be used to get the active CVEs of all the Debian packages installed in the system. These security incidents need to be properly tracked by the product supplier and should be reported to the end user.

Additionally CIP has also defined detailed security hardening guidelines for following.

Default security policies to meet IEC-62443-4-1 security requirements
Compilation flags
Other configs

Those details can be found in this CIP security hardening document.

36. [SG-4] Secure Disposal Guidelines 

This requirement is not applicable to CIP. Disposable guidelines are applicable to end products.

Some general guidelines for secure disposal of equipment are mentioned below for reference.

Users must ensure that process of equipment disposal is strictly controlled or else it may impact the confidentiality of data making it available to unauthorized audience.

Devices that contain sensitive information should be physically destroyed or the information must be destroyed, deleted or overwritten using techniques that make the original data non-retrievable.

If the equipment is going to be re-used it is important any previous data and potentially installed software is securely wiped and the device is returned to a known “clean” state.

As an additional layer of security, the information can be encrypted before disposal. In this way, a hypothetical case that someone could recover the information through some mechanism, then decryption is required.

There are various security compliance based on the type of device and related domain, CIP users are advised to refer specific compliance details for media sanitization. NIST published media sanitization standard NIST SP.800-88r1 is one of the most popular standard.

37. [SG-5] Secure operation guidelines 

CIP developers and maintainers strive to keep all CIP layers secure by applying latest security fixes as well as by publishing advisories to CIP users via CIP-DEV mailing list on regular basis. CIP Kernel maintainers regularly review the security issues (CVEs) reported in upstream and after careful analysis and multiple rounds of reviews relevant fixes are applied in CIP kernel.

As a platform CIP has IEC layer which can be configured for specific security requirements. Some of the default security configurations provided in CIP security image are following.

This policy enforces user creation with strong password. policies.

Account will be locked if consecutive unsuccessful login attempts are made. All transactions are recorded in the audit logs and any failure in audit logging shall be reported to the user. Multi-factor authentication mechanism is configured in CIP.

aide tool installed in the system provides intrusion detection functionality. Administrator needs to run the aide check to monitor any integrity failures in the system.

CIP user security manual provides details of how to configure CIP IEC layer to enforce specific security policy.

38. [SG-6] Account management guidelines 

CIP security image has only one default user account which is root account with the password root. For creating additional user accounts, follow standard Linux user management steps.

CIP end users should decide about how many additional users should be created based on their requirements. Here we are providing few best practices which will help to improve security of the system.

Monitor and audit each user account
Follow least privilege principal
Implement strong password policies
All user accounts should be reviewed periodically to ensure that they are still necessary and being used appropriately.
Inactive users should be removed from the system, while active users should be monitored for suspicious or unusual behavior.

39. [SG-7] Documentation Review 

All CIP development documents are maintained in Gitlab. Reviewers share feedback and comments using any of the following methods.

Create Gitlab issues with comments
Send review comments in email
Send MR with the changes

Only few CIP members have rights to merge review comments changes in the documents. All the review comments are maintained as part of Gitlab commits messages which are signed messages.