Limitations

Objective

The aim of this section is to create a similar pipeline for SuiteCRM, a different application than DVNA, to identify assumptions made whilst creating the pipeline for DVNA to provide a solution for 1st point of the problem statement under Task 5.

SuiteCRM

The application that I chose for the validation of the previously built pipeline was SuiteCRM. It is a Customer Relationship Management tool which is the open-source forked version of SugarCRM. SuiteCRM adds a few additional features to its fork and is free to use.

I chose the application because of it being a bigger application than DVNA and it is also an application that is used in the real-world and is not just a dummy application. Another reason was that it is written in PHP and hence, the difference in the stack from DVNA also would help to identify assumptions made in the previous pipeline created as the solution to the problem statements.

Configuration

Before I began setting up tools on the virtual machine, I forked the SuiteCRM GitHub repository on my account. My fork of SuiteCRM can be found here.

I also created a new pipeline, named suitecrm-aws-pipeline, following the steps from this report's earlier section on Setting Up Pipeline.

Since SuiteCRM is written in PHP, I had to install a few things on the Jenkins EC2 instance. I followed this article to perform the necessary configurations for the application. I did, however, skipped all steps related to installing and configuring MariaDB as I already had MySQL Community installed on the system. I also skipped the 10th step as I cloned my fork of the SuiteCRM GitHub repository and did not use the zipped archive to get the source code of the applications. Lastly, I also skipped steps 19 and 20 because I did not need to set up the Cron job as it was not required in the context of the problem statement.

  • Trying to view SuiteCRM from the browser (step 12 from the article) I faced an issue that the packages were not built. This was due to me cloning the repository instead of using the zip archive. Hence, I had to install Composer, the package manager for PHP. I followed the official documentation and performed to required steps to install Composer globally. After a successful installation, I ran composer install in the project's root directory to build the dependencies for SuiteCRM.

  • Since SuiteCRM is a PHP application, I also needed to install a web-server to serve the application. I chose Apache for this as mentioned in the article I was using to set up SuiteCRM. Since I had previously mapped port 80 to Jenkins' port 8080, I had to change the port Apache would listen on. To do this, I used the command - sudo nano /etc/apache2/ports.conf and made the following changes to make Apache listen on port 9090:

#Listen 80
Listen 9090

<IfModule ssl_module>
    Listen 443
</IfModule>

<IfModule mod_gnutls.c>
    Listen 443
</IfModule>

SAST

The first thing that came to light while creating the pipeline for SuiteCRM was the fact that since it was based on a different tech stack than DVNA, most of the SAST tools I used previously were incompatible with it. The only two tools that worked with both DVNA and SuiteCRM are mentioned below:

Dependency Check

Since it was already installed I did not need to do a fresh installation for Dependency Check. I amended the command to point to the workspace directory of the new pipeline (/{JENKINS HOME DIRECTORY}/workspace/suitecrm-aws-pipeline). I also renamed the output to be called suitecrm-dependency-check-report.

The complete report generated by Dependency Check for SuiteCRM can be here.

Snyk

Like Dependency Check, since Snyk was already installed I just created a new script to run Synk from. I amended the script, like Dependency Check, to point to /{JENKINS HOME DIRECTORY}/workspace/suitecrm-aws-pipeline and renamed the output report as suitecrm-snyk-report. The rest of the script remained as it is.

The complete report generated by Snyk for SuiteCRM can be here.

Since the majority of the previously used SAST tools were not suitable to test a PHP application, I found two new tools built to perform SAST specifically on PHP applications. These applications are mentioned below:

Symfony

I followed the official documentation to download the Symfony CLI executable from this link mentioned in the documentation and moved it to /usr/local/bin to be accessible for all users. I used the binary executable as it was more convenient than using Composer to install it as a package. Symfony gave a non-zero status code on identifying issues so I wrote a bash script (suitecrm-symfony.sh) and placed it under tool_scripts/ directory.

The complete report generated by Symfony for SuiteCRM can be here.

The contents of the script (suitecrm-symfony.sh) are mentioned below:

#!/bin/bash

symfony security:check --dir /{JENKINS HOME DIRECTORY}/workspace/suitecrm-aws-pipeline --format json > /{JENKINS HOME DIRECTORY}/reports/suitecrm-symfony-report

echo $?> /dev/null

PHP Malware Finder

I followed the documentation present in the official GitHub repository for PHP Malware Finder (PMF). I first installed Yara with apt and then cloned the repository as instructed in the documentation in the tool_scripts directory. Since PMF did not exited with a non-zero code even after identifying a security issue, I added the scan command /{JENKINS HOME DIRECTORY}/tool_scripts/php-malware-finder/php-malware-finder/phpmalwarefinder $(pwd) > /{JENKINS HOME DIRECTORY}/reports/suitecrm-pmf-report as it is to a stage in the pipeline.

The complete report generated by PHP Malware Finder for SuiteCRM can be here.

DAST

To perform DAST, I had to deploy SuiteCRM on the Jenkins EC2 instance for the tools to perform scans and attack the application. Another limitation that became evident was the fact that removing all application files and cloning a new copy from the source code repository required me to do the configuration for the application through the web-based console even if the database had the relevant entries persisted on the database. I looked around in the changes that happened after doing the initial configuration with the web-based console and found that SuiteCRM creates a new file, config.php, which stored the settings. The solution to this was to retain this configuration file while being able to fetch changes made to the source code of the application. I thought of two solutions - store this config file locally and copy it after a fresh fetch of the source code or, the simpler solution, pull changes from the repository with a git pull. I chose the latter as it was the simpler solution.

The next limitation surfaced when I had to use Apache as the web-server as SuiteCRM, unlike DVNA, does not ship with a development server. The problem was that starting and stopping Apache's service on the virtual machine required sudo privileges, that were not granted to the Jenkins system user. I found this answer on StackOverflow which explained how to grant passwordless sudo privileges for specific executables/scripts. I used this answer as a reference to add the Jenkins system user to the sudoers file and grant it sudo privilege to start and stop the Apache web-server. I amended the sudoers file by adding the following line to it under User privilege specification:

jenkins ALL= NOPASSWD: /etc/init.d/apache2

After granting the required privilege to the Jenkins system user, I used the following commands to start and stop the Apache web-server:

# To start the web-server
sudo /etc/init.d/apache2 start

# To stop the web-server
sudo /etc/init.d/apache2 stop

Since DAST is black-box testing, where the tool interacts with the application being tested by performing actions like an attacker would, both the previously used tools (OWASP ZAP and W3AF) worked with SuiteCRM without any issues. I used the Jenkins Agent EC2 instance to run the tests as I did previously. The changes I made are mentioned below:

OWASP ZAP

I wrote an identical script as before and changed the output report's name to suitecrm-zap-report. I added a stage in the pipeline to use this new script to perform DAST on SuiteCRM.

The complete report generated by OWASP ZAP for SuiteCRM can be here.

W3AF

I wrote an identical configuration script for W3AF as before and changed the values to reflect the names of 'username' and 'password' fields on the login page for SuiteCRM. I also renamed the output report's name to suitecrm-w3af-report. I added a stage in the pipeline to run the W3AF console executable with the new configuration script.

The complete report generated by W3AF for SuiteCRM can be here in a zipped format.

Code Quality Analysis

Since the tools used for Code Quality Analysis are language-specific, I could not use the tools I used before for DVNA. Hence, I found a couple of tools that are meant for PHP applications. The tools I used are PHP Code Sniffer (PHPCS) and PHP Mess Detector (PHPMD).

PHP Code Sniffer

PHP Code Sniffer or PHPCS is a set of two PHP scripts - one that analyzes the code for violations of coding conventions and the other which can automatically fix the identified issues. For the purpose of the task in the problem statement, I was only concerned with identifying the linting issues and hence skipped the second script.

  • To install PHPCS for Code Quality Analysis, I downloaded the phar executable for the scanner with wget (as instructed in the documentation), made it executable with chmod and moved it to /usr/local/bin/ for it to be accessible to all system users. The commands I used to achieve these steps are mentioned below:
wget https://squizlabs.github.io/PHP_CodeSniffer/phpcs.phar
chmod +x phpcs.phar
mv phpcs.phar /usr/local/bin/phpcs
  • Next, I ran it PHPCS on the SuiteCRM project directory with the command mentioned below:
phpcs /{JENKINS HOME DIRECTORY}/workspace/suitecrm-aws-pipeline

Note: Executing PHPCS on the whole project directory caused the virtual machine to run out of free memory which crashed PHPCS. So, I wrote a Python script to identify all PHP files present in the SuiteCRM project directory and ran PHPCS on the files individually and appended these individual results to suitecrm-phpcs-report. The contents of the script I wrote are mentioned below:

#!/usr/bin/python3

import os
import sys

print('[+] Starting scan with PHP Code Sniffer...')

try:
    BASE_PATH = sys.argv[1]

except IndexError:
    print('[-] Path not supplied...')
    sys.exit(1)

paths = [BASE_PATH]
php_files = []

print('[+] Scanning directory for PHP files...')
while paths != []:
    base_path = paths.pop()

    try:
        with os.scandir(base_path) as entries:
            for entry in entries:
                if entry.is_file():
                    if entry.name.endswith('.php'):
                        php_files.append(os.path.join(base_path, entry.name))
                else:
                    paths.append(os.path.join(base_path, entry.name))

    except PermissionError:
        print(f'[-] Could not open {base_path} due to insufficient permission...')

print('[+] Scan completed...')

print('[+] Running PHPCS on PHP files...')
try:
    for php_file in php_files:
        print(f'[+] Scanning {php_file}')
        os.system(f'phpcs {php_file} >> /var/lib/jenkins/reports/suitecrm-phpcs-report')
    print(f'[+] {len(php_files)} PHP files scanned...')
    print('[+] Code Quality Report generated...')

except KeyboardInterrupt:
    print('[-] Exiting...')
  • Lastly, I added a stage in the pipeline to execute the Python script by supplying it the path of the project directory to scan.

The code analysis report generated by PHP Code Sniffer can be found here.

PHP Mess Detector

PHP Mess Detector is another tool that finds Linting issues in PHP code. It is quite similar to PHPCS in terms of installation but is more customizable by allowing the user to write custom rules for the analysis as well as writing the output to a file in XML, HTML, Text and JSON formats.

  • To start, I downloaded the phar executable with the below-mentioned command, as instructed in the official documentation:
wget https://phpmd.org/static/latest/phpmd.phar
  • Next, I made it executable with chmod and then moved the executable to /usr/local/bin to make it accessible to all system users:
chmod +x phpmd.phar
mv phpmd.phar /usr/local/bin/phpmd
  • Then I tested the tool by executing it against the project directory for SuiteCRM with the below-mentioned command, where the output report would be in XML format and the rule-set used would be cleancode that comes along with PHPMD:
phpmd /path/to/source/ xml cleancode --reportfile /{JENKINS HOME DIRECTORY}/reports/suitecrm-phpmd-report

Note: PHPMD too ran out of memory trying to load all the files at once, so I reused the script I wrote earlier and edited it to execute PHPMD, instead of PHPCS, on individual PHP files present in SuiteCRM. The contents of the script are mentioned below:

#!/usr/bin/python3

import os
import sys

print('[+] Starting scan with PHP Mess Detector...')

try:
    BASE_PATH = sys.argv[1]

except IndexError:
    print('[-] Path not supplied...')
    sys.exit(1)

paths = [BASE_PATH]
php_files = []

print('[+] Scanning directory for PHP files...')
while paths != []:
    base_path = paths.pop()

    try:
        with os.scandir(base_path) as entries:
            for entry in entries:
                if entry.is_file():
                    if entry.name.endswith('.php'):
                        php_files.append(os.path.join(base_path, entry.name))
                else:
                    paths.append(os.path.join(base_path, entry.name))

    except PermissionError:
        print(f'[-] Could not open {base_path} due to insufficient permission...')

print('[+] Scan completed...')

print('[+] Running PHPMD on PHP files...')
try:
    for php_file in php_files:
        print(f'[+] Scanning {php_file}')
        os.system(f'phpmd {php_file} xml cleancode >> /var/lib/jenkins/reports/suitecrm-phpmd-report')
    print(f'[+] {len(php_files)} PHP files scanned...')
    print('[+] Code Quality Report generated...')

except KeyboardInterrupt:
    print('[-] Exiting...')
  • Lastly, I added a stage to the pipeline to run the required command, generate the report and store it in the reports/ directory.

The code analysis report generated by PHP Mess Detector can be found here.

Generating Software Bill of Materials

Unlike DVNA, which is built on Nodejs, SuiteCRM is a PHP application and hence, I could not have used the CycloneDX's Nodejs binding. So, I had to switch to the PHP binding available from CycloneDX which can be found here.

  • I had to first install cyclonedx-php-composer for which I used the command mentioned in the documentation present in the GitHub repository:
composer require --dev cyclonedx/cyclone-php-composer
  • Then to verify it was working, I ran the following command to generate the SBoM:
composer make-bom
  • After verifying the SBoM generated, I added a stage in the pipeline to install cyclonedx-php-composer as a dev dependency, run it to generate the SBoM and lastly, move the SBoM to the reports/ directory.

The Software Bill of Materials generated by CycloneDX can be found here.

Deploying SuiteCRM

SuiteCRM posed another challenge when it came to deploying it to production. Since it created a dynamic configuration file (config.php) after the initial installation and using the Docker image would require to go through the set up after each deployment, I chose to use an EC2 instance to deploy SuiteCRM. Also, now that I was using a full-fledged virtual machine, I set up the database for SuiteCRM on the instance itself. The following are the steps I added to the stage to deploy SuiteCRM to the production EC2 instance:

  • First, I copied the dependencies built locally in the Jenkins EC2 instance over to the production instance in a directory suitecrm/.
scp -r * ubuntu@<PRODUCTION VM IP>:/home/ubuntu/suitecrm
  • Next, I stopped the Apache service running on the production instance.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo systemctl stop apache2"
  • I removed the existing files from /var/www/html/ for SuiteCRM's previous deployment.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo rm -r /var/www/html/*"
  • I copied over the new files from suitecrm/ to /var/www/html.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo cp -r suitecrm/* /var/www/html"
  • I also copied the config.php file that contained all the configuration created from the first deployment of SuiteCRM on the production instance.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo cp config.php /var/www/html"
  • I changed the owner of the copied files to www-data with sudo chown -R www-data: /var/www/html.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo chown -R www-data: /var/www/html"
  • Next, I restarted the Apache web-server.
ssh -o StrictHostKeyChecking=no ubuntu@<PRODUCTION VM IP> "sudo systemctl start apache2"
  • Lastly, I created a stage in the pipeline to add all the above-mentioned steps for deployment.

Conclusion

Recreating the steps from the pipeline for DVNA to create a similar one for SuiteCRM helped in identifying some assumptions that were not apparent at the time when I made them. Most of it came in the form of tools that were specific to a particular tech-stack as I had to find new tools to perform similar functions in the new pipeline for SuiteCRM.

For SAST, most of the tools I had used for DVNA were incompatible with SuiteCRM as it is written in PHP. I did reuse the few tools that did support multiple languages but also had to find a few new tools to have multiple checks.

For DAST, there was no need to find new tools primarily because of how DAST tools work. However, a difference in this stage arose when I had to deploy SuiteCRM locally for the DAST tools to run a scan on it. Unlike Nodejs, which has built-in development server, I had to make use of Apache as the web-server to deploy the application.

For code quality analysis, no previous tools worked as they all were meant to be used with JavaScript applications. Hence, I had to find different tools that performed lint scans for PHP.

For deploying SuiteCRM, I again had to use Apache as the web-server on the new EC2 instance.

Another inference came in the form of the size of the reports generated as part of the various scans and analysis stages in the pipeline. DVNA, being a small application, generated minuscule reports when compared to the reports generated for SuiteCRM. Since I had initially allotted 10GB of storage to each EC2 instance I was using for the pipelines, the reports filled the entirety of the available disk space and I had to increase the storage capacity. Also, I had not realized how small DVNA was in terms of application sizes and functionality as the total runtime for the pipeline made for DVNA was about 25 minutes whereas the pipeline for SuiteCRM ran for over 4 hours.

In conclusion, I realized that each pipeline has its own quirks based on the application the pipeline is built for. Though, this seems like an obvious inference but the realization by actually experiencing it was more a appreciative way.