CircleCI Field Guide
GitHub Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Policy as Pipeline

Policy Automation

Policy Management continues our embrace of Config-as-Code, with policies defined in Open Policy Agent’s rego text files.

These files can be applied by org admins via the CLI.

However our field team highly encourages the use of a priveleged pipeline to ensure consistent application of policy.

This recipe uses policy testing as an ingredient.

The Flow

  • Policy Admins author new policy rules or changes
  • Policy changes are commited to piveleged repository, non-default branch
  • [OPTIONAL] - job update_ids will sync data with external system
    • Project IDs pulled from CircleCI API so rules use clear variable names.
    • Policy Assignment are pulled from external system like GitHub, Compass, Service Center, etc
    • Project IDs and Policy Groupings (as rego) are commited via CI automation to feature branch
  • Policy-as-code is tested against sample policies to ensure desired output
  • Changes are approved via a PR.
  • Merging to default branch and the tested policies are applied.

The Config

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
version: 2.1
orbs:
  cli: circleci/circleci-cli@0.1.9

commands:
  list-policies:
    description: "List active policies in org"
    parameters:
      org-id:
        type: string
        default: "efc130dc-284f-4533-964e-844f5c173860"
      file_suffix:
        type: string
        default: ""
    steps:
      - run: circleci policy fetch --owner-id << parameters.org-id >>
      - when:
          condition: << parameters.file_suffix >>
          steps:
            - run: mkdir -p /tmp/policy_debug
            - run: circleci policy fetch --owner-id << parameters.org-id >> > /tmp/policy_debug/policies_<< parameters.file_suffix >>.json
            - store_artifacts:
                path: /tmp/policy_debug/
    
  apply-policy-bundle:
    description: "Apply all AweseomCICD policies as single org wide bundle"
    parameters:
      org-id:
        type: string
        default: "efc130dc-284f-4533-964e-844f5c173860"
    steps:
      - run: circleci policy push ./applied_policies --owner-id << parameters.org-id >> --no-prompt

  test-policies:
    description: "Run EVAL on policies to ensure no code failures (which breaks policy enforcement)"
    steps:
      - run: 
          name: Setup environment for pytest
          command: |
            pip3 install pytest pyyaml
            mkdir -p tests/results            
      - run: 
          name: Run Tests
          command: pytest tests --junitxml=tests/results/junit.xml
      - store_test_results:
          path: tests/results


  #OPTIONAL: If using dynamic assignment, commit any relevant variables now.
  update-ids:
    steps:
      - run: 
          name: Setup environment for scanner
          command: |
                        pip3 install -r scripts/requirements.txt
      - run: 
          name: Scan for latest Project IDs and Mappings
          command: |
            export SCANNER_CIRCLECI_TOKEN=${CIRCLECI_CLI_TOKEN}
            export SCANNER_GITHUB_TOKEN=${GITHUB_TOKEN}
            python3 scripts/scanner.py --topic-prefix 'policy-' --organization-id efc130dc-284f-4533-964e-844f5c173860 --file-path applied_policies            
      - store_artifacts:
          path: scanner.log
          when: always
      - run: 
          name: Capture Diff
          command: git diff | tee scanner_diff.txt
      - store_artifacts:
          path: scanner_diff.txt
      - add_ssh_keys
      - when: 
          condition: 
            not:
              equal: ["main", << pipeline.git.branch >>]
          steps:
            run:
              name: Push Latest to Branch
              command: |
                git config --global user.email "solutions@circleci.com"
                git config --global user.name "AwesomeCICD PolicyBot"
                git push --set-upstream origin << pipeline.git.branch >>
                git add applied_policies/ && git commit -m'[skip ci] auto commit from CI Policy Pipeline' && git push                
    
      

jobs:
  apply-policies:
    executor: cli/default
    steps:
      - checkout
      - cli/install
      - cli/setup
      - list-policies:
          file_suffix: before
      - apply-policy-bundle
      - list-policies:
          file_suffix: after

  test-policies:
    docker:
      - image: cimg/python:3.8
    steps:
      - checkout
      - cli/install
      - cli/setup
      - update-ids #optional job to use dynamic policy assigment from external source (SC, CMDB, etc)
      - test-policies

workflows:
  se-policies:
    jobs:
      - test-policies
      - apply-policies:
          requires:
            - test-policies
          filters:
            branches:
              only: [ main ]