Centralized Notifications Service

Introduction

This Centralized Notification Service is designed to streamline your development process by providing immediate alerts when CircleCI workflows fail. This proactive notification system helps teams identify and address issues quickly, reducing downtime and improving overall productivity.

Key Benefits

Reduced Resolution Time: Decreases time-to-fix by alerting the responsible developer immediately
Enhanced Team Communication: Delivers critical alerts to your team’s Slack channel
Improved Accountability: Links directly to the failed workflow for quick access
Low Maintenance: Once configured, operates automatically with no manual intervention required
Centralized System: Provides a centralized notifications service that can be used across all projects on CircleCI

How It Works

When a CircleCI workflow fails, the system captures essential information about the failure
The system identifies the commit author responsible for the change
A personalized notification is sent to the developer with workflow details and a direct link
A notification is also posted to your designated Slack channel for team visibility
The system intelligently filters notifications, only alerting on actual failures

Who is this for?

This concept is better suited towards orgs that have a current OAuth integration, and are layering Github App usage on top of this
Those familiar with the Slack orb

Prerequisites

A CircleCI account with admin access
A Slack workspace with permissions to add integrations
Basic familiarity with YAML configuration files
A repository, which has been authorised to use the Github App integration on CircleCI

Step 1: Set Up Slack Integration

To get started with the Slack integration, check out this tutorial

Step 2: Create the repo on Github and CircleCI

Create a repo on Github which will be used for central notifications management

Step 3: Configure the Webhook Notification Job

Add the following job to a config file in the centralized notifications repo e.g. .circleci/notifications.yml:

   
    1
    2
    3
    4
    5
    6
    7
    8
    9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   
   
   version: 2.1
   
   orbs: 
     slack: circleci/slack@5.1.1
   
   jobs:
     notify-on-fail:
       docker:
         - image: cimg/base:stable
       resource_class: small
       steps:
         - run: 
             name: Process webhook and prepare notification
             environment:
               PAYLOAD: << pipeline.trigger_parameters.webhook.body >>
             command: |
               export WORKFLOW_NAME=$(echo "$PAYLOAD" | jq -r '.workflow.name')
               export WORKFLOW_STATUS=$(echo "$PAYLOAD" | jq -r '.workflow.status')
               export LATEST_REVISION=$(echo "$PAYLOAD" | jq -r '.pipeline.vcs.revision')
               export COMMIT_AUTHOR=$(echo "$PAYLOAD" | jq -r '.pipeline.vcs.commit.author.email')
               export WORKFLOW_URL=$(echo "$PAYLOAD" | jq -r '.workflow.url')
   
               if [ "$WORKFLOW_STATUS" = "failed" ]; then
                   echo -e "Hi $COMMIT_AUTHOR.\nThe workflow '$WORKFLOW_NAME' completed with the status '$WORKFLOW_STATUS'.\nYou can view this workflow here: '$WORKFLOW_URL'"
                   # failing in order to trigger the Slack notification
                   exit 1
               else
                   echo "workflow did not fail. No need to send a notification"
               fi
         - slack/notify:
             channel: YOUR_CHANNEL_ID_HERE  # Replace with your actual channel ID or name
             template: basic_fail_1
   
   workflows:
     webhook-workflow:
       jobs:
         - notify-on-fail

   

Replace YOUR_CHANNEL_ID_HERE with your actual Slack channel ID or name

This config uses the payload which is sent from the outbound webhook service within CircleCI, and is available within the << pipeline.trigger_parameters.webhook.body >> pipeline value. An example of this payload can be found here.

Step 4: Set Up a pipeline and trigger in the Centralized Notifications Repo

Add a pipeline, which links to the new config file above.
Add a trigger
- The Trigger type will be Custom Webhook
- Make note of the Webhook URL, as this will be required when creating the outbound webhook later

Step 5: Set Up CircleCI Outbound Webhook from another CircleCI project

Now that an inbound webhook trigger has been created, the generated URL can be used to trigger that specific pipeline.

To trigger this pipeline from another project on CircleCI, set up a webhook from the project you want to notify on. To do this:

Go to your CircleCI project settings
Navigate to “Webhooks” and click “Add Webhook”
Configure the webhook:
- Name: Workflow Status Notification
- Receiver URL: Your CircleCI API endpoint (format: https://circleci.com/api/v2/webhook/[your-token])
- Events: Select “Workflow Completed”
- Certificate Verification: Enabled

Step 6: Test the Integration

Trigger a Failed Workflow
- Push a commit that will intentionally fail a CircleCI workflow
- Watch for the notification in your configured Slack channel
Verify the Notification
- Check that the notification includes:
  - The workflow name
  - The status (failed)
  - The commit author’s email
  - A link to the workflow

Step 7: Further Customization (Optional)

Customize the Slack Message Template
- CircleCI’s Slack orb supports custom templates
- Modify the template parameter or create a custom template for more detailed notifications
Add Additional Notification Channels
- You can configure email notifications or integrate with other platforms by adding more steps to the job
Enhance the Payload Processing
- Add more information to the notification by extracting additional fields from the webhook payload
- Customize the format of the notification message

Troubleshooting

Webhook Not Triggering: Verify your webhook URL and token are correct
Missing Payload Data: Check the webhook payload structure with echo "$PAYLOAD" | jq in your job
Slack Notifications Not Working: Verify your Slack token has the correct permissions