As cloud infrastructures grow, so do the complexities and the need for centralizing for commonly used resources. As Azure Cloud Integration Engineers, this is doubly so with the data and resources for the many communication partners there could be.
I’ve found this to be the same as well in a CI/CD environment. In this blogpost, I will show you how you can set up an Azure DevOps project containing common resources such as scripts (used in this blogspot), ARM templates, pipeline templates, mappings, etc.
The interesting part is consuming the artifacts this project generates in another. We’re going to create this second project and take a closer look at the important steps to take to ensure an efficient CI/CD environment for all your future Azure endeavors!
Let’s get into it!
Common Resources project
I have created a project called Common Resources, containing a repository ScriptMaster which holds its own pipeline (Deployment.yml) and a dummy script (Netic-FakeScript.ps1) which can be downloaded from this zip.
Looking at the pipeline file, the important parts are the trigger and the PublishPipelineArtifact task. Take note of the name used for the artifact.
When a change is made to the main branch, ideally only via a pull request, a trigger for the pipeline is executed, creating the artifact. This ensures that pipelines consuming the artifact always have the latest, stable version of the scripts.
In Project settings > Pipelines > Settings, set the number of recent runs to retain per pipeline to any number bigger or equal to 1.
To guarantee that an artifact is always available, the branch chosen to trigger builds needs to be the default branch or a protected branch (meaning that a branch has any branch policies configured) so that the above setting applies to it.
For more info on retention policies, see set-run-retention-policies
Create a pipeline based on the Deployment.yml file and let it run once. Take note of the name you give the pipeline.
Consuming project
I have created another project called The Great Consumer containing one pipeline which can be downloaded from this zip.
We will look at what needs to happen in the pipeline and project in order to get The Great Consumer to actually consume artifacts from other Azure DevOps projects.
Pipeline
The reference to the artifact in another project is made here. Deployment.yml has a resources item that defines sources of pipelines, builds, repositories, containers, packages and webhooks. We will only define one pipeline resource for this example.
For more info on all the possibilities with the resources/pipeline items, visit Define resources in YAML.
The alias used here will be used in the stages to refer to the pipeline resource.
I have also added a variable that defines where the artifact will be downloaded and one with the name of the external artifact to centralize the DevOps information. By default, it is downloaded in $(Pipeline.Workspace)/[PipelineAlias]/[ArtifactName].
As of writing this, I have not yet found a way to target the alias without using the shorthand download task, which does not allow setting paths.
Now you just need to download this artifact in the desired stage and call it in a desired stage as shown in the Artifact yaml file. Sadly, this won’t work yet as we still have the project settings to update.
Project settings
Navigate to Project Settings > Pipelines > Settings in the consuming project.
The first setting in the screenshot –
Limit job authorization scope to current project for non-release pipelines – allows pipelines, or rather the agent running the job, to access other projects in your organization. Normally, this would be turned on for security. Turn it off.
Luckily, the second setting shown –
Limit job authorization scope to referenced Azure DevOps repositories – denies the pipeline access to everything except for projects that were explicitly referenced. If an agent would be breached that was running your job, the breach would stay contained.
These settings can be set on organization level and appear greyed out.
For more info on authorizations, see limit-job-authorization-scope
That’s all!
Create the pipeline, run it and bask in your greatness.
Quick recap
The recap assumes you have experience and should be enough paired with the screenshots.
- Create a pipeline that produces a common resource
- Take note of the pipeline and artifact name you have chosen here
- In a consuming project
- Add the pipeline resource item in your pipeline, using the names of the previous step
- Update the two settings on project level
Source code
Scripts used in this blog can be retrieved from:
Azure, Azure DevOps, CI/CD, DevOps, YAML
