Intro
This year I set a goal to explore and learn most popular CI solutions like GitLab, including all cloud native ones. My first pick had to be GitHub Actions, which I heard a lot of but never used. So I decided to plunge into it to see how rich the solution was. Since I only use CI for terraform deployments, I aimed to cover the features not the dev part.
This post is rather a long-mixed pack of titbits & notes out of my 2 weeks immersion, highlighting the particularities and features of GitHub Actions in the CI/CD landscape, with no specific order. This will hopefully get you started with GitHub Actions, whether you're familiar with CI or just a curious newcomer.
Table of contents
GitHub Actions basics
As most of you know, GitHub Actions is a platform that allows developers to automate workflows and tasks directly within GitHub repositories. Below is a brief description of key components of GitHub Actions:
Workflows: Configurable, automated process that executes one or more actions and defined by YAML files checked into the repository under .github/workflows.
Runner: a GitHub or self-hosted vm used to run your workflow with a list of tools preinstalled.
Triggers: Workflows can be triggered by specific events, such as push, pull requests, or scheduled times.
Actions: Are prebuilt tasks written in multiple languages that can be called in a workflow.
Marketplace: Where developers can discover, and use actions and workflows created by the community.
Secrets: Allow Devs to store and use sensitive information (credentials) securely in their workflows.
Integration: integrates with a numerous tools and services, to streamline dev & deployment workflows.
GitHub plans: Personal (GitHub Free or GitHub Pro), Organizations (GitHub Team, GitHub Enterprise)
Ok, time to unveil the coffers of valuable git bits, curated from 35+ tabs of doc during my 2 weeks immersion.
Buckle up!
1. Default Runners
GitHub Actions has a variety of default runners with 3 OS allowing you to run your workflows on different platforms
- Ubuntu: 20, 22.04 LTS
- MacOS: 11,12,13
- Windows: 2019,2022
Runners are vms that come with preinstalled software which allows to run any app or code in your pipeline.
Bash, Node.js, Perl, Python,Ruby, Swift, Dash, C++, Julia, kotlin, Mono, MSbuild,…
Pip/pip3, Cpan, Helm,Yarn, Homebrew, Miniconda, Npm, NuGet, Pipx, RubyGems, Vcpkg,…
Git,SVN, Ansible,Packer, Terraform, Pulumi, Kubectl,Minikube, R, Heroku, Docker, Apt-fast, AzCopy,…
AWS CLI, Azure CLI, GitHub CLI, Google Cloud SDK, Alibaba Cloud CLI, Hub CLI, OpenShift CLI, ORAS CLI, Vercel CLI
Other
JAVA, .NET Tools, Cached tools (Go, Node.js, Python, Pypy, Ruby), Cached Docker Images
PHP Tools, Haskell Tools, Rust Tools, PowerShell Tools and modules
Browsers: Google Chrome, chromium, Microsoft edge, Mozilla Firefox, Solenium server
Databases: sqllite, MySQL, PostreSQL, MSSQL tools (sqlcmd,sqlPackage)
Webservers: Nginx, Apache2
Mobile OS: Android Command Line Tools, Android Emulator, Android SDK platforms, Google play services, Google repo…
2. Basic Structure of a Workflow
The basic structure of a GitHub Actions workflow consists of triggers, jobs, workflow syntax, and commands.
Triggers define the events that can trigger a workflow, such as pushes to the repository or pull requests
Jobs define the individual tasks or steps that need to be executed as part of the workflow
Example
Name: Workflow name
on: Events that trigger a workflow (push on the branch git_actions for any change in paths section)
env: Variables definition (hardcoded or imported from the environment such as secrets, vars)
Permissions: To allow your actions to use the token_id
Jobs section:
JobName, runner OS (runs-on) , environment, default shell and working directory
Steps section: checkout your code repo + other steps(run tests, build artifacts..)
Same for the next job…
3. How are public Actions used?
As show below, these prebuilt tasks can be easily called in a workflow and are written in multiple languages. But public actions can’t be referenced from self-hosted runners.They are publicly stored in GitHub Market place.
# example: “setup-node” action that downloads node.js and add it to the PATH
steps:
...
- uses: actions/setup-node@v3
with:
node-version: 18
4. Are public actions safe?
Public GitHub actions can also be risky to your security and privacy. For instance, malicious actions could steal your secrets, modify your code, or compromise your server. Even if the actions are not malicious, they could have vulnerabilities, or outdated dependencies that could affect your project.
Read more in this stackoverflow thread
Personally I don’t have time to check random people’s code for backdoors, so I only use actions from trusted sources, like official authentication actions from major cloud platforms and those made by GitHub.
5. Difference between Public & Private Repositories
Beware : GitHub Actions logs of your public repo are visible to anyone as opposed to private ones.
For access to environments, environment secrets, and deployment branches in private or internal. repositories, you must use GitHub Pro,Team, or GitHub Enterprise.
Actions and reusable workflows stored in private repositories cannot be used in public repositories.
GitHub doesn’t allow individual accounts to use self-hosted runners on public repositories.
You can share actions and reusable workflows from your private repo without making them public, by allowing GitHub workflows to access a private repository that contains the action or reusable workflow.
6. GitHub Environment
Environment is an abstraction allowing to differentiate deployments(dev,prod,staging), prevent unauthorized deployments, preserve secrets, track changes and much more.
It’s Available for all public repositories and private repositories for Pro,Team, and Enterprise accounts
Referencing environment in Git actions has 3 scopes:
The entire workflow, by using env at the top level of the workflow file.
The contents of a job within a workflow, by using jobs.<job_id>.env.
A specific step within a job, by using jobs.<job_id>.steps[*].env.
7. Environment Files
You can share custom environment
variable with any subsequent steps in a workflow job by defining or
updating the environment variable and writing this to the GITHUB_ENV environment
file.
-
echo
"{environment_variable_name}={value}" >> "$GITHUB_ENV"
For sharing environment between jobs or
between workflows you will need GITHUB_OUTPUT
8. GitHub Actions contexts & Variables
You can have repository, environment, as well as intra-workflow (job,step) variable level.
Context is a collection of variables describing workflow runs, runner environments, jobs, steps, secrets & much more.The context reference syntax is ${{ context.variable }}.
-
github reference information
about the workflow run & events that triggered the run. i.e github.repository
-
env
Reference environment variable
defined in the workflow.i.e
${{ env.MY_VARIABLE }}
-
vars context to access configuration variable values ${{ vars.MY_VAR
}}
There’s a similar collection of variables called default variables within a runner (i.e GITHUB_RUN_ID).
If you want to see all the information that GitHub Actions has in a context, use the handy toJson function
Runner env variables are always interpolated on the runner vm. However, parts of a workflow are processed by GitHub Actions and are not sent to the runner
9. GitHub Actions Security (Secrets)
In GitHub Actions, secrets are used to store sensitive information like passwords, API keys, tokens etc.
There are organization, repository and environment level secrets.
In case of conflict, organization is overridden by repository and repository is overridden by env values.
Secret names can’t contain spaces, not case-sensitive, & must be unique within the same level.
Secret names must not begin with the prefix GITHUB_.
secrets.GITHUB_TOKEN is a temporary token for each workflow run.
Exploitability and impact of untrusted input is real, read more in this excellent GitHub Securitylab post
How safe is it on public repos and forks
Secrets are safe to use in public repositories as they are automatically masked in build logs & show as *
But If in your workflow you create a credential from a secret (e.g. base64 an API key) then you should mask the new value so it doesn't leak in the build log.
With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository.
Secrets are not automatically passed to reusable workflows.
If you have a public repository, make sure all outside collaborators' PR require approval.
What is a mask and is it really useful?
You can mask, or hide, any sensitive data in GitHub Actions logs by adding a new step add-mask in a Workflow.
⛔ Unfortunately a variable will still be visible at least once before a mask is applied to it, and from then that value cannot be passed between runners. More examples here
10. Job dependency (Needs)
needs context: contains outputs from all jobs that are defined as a direct dependency of the current job.
By default all your jobs run in parallel in a workflow, but you often need them to run sequentially.
Use needs to add dependency between jobs
Other needs variables
needs.<job_id> : A single job that the current job depends on
needs.<job_id>.outputs : outputs of a job that the current job depends on.
needs.<job_id>.result: The result of a job that current job depends on. Possible values are success, failure, cancelled, or skipped.
11. Artifact vs. Caching
Artefacts:
Allow you to share data between running jobs and save them after the workflow is complete.
An artifact is a file or collection of files produced during a workflow run.
Difference
Artifacts are used to save files after workflow ended. (handy for : Logs, manifest, statefile, config file, Tests Results, Reports, etc)
Caching is used to re-use data/files between jobs or workflows (i.e sharing build dependencies files that don't change often between jobs) 
Beware: Both artifact and Caches are accessible to anyone in public repositories. Artifact can be downloaded, cache can be reused.
12. Approval Triggers best practice
The default behavior of environment protection rules is to set manual approval for first-time contributors.
In that case, an attacker could create a simple and innocent pull request (like documentation update).
When accepted, his subsequent pull request could be malicious and automatically trigger the workflow.
So you need to set “Require approval for all outside collaborators” to ensure a more robust defense.
Private Manual approval: use below action if you don’t have an Enterprise/Pro account but still want manual approval without the use of environments.
13. Random Actions Tips
Step ID vs step name:
ID is used as a reference, from other jobs or steps (i.e, in jobs. <job_id>. needs ).
Name is used for display purposes on GitHub.
How to run commands without specifying a step name: run: echo "Run my shell command "
Why install custom software via actions vs use local one in the runner (example setup-terraform):
terraform_setup action downloads the right version for you to ensure that updates to your infrastructure are safe and predictable (required_version).
Local version of Terraform might be too new for the required version, and will fail terraform init
Conclusion:
That’s it, a long but very useful cheat sheet that helped me and hopefully helps you explore GH Actions
Again if your workflow is in a public repo, remember below safety measures ⏬
Don't have any workflow with a pull_request trigger and never plan to make one.
Have the "Require approval for all outside collaborators" option for GitHub Actions turned on.
You must only invite trustworthy outside collaborators.
Next, I will write a series about Multicloud terraform deployments in GitHub actions
Stay tuned
No comments:
Post a Comment