How to Integrate ShellRun into Your CI/CD PipelineContinuous Integration and Continuous Deployment (CI/CD) pipelines automate building, testing, and delivering software. Integrating ShellRun — a lightweight, secure shell automation tool — can streamline script execution, enforce safety checks, and make pipeline steps more reproducible. This guide walks through planning, configuring, and operating ShellRun inside common CI/CD systems (GitHub Actions, GitLab CI, Jenkins, and CircleCI), with examples, best practices, and troubleshooting tips.
What ShellRun brings to CI/CD
- Consistency: ShellRun standardizes how shell commands run across environments, reducing “works on my machine” issues.
- Security: Built-in sandboxing, permission scoping, and script validation lower the risk of accidental or malicious command execution.
- Observability: Structured logs and exit-code metadata make debugging easier.
- Reusability: Encapsulate common operations (deploy, migrate, test) as ShellRun tasks that are portable between pipelines.
Plan your integration
- Inventory existing shell scripts and pipeline steps that invoke bash/sh commands.
- Classify steps by sensitivity:
- Low-risk: formatting, linting, unit tests.
- Medium-risk: build, dependency installation.
- High-risk: migrations, deployments, secret handling.
- Decide where to replace raw shell calls with ShellRun tasks versus wrapping them in ShellRun for added safety.
- Define access controls for runtime secrets and privileged operations.
- Create a small proof-of-concept pipeline that runs a few representative steps under ShellRun.
ShellRun patterns to use in pipelines
- Task wrapper: Wrap existing scripts to get sandboxing and logging without changing script contents.
- Declarative tasks: Define simple commands and environment needs in a ShellRun manifest (example below).
- Remote execution: For deployment tasks, run ShellRun agents only on trusted runner/agent machines.
- Secrets injection: Use CI secret stores to inject into ShellRun via ephemeral environment variables or secret mounts.
- Retry and cleanup policies: Configure retry counts and guaranteed cleanup hooks for resources created during jobs.
Example ShellRun manifest (task.yml)
# task.yml name: migrate_db command: ./scripts/migrate.sh env: DATABASE_URL: ${DATABASE_URL} timeout: 1800 allowed_paths: - ./migrations secrets: [DB_PASSWORD] cleanup: - ./scripts/rollback_tmp.sh Place manifests in repo under .shellrun/tasks/ to keep them discoverable.
Integrating with GitHub Actions
- Install ShellRun on the runner (self-hosted or GitHub-hosted runner). For GitHub-hosted runners, install at job start:
jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install ShellRun run: curl -sSL https://example.com/shellrun/install.sh | bash - name: Run tests via ShellRun env: DATABASE_URL: ${{ secrets.DATABASE_URL }} DB_PASSWORD: ${{ secrets.DB_PASSWORD }} run: shellrun run ./.shellrun/tasks/test_suite.yml - For faster runs, cache the ShellRun binary or use a pre-built Action that bundles ShellRun.
Integrating with GitLab CI
- Use the image or install ShellRun in the before_script. Example .gitlab-ci.yml:
stages: - test - deploy variables: SHELLRUN_CACHE: "$CI_PROJECT_DIR/.cache/shellrun" test: stage: test image: ubuntu:22.04 before_script: - apt-get update && apt-get install -y curl - curl -sSL https://example.com/shellrun/install.sh | bash -s -- --dest $SHELLRUN_CACHE script: - $SHELLRUN_CACHE/shellrun run ./.shellrun/tasks/test_suite.yml secrets: DATABASE_URL: $DATABASE_URL - Use protected runners for high-privilege ShellRun tasks.
Integrating with Jenkins
- Install ShellRun on Jenkins agents (or as part of the pipeline). Example Declarative Pipeline snippet:
pipeline { agent any environment { DATABASE_URL = credentials('db-url') } stages { stage('Install ShellRun') { steps { sh 'curl -sSL https://example.com/shellrun/install.sh | bash' } } stage('Run ShellRun task') { steps { sh 'shellrun run ./.shellrun/tasks/migrate_db.yml' } } } } - Use Jenkins credentials store to inject secrets and limit which jobs/agents can run ShellRun tasks that access production systems.
Integrating with CircleCI
- Install ShellRun in the job, or use a custom Docker image with ShellRun preinstalled. Example:
version: 2.1 jobs: build: docker: - image: cimg/base:stable steps: - checkout - run: name: Install ShellRun command: curl -sSL https://example.com/shellrun/install.sh | bash - run: name: Run ShellRun task command: shellrun run ./.shellrun/tasks/build_and_test.yml - Use CircleCI contexts to share secrets safely across projects.
Secrets management
- Don’t store secrets in repo. Use CI secret stores (GitHub Secrets, GitLab CI Variables, Jenkins Credentials, CircleCI contexts).
- Prefer ephemeral environment variables or short-lived tokens.
- For extra safety, configure ShellRun to read secrets from a secure vault only when running on trusted agents.
Logging, observability, and artifact handling
- Configure ShellRun to emit structured logs (JSON) so CI systems can parse test results and statuses.
- Upload artifacts (build outputs, coverage reports) using native CI artifact mechanisms after ShellRun completes.
- Set retention policies for logs and artifacts to balance debugging needs and storage costs.
Testing and rollback
- Add a dedicated pre-deploy stage that runs ShellRun tasks against a staging environment.
- Use canary or blue/green deployment patterns where ShellRun handles targeted rollout commands.
- Implement cleanup and rollback tasks in ShellRun manifests and call them from pipeline failure hooks.
Access control and least privilege
- Limit which runners/agents can execute ShellRun tasks that access production systems.
- Run ShellRun under a non-root user on agents where possible.
- Use ShellRun’s allowed_paths and capability flags to restrict file access and system calls.
Performance and caching
- Cache the ShellRun binary or use an image with it installed to reduce setup time.
- Cache dependency directories (node_modules, .venv) as usual; ShellRun tasks should reference those caches.
- For parallel jobs, ensure ShellRun tasks that modify shared resources coordinate to avoid race conditions.
Example full GitHub Actions workflow (short)
name: CI on: [push, pull_request] jobs: ci: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install ShellRun run: curl -sSL https://example.com/shellrun/install.sh | bash - name: Run test task env: DATABASE_URL: ${{ secrets.DATABASE_URL }} run: shellrun run ./.shellrun/tasks/test_suite.yml - name: Upload artifacts uses: actions/upload-artifact@v4 with: name: test-results path: ./artifacts/test-results Troubleshooting common issues
- Fails to find shellrun: ensure binary is installed and on PATH; prefer caching or prebuilt images.
- Secrets not available: confirm secret names and pipeline variable scoping (job vs environment).
- Permission errors on runners: confirm user permissions and allowed_paths in manifest.
- Long-running tasks time out: increase job timeout or set ShellRun task timeout appropriately.
Best practices checklist
- Store ShellRun manifests in a dedicated directory (.shellrun/tasks).
- Use protected runners for production tasks.
- Inject secrets only at runtime; never commit them.
- Pre-install ShellRun in reusable CI images or cache it.
- Add cleanup and rollback steps for side-effectful tasks.
- Emit structured logs for easier debugging and monitoring.
If you want, I can: provide ready-to-use ShellRun manifests for your specific repo layout, convert an existing pipeline to use ShellRun step-by-step, or produce a Docker image with ShellRun preinstalled.
Leave a Reply