Joint Forces: AWS and Docker

The GitLab CI/CD and its partners offer an extensive library of templates, components, and ready-to-ship solutions to automate your application builds and deployments. But real-life cases make you search for non-trivial scenarios. Here is a simple one that may save you days down the road.

I have a cloud-agnostic Python app that can be deployed to various platforms, including AWS. The source code resides in the GitLab private repo, and it's only natural to use the project's container repository as a single source of the code. A simple diagram is often more effective than a lengthy description of the image's building and flow.

The steps sound easy:

• Build an image with the appropriate GitLab and Amazon ECR tags
• Push it to the GitLab project registry.
• Use AWS CLI to access Amazon ECR
• Pull it using the ECR tags.

Yet there is one complication: the default CI/CD Docker image does not have the AWS client installed. The standard AWS image offers Docker binaries, but it fails because the service is not available. A few trials and errors led to the solution below:

build_application:
  stage: build
  image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
  services:
  # Enable Docker-in-Docker service to allow Docker
  # commands inside the container
      - name: docker:dind  
        alias: docker
  variables:   
    FF_SCRIPT_SECTIONS: "true"  
    DOCKER_HOST: tcp://docker:2375
    DOCKER_DRIVER: overlay2
    DOCKER_TLS_CERTDIR: ""
  before_script:
    # Log in to the Docker registry using GitLab CI token
    - |
        echo "${CI_JOB_TOKEN}" | docker login -u "gitlab-ci-token" \
        --password-stdin "${CI_REGISTRY}"
    # Login to the ECR 
    -| 
       aws ecr get-login-password \
       --region "${AWS_DEFAULT_REGION}" | docker login --username AWS \
       --password-stdin \
       $AWS_ACCOUNT.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com        
  script:
    # Build the Docker image using the 
    - docker build --pull -t "${GLB_TAG}" .
    # Push image to the registry
    - docker push "${GLB_TAG}"
    # Pull thorugh the ECR cache.    
    - docker pull "${ECR_TAG}"

Let's walk it from top to bottom:

1. For a build task, use a GitLab-provided AWS image. Docker binaries are included, but the service is not running.
2. Enable a Docker-in-Docker service in the services section, with the alias docker. Later commands will use it as the hostname.
3. Define the container environment variables for Docker commands.
4. Log in to the GitLab and Amazon registries.
5. Build the application image.
6. Push it into the GitLab project registry.
7. Use the ECR pull-through tag to update the image cache on the AWS side.

My inner perfectionist tells me that the image in GitLab should be sufficient; that it's only a matter of AWS access policies and UI tricks. Still, sometimes a simple workaround can be more helpful.