Build hangs and time out without any error or feedback

GitLab CI/CD
, , ,
1

I am having a Angular project which have been build and deployed perfectly fine for years but within the last month or so I have seen increasingly failure/timeout of the build process and it has gone worse by the day and now it will never or hardly success in building it.

I’ve added --verbose parameter to my npm install command to get some insights and I do see that it stops at the same line every time. However, I do not see anything wrong with these dependencies. Nor do I experience any issues building it locally.
Both NPM and Node.js is updated to latest versions (7.21.1 & 16.9.1 respectively). I’ve increased the memory on the server that is running Gitlab-runner (to 4GB). I’ve checked the log for Gitlab-runners (journalctl -u gitlab-runner) and nothing is there. I’ve checked logs for NPM and Node, nothing is there.
It hangs until it times out no matter if the time out is 10 minutes or 60 minutes. I’ve cleared the Gitlab-runner cache.
I’ve tried to restart the Gitlab-runner.
In Gitlab CI/CD settings the Runner is marked as active and online.
We are not using shared runner, only specific.

I’m completely out of ideas to what the issue might be and what the solution could be.
The angular code have of coursed changed but the Gitlab CI/CD setup hasn’t changed since last many month.

(P.S. I know there are similar questions but wasn’t able to get any solution from them)

INFO:

  • Gitlab-runner server is running on AWS EC2
  • Memory: 4G
  • Space avail: 19G
  • AWS CodePipeline: using AWS CodePipeline for when Gitlab Build & Deploy is done and uploaded to S3 (I never get to this phase now)

Gitlab-runner & Git info:

  • Version: 10.7.0
  • Git revision: 7c273476
  • Git branch: 10-7-stable
  • GO version: go1.8.7
  • Built: 2018-04-22T13:43:35+00:00
  • OS/Arch: linux/amd64

gitlab-ci.yml 

# Node Image for docker on which code will execute
image: node:latest

# This is the stages / task to perfom in jobs
stages:
  - buildStaging
  - deployStaging

variables:
  BUNDLE_PATH_STAGE: /var/tmp/frontend-ng10-stage.zip    # defines the path for the generated ZIP file

# Job One for making build
buildStaging:
  stage: buildStaging
  script:
    - npm install
    - npm run staging
  artifacts:
    paths:
      - dist/
  only: ['staging']

# Job Two for deploy build to server on staging
deployStaging:
    image: "python:latest"
    stage: deployStaging
    dependencies:
      - buildStaging
    before_script:
    - pip install awscli
    script:
      - apt-get update -y && apt-get install zip
      - zip -r $BUNDLE_PATH_STAGE dist/* appspec.yml scripts
      - aws s3 cp $BUNDLE_PATH_STAGE $AWS_S3_STAGE_LOCATION # Uploads the zipfile to S3 and expects the AWS Code Pipeline/Code Deploy to pick up
    only: ['staging']

Any help or suggestions would be extremely appreciated.

Best,
Emil

2

Hello Emil!

Suddenly I am experiencing the exact same problem. Did you find any solution?