most of the time I’m developing websites with TYPO3 CMS.
After some projects I created a workflow which I’m very happy with:
Commit my files to my local Git
Do some squashing
Pushing to my remote Git
Automated starting of my CI via .gitlab-ci.yml
My .gitlab-ci.yml is very simple and straightforward. It only contains one job which builds and deploys the project. I would love to split up building and deploying into two seperate builds. My problem is, that every build job starts from zero (by checking out the project). Since I decided to choose git clone in my CI/CD settings. So my problem is, that all the things that are done in the build stage are lost in the deploy stage.
I’m absolutely sure, that there is a solution for that
@hirnschmalz , If you don’t mine still I’m not able to figure out how to deploy my app into my local server whenever I merge my code with master, for this do I need to setup any config files or ssh keys, or git runner on my server.
If I understand you correctly you want to know how to deploy your code. I use deployer for this. You can specify your SSH credentials or SSH keys (this is wat I preffer). Let me know if I you have any trouble with it.
Do I need to install runner on my server - if yes while installing should I give the same token and URL i.e ( Specify the following URL during the Runner setup: https://gitlab.com/ci Use the following registration token during setup: MH5QZRf43A7T13Ry**** )
What type of executor can I use, Shell is enough ?
If you want to use your script as shown than I think you have to install the gitlab-runner on the server you want to deploy to. Since I do use my own hosted GitLab instance and do not use any GitLab.com repositories I can not tell you the correct URL for the runner. The token should be shown in the Runners section of the GitLab server (this may be different on GitLab.com and my self hosted installation). A shell executor should be fine and from my point of view you do not need to enter any SSH keys since the gitlab-runner runs on the target server and does a git clone/fetch there.
use Gitlab CI Multi Runner. You can use fetch to make it faster, that’s ok. I recommend running your own CI runner VMs, with a shell runner inside, if you have resources for that.
use stages. You want stages for many reasons. One is so you can see WHERE In the build it failed. Did it fail in compile_dependencies or did it fail in unit_tests or did it fail in integration_tests, or main_app_build, or did it fail at deploy?
add tasks, and assign them to stages. Set what they depend on. Things which can be done in parallel, make part of same stage and not depend on each other. Imagine you want to run “test_suite_a” and “test_suite_b” in parallel, make them separate tasks in the “testing” stage.
Use the “allow_failure” flag for stages that might fail for non-code-errors. Deployment can fail due to network issue or credentials or something else.
use artifacting to pass outputs of one stage to another. Also use artifacting to let you download each build as a zip, and maybe manually deploy it somewhere for testing, or just to “look at yesterday’s build outputs and compare them to todays”.
use caching to speed up repeated builds on languages like C and C++ that generate a lot of .obj files and where their compilers are faster the second time, if you cache the intermediate files. Caching does not pass from one stage to another unless you define both caching and artifacting. On your PHP based projects, caching generally won’t help, because I’m not aware of any “incremental build” features in any PHP frameworks. If I’m wrong there, correct me.
Don’t make stages that last more than 30 minutes. Rule of thumb.
When you have a lot of tests, split your test task into multiple tasks and let them run in parallel.
maybe make production deployment a task, which is Manual? maybe make review/integrations deployment automatic, to a different server?
Due to the nature of our system, we have different repos for different modules of the system, that we then need to checkout independently in order to then compile the top-level application.
Do you have any thoughts or any source of documentation on how to deal with this? Many thanks in advance!
If you have understood stages you would understand what I am suggesting.
If you have two or more git checkout actions required to build one deliverable binary asset (such as an installer, or a tarball of a website to deploy) then you are doing it Wrong.
If one git clone and one script inside that git repo can build one thing which can be then combined with others at some later stage, then you are doing it right.
build-website-part-1
build-website-part-2
combine-part-1-and-part-2-and-ship-combination
Above are some names of stages that should match with a single git repo, or if you really need it, one master git repo with one or more submodules. Unfortunately you can’t automatically build a top level module when ever one of its submodules changes. Do not believe that submodules are your friend. Submodules are NOT your friends. They are your nemesis.
