Hello,
I have a Gitlab pipeline for building IaC with Terraform/Terragrunt. I would like to get the pipeline job’s logs so I can search in the logs for the string “Warning: Argument is deprecated” and then get the next 4 lines after this string. In the job logs they appear like this:
# (7 unchanged attributes hidden)
}
Plan: 0 to add, 2 to change, 0 to destroy.
Warning: Argument is deprecated
with aws_msk_cluster.default[0],
on main.tf line 70, in resource "aws_msk_cluster" "default":
70: ebs_volume_size = var.broker_volume_size
use 'storage_info' argument instead
─────────────────────────────────────────────────────────────────────────────
Saved the plan to: plan
To perform exactly these actions, run the following command to apply:
terraform apply "plan"
I started creating a python script with Python-Gitlab Pipelines and Jobs - python-gitlab v4.1.0 .
I obtained the pipeline jobs from the project id and pipeline id. Then I select in a new list the jobs ids only with stage = “plan”. The I iterate over this list and run gitlab.v4.objects.ProjectJob.trace for each job id.
trace_job = single_job.trace(job_id)
print(trace_job)
Then I print or pprint the trace_job. But the output is not very human friendly or it seems difficult to extract what I need (the line with “Warning: Argument is deprecated” and the next 4 lines after).
- Is there a way to format in a more friendly way the output of print(trace_job) so to extract the line with “Warning: Argument is deprecated” and the next 4 lines after?
- Would you have other suggestions on how to get the desired lines/strings from the job log using python-gitlab? I mention that CLI reference (gitlab command) - python-gitlab v4.1.0 shows exactly as in Gitlab GUI logs but not sure on how to integrate with python.
- Let’s say the formatting may not be an issue (I am not sure, I am not very familiar with Python), how do you suggest to extract the desired lines of logs, that is the line with “Warning: Argument is deprecated” and the next 4 lines after?
Thank you.
Example of output:
