Gitlab CI for Android project

Hello guys,

I am new to the CI, and Here is what I have setup for my android project with specific runner (I setup specific runner from gitlab docs for CI)

my gitlag-ci.yml:

# from : http://www.greysonparrelli.com/post/setting-up-android-builds-in-gitlab-ci/
before_script:
  - export ANDROID_HOME="/home/demo/project/android/android_sdk"
  - export JAVA_HOME="/usr/lib/jvm/java-8-oracle/"
  - chmod +x gradlew

dev:
  script:
    - ./gradlew jar
    - ./gradlew assembleDebug

And here is the build logs

Running with gitlab-ci-multi-runner 1.8.0~beta.30.g5d23907 (5d23907)
Using Shell executor...
Running on trial...
Fetching changes...
HEAD is now at 9adf3b5 Updated gradle related things in .gitlab-ci.yml
From https://gitlab.com/android-tks-libraries/material-design-libraries
   9adf3b5..7831b20  master     -> origin/master
Checking out 7831b20d as master...
$ export ANDROID_HOME="/home/demo/project/android/android_sdk"
$ export JAVA_HOME="/usr/lib/jvm/java-8-oracle/"
$ chmod +x gradlew
$ ./gradlew jar
Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain
ERROR: Build failed: exit status 1

I’m unable to understand why this fails?


My main module specific build.gradle

apply plugin: 'com.android.application'

android {
    compileSdkVersion 24
    buildToolsVersion '24.0.3'
    defaultConfig {
        applicationId "com.package.app"
        minSdkVersion 19
        targetSdkVersion 24
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    productFlavors {
    }
}

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:support-annotations:24.2.1'
    compile 'com.android.support:support-vector-drawable:24.2.1'
    compile 'com.android.support:design:24.2.1'
}

Can any one help me to use Gitlab CI for Android Projects…

If you ssh into that machine named trial and run the same .\gradlew jar command it works?

Looks like a system PATH or CLASSPATH issue?

Or perhaps like you would have to do in Jenkins, Special steps are required to get gradle wrappers working in CI?

Like this:

https://www.mkyong.com/gradle/jenkins-could-not-find-gradlewrappermain/

Also check out this blog post:

https://t.co/LSWaesAd7b

1 Like

Hi @warren.postma

Thanks for reply.

Here is the output :

demo@trial:~/project/android/demos/material-design-libraries$ ./gradlew jar
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: The Gradle Daemon - Gradle User Guide Version 2.14.1.
Observed package id ‘extras;android;m2repository’ in inconsistent location ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’ (Expected ‘/home/demo/project/android/android_sdk/extras/android/m2repository’)
Already observed package id ‘extras;android;m2repository’ in ‘/home/demo/project/android/android_sdk/extras/android/m2repository’. Skipping duplicate at ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’
Observed package id ‘extras;android;m2repository’ in inconsistent location ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’ (Expected ‘/home/demo/project/android/android_sdk/extras/android/m2repository’)
Already observed package id ‘extras;android;m2repository’ in ‘/home/demo/project/android/android_sdk/extras/android/m2repository’. Skipping duplicate at ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’
Incremental java compilation is an incubating feature.
Download https://jcenter.bintray.com/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.pom
Download https://jcenter.bintray.com/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar

FAILURE: Build failed with an exception.

  • What went wrong:
    Task ‘jar’ is ambiguous in root project ‘material-design-libraries’. Candidates are: ‘jarDebugClasses’, ‘jarReleaseClasses’.
  • Try:
    Run gradlew tasks to get a list of available tasks. Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 3 mins 41.032 secs

When I tried with Tasks :

demo@trial:~/project/android/demos/material-design-libraries$ ./gradlew tasks
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: The Gradle Daemon - Gradle User Guide Version 2.14.1.
Observed package id ‘extras;android;m2repository’ in inconsistent location ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’ (Expected ‘/home/demo/project/android/android_sdk/extras/android/m2repository’)
Already observed package id ‘extras;android;m2repository’ in ‘/home/demo/project/android/android_sdk/extras/android/m2repository’. Skipping duplicate at ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’
Observed package id ‘extras;android;m2repository’ in inconsistent location ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’ (Expected ‘/home/demo/project/android/android_sdk/extras/android/m2repository’)
Already observed package id ‘extras;android;m2repository’ in ‘/home/demo/project/android/android_sdk/extras/android/m2repository’. Skipping duplicate at ‘/home/demo/project/android/android_sdk/temp/ExtraPackage.old01’
Incremental java compilation is an incubating feature.
:tasks


All tasks runnable from root project

Android tasks

androidDependencies - Displays the Android dependencies of the project.
signingReport - Displays the signing info for each variant.
sourceSets - Prints out all the source sets defined in this project.

Build tasks

assemble - Assembles all variants of all applications and secondary packages.
assembleAndroidTest - Assembles all the Test applications.
assembleDebug - Assembles all Debug builds.
assembleRelease - Assembles all Release builds.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
clean - Deletes the build directory.
compileDebugAndroidTestSources
compileDebugSources
compileDebugUnitTestSources
compileReleaseSources
compileReleaseUnitTestSources
mockableAndroidJar - Creates a version of android.jar that’s suitable for unit tests.

Build Setup tasks

init - Initializes a new Gradle build. [incubating]
wrapper - Generates Gradle wrapper files. [incubating]

Help tasks

buildEnvironment - Displays all buildscript dependencies declared in root project ‘material-design-libraries’.
components - Displays the components produced by root project ‘material-design-libraries’. [incubating]
dependencies - Displays all dependencies declared in root project ‘material-design-libraries’.
dependencyInsight - Displays the insight into a specific dependency in root project ‘material-design-libraries’.
help - Displays a help message.
model - Displays the configuration model of root project ‘material-design-libraries’. [incubating]
projects - Displays the sub-projects of root project ‘material-design-libraries’.
properties - Displays the properties of root project ‘material-design-libraries’.
tasks - Displays the tasks runnable from root project ‘material-design-libraries’ (some of the displayed tasks may belong to subprojects).

Install tasks

installDebug - Installs the Debug build.
installDebugAndroidTest - Installs the android (on device) tests for the Debug build.
uninstallAll - Uninstall all applications.
uninstallDebug - Uninstalls the Debug build.
uninstallDebugAndroidTest - Uninstalls the android (on device) tests for the Debug build.
uninstallRelease - Uninstalls the Release build.

Verification tasks

check - Runs all checks.
connectedAndroidTest - Installs and runs instrumentation tests for all flavors on connected devices.
connectedCheck - Runs all device checks on currently connected devices.
connectedDebugAndroidTest - Installs and runs the tests for debug on connected devices.
deviceAndroidTest - Installs and runs instrumentation tests using all Device Providers.
deviceCheck - Runs all device checks using Device Providers and Test Servers.
lint - Runs lint on all variants.
lintDebug - Runs lint on the Debug build.
lintRelease - Runs lint on the Release build.
test - Run unit tests for all variants.
testDebugUnitTest - Run unit tests for the debug build.
testReleaseUnitTest - Run unit tests for the release build.

Other tasks

clean
extractProguardFiles
jarDebugClasses
jarReleaseClasses
transformResourcesWithMergeJavaResForDebugUnitTest
transformResourcesWithMergeJavaResForReleaseUnitTest

To see all tasks and more detail, run gradlew tasks --all

To see more detail about a task, run gradlew help --task

BUILD SUCCESSFUL

Total time: 3 mins 39.025 secs
demo@trial:~/project/android/demos/material-design-libraries$

Seems like project build is working fine on local.

I do have the gradle wrapper shown in the link https://www.mkyong.com/gradle/jenkins-could-not-find-gradlewrappermain/

If I go with the 3rd option - the blog post tutorial from given link, the script will download the sdk defined in the scripts every time we run the CI build, am I right?.

Can we have any option where we can use our local installed sdk inside the script? (As I have setup specific runner in my local machine !!! )
Or any Docker image, containing the required sdk component downloaded, that is located at https://hub.docker.com/?

Thanks.

If it was my project I would find a docker container image that contains all my tools and failing to find one I might go back to VM based gitlab CI runners and leave out docker.

Android + Dockerized CI == Pain?

1 Like

I have tried a demo of docker image few days ago and here is the docker image - https://hub.docker.com/r/tks19/android-tks-libraries/

But When I looked into the setting up gitlab-ci using Docker images, like in the blog you referenced, Every time it downloads all tools - costing around 2 gb - I have very limited Internet data available with me.

So I was looking for if locally installed tools can help me!!

I ma newbie to CI so I have no idea which one pains more :slight_smile: but How can I use the existing Docker Image, I mentioned above, in the following script - a copy from blog post? What are the changes I required to do, if I do not want to download any of the tools but use from Docker image?

image: openjdk:8-jdk

variables:
  ANDROID_COMPILE_SDK: "25"
  ANDROID_BUILD_TOOLS: "24.0.0"
  ANDROID_SDK_TOOLS: "24.4.1"

before_script:
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
  - wget --quiet --output-document=android-sdk.tgz https://dl.google.com/android/android-sdk_r${ANDROID_SDK_TOOLS}-linux.tgz
  - tar --extract --gzip --file=android-sdk.tgz
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter android-${ANDROID_COMPILE_SDK}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter platform-tools
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter build-tools-${ANDROID_BUILD_TOOLS}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-android-m2repository
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-google_play_services
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-m2repository
  - export ANDROID_HOME=$PWD/android-sdk-linux
  - export PATH=$PATH:$PWD/android-sdk-linux/platform-tools/
  - chmod +x ./gradlew

stages:
  - build
  - test

build:
  stage: build
  script:
    - ./gradlew assembleDebug
  artifacts:
    paths:
    - app/build/outputs/

unitTests:
  stage: test
  script:
    - ./gradlew test

functionalTests:
  stage: test
  script:
    - wget --quiet --output-document=android-wait-for-emulator https://raw.githubusercontent.com/travis-ci/travis-cookbooks/0f497eb71291b52a703143c5cd63a217c8766dc9/community-cookbooks/android-sdk/files/default/android-wait-for-emulator
    - chmod +x android-wait-for-emulator
    - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter sys-img-x86-google_apis-${ANDROID_COMPILE_SDK}
    - echo no | android-sdk-linux/tools/android create avd -n test -t android-${ANDROID_COMPILE_SDK} --abi google_apis/x86
    - android-sdk-linux/tools/emulator64-x86 -avd test -no-window -no-audio &
    - ./android-wait-for-emulator
    - adb shell input keyevent 82
    - ./gradlew cAT

Thanks in advacne :slight_smile:

It seems to me you should not be using Docker then. If you are new to CI and new to DOCKER, start with a Shell Runner. Build on a VM or even on your main Developer machine where your build commands will all work, and when that works, the move to a Shell Runner inside a dedicated “test VM” that is always running, and is hosted on a Server on your network.

When builds fail, you can remotely connect into that VM. If you’re a windows guy, make it a windows vm. If you’re a Linux guy, make it a Linux vm.

If you’re an expert Docker user and simply confused by Gitlab’s details of using Docker, then perhaps gitlab CI in docker is the exact thing you want. But from your problems above, I doubt it’s right for you.

I also don’t think you want to be explicitly writing commands that force a download, which is what you just did right here:

before_script:
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
  - wget --quiet --output-document=android-sdk.tgz https://dl.google.com/android/android-sdk_r${ANDROID_SDK_TOOLS}-linux.tgz
  - tar --extract --gzip --file=android-sdk.tgz
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter android-${ANDROID_COMPILE_SDK}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter platform-tools
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter build-tools-${ANDROID_BUILD_TOOLS}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-android-m2repository
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-google_play_services
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-m2repository
  - export ANDROID_HOME=$PWD/android-sdk-linux

This seems like a disastrous idea. If you were an expert Container user you would have created a container that contains exactly what you wanted, instead of trying to take a generic container and update it and apt-get upgrade it and then android-sdk-download on every build, which is a terrible idea!

Where I work, I recommend a GENTLE introduction to CI that does not INCLUDE CONTAINERS:

  1. shell executor inside a VM.
  2. take snapshot before apt-get updates.
  3. apt-get update your VM every week and leave it stable for the week.
  4. at same time as apt-get update, apply any android tool updates.
  5. revert to snapsohot if updates break you.

If you can handle that, and your internet bandwidth can handle that, fine.

If you want to work with containers but maintain and update the container images on some regular basis, fine. But do you want to? It seems to me, you will hate it.

You could consider using a Droplet on Digital Ocean instead of locally hosting your own Runner VMs, also.

2 Likes

Good explanation !!! +1

I set up my machine earlier using SHELL. but initial builds were failing, so I looked for if Docker can help me. But that isn’t worked … issue I created at https://gitlab.com/gitlab-org/gitlab-ce/issues/24864

#from: https://github.com/jangrewe/gitlab-ci-android/blob/master/.gitlab-ci.yml

image: registry.magic-technik.de/gitlab-ci/android

stages:
- build

before_script:
- export GRADLE_USER_HOME=$(pwd)/.gradle
- chmod +x ./gradlew

cache:
  key: ${CI_PROJECT_ID}
  paths:
  - .gradle/

build:
  stage: build
  script:
  - ./gradlew assembleDebug
  artifacts:
    paths:
    - app/build/outputs/apk/app-debug.apk 

So I switched to another CI setup I mentioned in My Question… even that one also failed.

From your explanation, I turned to local build using SHELL because of terrible idea of downloading tools every time.

I am not a Docker expert. :expressionless:

How Can I use Droplets on Digital Ocean as you mentioned lastly? Do you mean I should own a server and setup and run CI with SHELL over there?

If you have never used Digital Ocean, then use whatever Cloud Provider you know how to use. Setting up a VM on a cloud provider is much like setting up a VM on your own computer, you just have to SSH into it and set it up. If you’ve never done anything like this, you should get some help from somehow who has set up a VM before. It’s a lot like setting your own computer up, but usually would be done purely from a bash prompt.

If your Gitlab instance is on a private network that has no internet access, then don’t bother with setting up a cloud based runner. A cloud based runner is appropriate for when you have a Gitlab instance that is itself reachable from the Cloud system. So if your project was hosted at githost.io or gitlab.com, or your own “personal” yet “internet visible” gitlab, then a cloud runner could be a good idea.

Suppose you get sick of managing your own gitlab instance. You move to Githost.io which hosts Gitlab for you. Then you could use their runners too:

https://githost.io/#runners

You have SO MANY options.

1 Like

Hi,

I just found today in my project’s pipeline that there were 2 builds got passed!!! But those are the commits my team mates made from their machine into different branch.

While till now I was trying in master branch where all the builds are failed … I am not able to understand how this is implemented in my case???
https://about.gitlab.com/images/ci/arch-1.jpg

Is it possible that my all builds failed because I am running the runner in the same machine, while those 2 builds were successful because some one else have used my machine as remote runner ???

Hi I’m trying to to setup a gitlab-ci integration to compile android apps and tried with the example in Setting up GitLab CI for Android projects

I’m using gitlab-ce 8.14.2 and gitlab-ci-multi-runner 1.8.1 (running on a RHEL 7 virtual machine on vSphere 5.5) with docker as executor. After some playing with proxy settings I was able to compile and test the android application but the functionalTest don’t work. I think I’m missing some configuration on the host. What happen is that the container don’t recognize the hardware acceleration and did not run.

$ echo no | android-sdk-linux/tools/android create avd -n test -t android-${ANDROID_COMPILE_SDK} --abi google_apis/x86
Android 7.1.1 is a basic Android platform.
Do you wish to create a custom hardware profile [no]Created AVD ‘test’ based on Android 7.1.1, Google apis Intel Atom (x86) processor,
with the following hardware config:
hw.lcd.density=240
hw.ramSize=512
vm.heapSize=48
$ android-sdk-linux/tools/emulator64-x86 -avd test -no-window -no-audio &
$ ./android-wait-for-emulator
emulator: WARNING: Increasing RAM size to 1GB
emulator: ERROR: x86 emulation currently requires hardware acceleration!
Please ensure KVM is properly installed and usable.
CPU acceleration status: KVM is not installed on this machine (/dev/kvm is missing).

Do you have any suggestion?
Also opened a issue here https://gitlab.com/greysonp/gitlab-ci-android/issues/1

@fduranti in Linux to run any virtual machine - let me correct myself - to provide virtualization support from kernel side, you need to install KVM, or compatible DKMS. Intel VT-x or AMD SVM are the Hardware virtualization which is base requirement, but these are the software / core libraries to support hardware virtualization from kernel.

The AVD that you have created is nothing but one kind of virtual machine like we create in VMWare / virtual box. So you need to install basic KVM and related libraries

Hope this link can help : https://www.google.co.in/search?client=ubuntu&channel=fs&q=install+kvm+in+linux&ie=utf-8&oe=utf-8&gfe_rd=cr&ei=LaVFWK32EPLx8AeZoJaoBA

What i don’t understand is “where” I should install kvm … on the VM that run docker or inside the docker container?
I’ve tried inside the docker container but some packages cannot be installed (and I’m not sure it will work).
If I’ve to install it in the “hosting vm”, should I pass the /dev/kvm or something like that to the container?

Install kvm in your RHEL os … once it is installed, the AVD command, where you were getting error, should pick the location it requires.

I am new to CI so I don’t much about Docker…

Ok I’ll start trying from the vm itself then I’ll check what to do on the contained :slight_smile:

Sure…

OK. I just confirmed that I have installed gitlab-ci in my machine… and if I try to commit anything - the build fails. But if someone commit to my repo- the build passes.

Well I am still unclear about the structure that is being used in my case…!!!
https://about.gitlab.com/images/ci/arch-1.jpg

Came across this question because I encountered the exact same problem setting up GitLab CI for a Java project using gradlew.

In my case I eventually worked out the problem was the default .gitignore for Java projects ignores *.jar, so gradle/wrapper/gradle-wrapper.jar was in my local workspace but git add silently decilined to add it to the index.
Consequently gradle-wrapper.jar is missing when GitLab CI attempts to use gradlew to build the project.

So for me the fix was simply to git add -f gradle/wrapper/gradle-wrapper.jar && git push

1 Like