Open in app

Sign in

Write

Sign in

STF — Run autotests on Android devices managed by Jenkins CI

Alexey Levin
9 min readJan 17, 2021

STF is probably the most popular open source platform for running tests on the real Android devices for now. It seems that there are not enough instructions on the Internet on how to make complete production infrastructure with CI for running instrumental and UI tests on Android devices.

In this post, I will try to show You (not how STF is installed and how it UI are looks) how to build expandable infrastructure for running autotests on Android devices managed by Jenkins CI with minimal effort and with minimal equipment.

Hardware recommendations for our task:

  • 1-st instance — Jenkins server (in cloud or Your hardware)
  • 2-st instance — Jenkins node — build app (in cloud or Your hardware)
  • 3-st instance — Jenkins node — build tests (in cloud or Your hardware)
  • 4-st instance — STF (Your hardware)

Map overview

For those who want to understand the structure of this example , I draw a scheme with the location of instances and connections between them.

Step 1 — Create the instances and connections between them

I will not describe the details of the Jenkins server installation, this information is widespread.

I will describe a few connection details:

When you have to configure Jenkins not in your private network (especially on the clouds like AWS), be sure to change Jenkins default ports. Someone may scan Your Jenkins default ports very often.

We will establish connections between instances via SSH with a key pair. Just in case, I will describe how this is works.

On a the agent-machine with direct access to the command line or via SSH provided by the cloud:

  1. Create a key pair ssh-keygen -t rsa -C “key_name”
  2. Copy the public SSH key with which we will connect to the SSH server cat ~/.ssh/id_rsa.pub

On a the server-machine with direct access to the command line or via SSH provided by the cloud:

  1. Look at the current port grep Port /etc/ssh/sshd_config
  2. If the port is standard (22), then open the SSH config file sudo nano /etc/ssh/sshd_config
  3. Change the port to 22210, for example Port 22210
  4. Restart ssh sudo systemctl restart sshd
  5. We look at the new open port for SSH netstat -tln | grep 22210
  6. Insert Your public SSH key from agent-machine (which was copied earlier) to authorized keys list on SSH server echo “input_here_your_public_ssh” >> ~/.ssh/authorized_keys
  7. In total, for all instances there will be the following connection scheme:

Step 2 — Install packages on Jenkins node (build app node)

Install Android SDK and command line tools

  1. Installing the Android SDK
  • sudo apt-get update
  • yes | sudo apt install android-sdk

2. Installing the Android command line tools

3. Installing the environment variable for current user echo -e “export ANDROID_SDK_ROOT=/usr/lib/android-sdk” >> ~/.bashrc

4. Installing the PATH

  • echo -e “export PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/latest/bin:$ANDROID_SDK_ROOT/cmdline-tools/tools/bin” >> ~/.bashrc
  • echo -e “export PATH=$PATH:$ANDROID_SDK_ROOT/emulator” >> ~/.bashrc

5. Restart .bashrc . ~/.bashrc

6. Give the current user permission to install Android SDK packages sudo chown $USER:$USER $ANDROID_SDK_ROOT -R

7. Checking Android SDK manager sdkmanager — version

Install Android build tools

I suggest installing the most complete set of packages Android build tools.

  • yes | sdkmanager “cmdline-tools;latest”
  • sdkmanager “build-tools;30.0.3”
  • sdkmanager “platform-tools”
  • sdkmanager “platforms;android-30”
  • sdkmanager “sources;android-30”

Checking for installed packages sdkmanager — list

Install Git

Updating packages sudo apt-get update

Installing Git yes | sudo apt install git

Checking for installed packages git — version

Step 3 — Setup STF instance

Not so long ago, STF developers have posted an option to install STF inside Docker compose (commit link)

Before that, you could either put everything natively, or write your own isolation, or use another solutions like this: https://github.com/nikosch86/stf-poc

In this post, I will describe an option for a quick start STF in Docker Compose.

Install Docker

  1. Install packages sudo apt-get -y install docker
  2. Start Docker sudo service docker start
  3. Give permission for launch sudo chmod 666 /var/run/docker.sock
  4. check installation sudo service docker status

Install Docker compose

Download and install packages sudo curl -L “https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose

Give permission for launch sudo chmod +x /usr/local/bin/docker-compose

Check installation docker-compose — version

Install STF

  1. Copy this project to your repository from here: https://github.com/DeviceFarmer/stf

2. Go to the config file (link here) and put in it your IP from which you want to see STF. It can be your local IP, public IP or localhost (127.0.0.1)

3. Create a directory for projects and go insite it mkdir ~/workspace && cd ~/workspace/

4. clone your project git clone — single-branch — branch <Your branch> https://github.com/panarik/stf.git

5. That’s it, now just run docker-compose and watch the magic: cd ~/workspace/stf/ && sudo docker-compose up -d — build

Run STF

Type in the browser your IP that was entered in the project. The default port is 7100. Enjoy the UI:

The default functionality allows any user. The simplest solution would be to block all incoming requests on this port on the server, except for the trusted list.

Configuring STF

Copy adbkey from client to STF. Create and write a token. We’ll need it a little later:

Connect devices

I’ll tell you about the devices. Usually you don’t need to do anything special. Allow installation of applications and security via USB and connect via USB to an instance with a running farm. STF will detect the device itself, download and install the STF agent on it. You will only need to allow installation on the device. After successful connection, the device will appear on STF and can be accessed via adb.

Step 4 — Install packages on Jenkins agent (build test node)

Interesting things will happen in this instance. Tests will be built here, also request devices from the farm via adb, and also manage the farm by means of its API to reserve and release devices on STF. You can also install an emulator.

I also explay, why we have separated the STF and build test node on two different instances. STF runs adb server to find connected devices. The test build node also runs the adb server to manage devices and test the application on them. If you raise everything on one machine, there will be at least conflicts of device ports.

Install Android SDK and command line tools

  1. Installing the Android SDK
  • sudo apt-get update
  • yes | sudo apt install android-sdk

2. Installing the Android command line tools

3. Installing the environment variable for current user echo -e “export ANDROID_SDK_ROOT=/usr/lib/android-sdk” >> ~/.bashrc

4. Installing the PATH

  • echo -e “export PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/latest/bin:$ANDROID_SDK_ROOT/cmdline-tools/tools/bin” >> ~/.bashrc
  • echo -e “export PATH=$PATH:$ANDROID_SDK_ROOT/emulator” >> ~/.bashrc

5. Restart .bashrc . ~/.bashrc

6. Give the current user permission to install Android SDK packages sudo chown $USER:$USER $ANDROID_SDK_ROOT -R

7. Checking for installed packages sdkmanager — version

Install Android build tools

I suggest installing the most complete set of packages Android build tools

  • yes | sdkmanager “cmdline-tools;latest”
  • sdkmanager “build-tools;30.0.3”
  • sdkmanager “platform-tools”
  • sdkmanager “platforms;android-30”
  • sdkmanager “sources;android-30”

Checking for installed packages sdkmanager — list

Now for the most interesting part. We can use three different options to send commands to STF:

  • use the Gradle plugin that the STF developers mention in the documentation (link here)
  • use a third party client that STF developers mention in the documentation (link here)
  • make your own microservice that will use STF API (link here)

We will use the second option. It’s easy to install and doesn’t affect your application code. Developers really don’t like it if testers put incomprehensible plugins in Gradle.

Install stf-console-client

  1. Download the latest version of the client. At the time of this writing, the link is: cd ~/Downloads && wget https://github.com/e13mort/stf-console-client/releases/download/0.3.4/stf-console-client-0.3.4.tar
  2. Create directory ~/DevTools mkdir ~/DevTools && cd ~/Downloads
  3. Unpack the archive into the ~/DevTools directory tar -C “~/DevTools/” -xvf stf-console-client-0.3.4.tar
  4. Installing the PATH for current user echo -e “export PATH=$PATH:/home/ubuntu/DevTools/stf-console-client-0.3.4/bin/” >> ~/.bashrc
  5. Restart .bashrc source ~/.bashrc

Configure your Android application

  1. Create an empty farm.properties file in the root directory of your Android application. Put in it a token that was created in STF and other credentials.

Example:

stf.url=http://192.168.1.70:7100/api/v1/
stf.key=92b0cb63c80f486d8f0311adb204b654ef64c81c4d41440db910a9a12721edb1
stf.timeout=60
android_sdk=/usr/lib/android-sdk

Link for example project: https://github.com/panarik/AndroidClearApp/blob/Jenkins-parrallelTesting/farm.properties

2. Clone your project with a mobile application to the ~/workspace/ catalog cd ~/workspace/ && git clone — single-branch — branch Jenkins-parrallelTesting https://github.com/panarik/AndroidClearApp.git

3. Go to the project catalog cd ~/workspace/AndroidClearApp

4. Check the functionality of stf-console-client. Enter into the command prompt: stf

5. Search for all devices: stf devices — all

Devices registered in STF are visible.

You can find a list of all commands in this way: stf -help

6. Connection to all possible devices. Run it in the background, because in the current version, the command blocks the console: stf connect — all &

7. We check the availability of reserved devices on our instance: adb devices

Shows a list of devices? Congratulations! We started using the STF! You can also go to STF UI to check. There, the reserved devices must be appropriately tagged.

8. Since the project has already been loaded into the instance, you can try running the first test on connected devices:

  • ./gradlew clean assembleDebug
  • ./gradlew connectedAndroidTest

Build successful? Client is configured and You can proceed to configuring CI.

Step 6 — Configuring Jenkins server

Finally, we will configure the environment for the run autotest on STF devices with CI.

Let’s connect all the nodes in Jenkins:

  • Jenkins agent (build app node)
  • Jenkins agent (build test node)

Create new job.

Create Jenkinsfile in the root folder of your mobile application project.

Work script here:

pipeline {

    agent none

    environment {
        ANDROID_SDK_ROOT = '/usr/lib/android-sdk'
    }

    stages {

        stage('Checkout scm') {
            agent {
                label 'build app node'
            }
            steps {
                checkout scm
            }
        }

        stage('Build') {
            agent {
                label 'build app node'
            }
            steps {
                echo 'build app'
                sh './gradlew clean assembleDebug --no-daemon'
                stash name: 'app', includes: '**', excludes: '**/.gradle/,**/.git/**'
            }
        }

        stage('OnFarmTest') {

            agent {
                label 'build test node'
            }
            environment {
                PATH = "/home/ubuntu/DevTools/stf-console-client-0.3.4/bin:$PATH"
            }
            steps {
                echo "PATH is: $PATH"
                echo 'test app'
                unstash 'app'
                sh 'pwd'
                sh 'stf'
                sh 'stf devices --all'
                sh 'sleep 5'
                sh 'stf connect --all &'
                sh 'sleep 5'
                sh 'adb devices'
                sh './gradlew connectedAndroidTest --no-daemon'

            }
            post {
                always {
                    junit '**/debugAndroidTest/**'
                }
            }

        }

    }
}

An example Jenkinsfile can be viewed on my test project at the link here.

Now start the job. You can later customize some triggers or schedule run scripts. Devices are booked, tests are run on them, the report is collected.

Profit!

Please note that we have executed almost all the commands from the console, which means that you can always pack the entire installation script using Packer into your instance.

I hope this post was interesting for you and will improve your understanding of the possibilities of mobile automation on real devices.

Thank you for your attention!

QA
Automation
Mobile Apps
Android
Testing

--

--

Alexey Levin

Written by Alexey Levin

3 Followers

QA Automation

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams