Please rely on the newly published project Build NXP iMX Embedded Linux instead of this one you are presently looking at. Although this project is still technically correct, the codeaurora repository no longer exists.
Contents of this Hackster project- Why use Docker?
- Terminology
- Command line prompts
- Install Docker on a workstation running Ubuntu 20.04
- Configuring Docker so we don't need to use sudo for every command
- Two Filesystems
- Creating our DockerFile
- Deploy a Docker container by running the DockerFile
- Confirming the Ubuntu version for our Docker Container
- Building Yocto Project within our new Docker Container
- How to Sudo from our Docker Container
- Useful Docker commands
My initial reason for using Docker is that Yocto Project version 3.0 "zeus" is tested to run on Ubuntu 18.04 and 16.04, but not on Ubuntu 20.04. I have personally found that running Yocto zeus on Ubuntu 20.04 is usually successful but sometimes strange things can happen that require hours or days to debug. I do not want to uninstall Ubuntu 20.04 from my workstation and install Ubuntu 18.04 just to get successful Yocto Project builds, and I do not want to continue to play games with the occasional broken build because of an apt upgrade.
Some other good reasons to use Docker
- The efficiency of Docker is far superior to running within a virtual machine because we are using the Linux kernel of the host machine (workstation)
- The isolation of a Docker Container allows us to get repeatable results that are not impacted by changes we make to our host machine (workstation)
This article provides excellent explanation for the Docker terminology I am using in this project.
Excerpts from the article are below
- Docker Images - A Docker image is an immutable (unchangeable) file that contains the source code, libraries, dependencies, tools, and other files needed for an application to run.
- Docker Containers - A Docker container is a virtualized run-time environment where users can isolate applications from the underlying system. These containers are compact, portable units in which you can start up an application quickly and easily.
- DockerFile - It all starts with a script of instructions that define how to build a specific Docker image. This script is called a DockerFile. The file automatically executes the outlined commands and creates a Docker image.
Should you get confused as to which terminal I am typing into, there are three command line prompts that I use throughout this project.
On the host (workstation), as user "flint". Your login prompt will look different than mine.
[flint@ZBook] $
Within the container, as user "build"
build@25e2c87a9f84:<PWD>$
Within the container, as user "root"
root@25e2c87a9f84:<PWD>#
Installing Docker on Ubuntu 20.04Below I will list some useful documentation on installing Docker, and then I will walk through the commands that I used. There is nothing unique about what I have done here, my commands very closely match the Docker documentation.
Documentation on installing Docker
Uninstall old versions
[flint@ZBook] $ sudo apt remove docker docker-engine docker.io containerd runc
Update repo sources
[flint@ZBook] $ sudo apt update
Install prerequisite packages
[flint@ZBook] $ sudo apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
Add Docker GPG key to your workstation
[flint@ZBook] $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Add Docker repo to apt sources
[flint@ZBook] $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
Update repo sources
[flint@ZBook] $ sudo apt update
Tell apt to use docker repo and not ubuntu
[flint@ZBook] $ apt-cache policy docker-ce
Install Docker
[flint@ZBook] $ sudo apt install docker-ce
Check that Docker is running
[flint@ZBook] $ sudo systemctl status docker
Configuring Docker so we don't need to use sudo for every commandBelow are the commands that I used. These are nearly identical to what is provided in the documentation.
Add our username to the docker group
[flint@ZBook] $ sudo usermod -aG docker ${USER}
Apply the new group membership
[flint@ZBook] $ su - ${USER}
Confirm that our user is now added to the docker group
[flint@ZBook] $ id -nG
Confirm that you can see "docker"
flint adm cdrom sudo dip plugdev lpadmin lxd sambashare vboxusers docker
Restart Docker
[flint@ZBook] $ sudo service docker restart
It is necessary to log out of your account and back in for the change to be effective.
Confirm that my username is in the docker group
[flint@ZBook] $ groups
You should see "docker" in the list
flint adm cdrom sudo dip plugdev lpadmin lxd sambashare vboxusers docker
Two FilesystemsThe files produced by running Yocto Project within a Docker Container, by default will not be accessible outside of the container. Additionally, trying to write SD cards from within the Docker Container is (most likely) impossible.
My preferred method is to attach the filesystem from my Docker Container to the host (workstation) filesystem. This allows me to work with the files as if they had been build on my host without Docker.
These are the two directories
<HostDirectory> This is the location on the host
(workstation) where we want to see the Yocto Project files
<DockerDirectory> This is the location within the Docker Container where our Yocto Project files will go
When we create our Docker Container, we will tell Docker that the files in <DockerDirectory>
will also be accessible at <HostDirectory>
The setup that I am using for this project is as follows
<HostDirectory> = /work/imx8mp/zeus/imx-5.4.70-2.3.2
<DockerDirectory> = /home/build/docker
Create a <HostDirectory> on your workstation
[flint@ZBook] $ mkdir /work/imx8mp/zeus/imx-5.4.70-2.3.2
The <DockerDirectory> will be created by the DockerFile, described later in this project.
Creating our DockerFileA DockerFile is a text file that allows us to automate building a Docker Image.
Below I will list some useful documentation on how to build your own DockerFile. I will then describe the functionality of the DockerFile I am providing with this project so that you have an easier time modifying it for your own purposes.
Documentation on building DockerFiles
- How to Build Docker Images with Dockerfile
- How to create Docker Images with a Dockerfile on Ubuntu 18.04 LTS
- Docker official documentation: Create a base image
Pull the Ubuntu Bionic image down so we can use it
[flint@ZBook] $ docker pull ubuntu:bionic
bionic: Pulling from library/ubuntu
Digest: sha256:7bd7a9ca99f868bf69c4b6212f64f2af8e243f97ba13abb3e641e03a7ceb59e8
Status: Image is up to date for ubuntu:bionic
docker.io/library/ubuntu:bionic
Note your UID and GID
When editing your personal DockerFile, it is essential to know your UID and GID. It is important for the user you create within the Docker Image to have a UID and GID matching your account on the workstation so that there are no issues with file permissions when sharing data between the two accounts.
[flint@ZBook] $ whoami
flint
[flint@ZBook] $ id flint
uid=1000(flint) gid=1000(flint) groups=1000(flint),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),120(lpadmin),131(lxd),132(sambashare),135(vboxusers),998(docker)
Download the DockerFile
I have attached my DockerFile "dockerfile-yocto18" to this project. Please download it and modify it as needed. I also have this DockerFile on GitLab. Sould I need to make any updates to the file in the future you will see them reflected on GitLab.
Explanation for the commands in my DockerFile
My DockerFile is fairly well documented, but for the sake of completeness I will provide additional commentary below.
# Specify Ubuntu digest to pull from
FROM ubuntu:bionic
We want to pull from an existing Ubuntu digest. We can either use the code name or the release version number. Examples for both are ubuntu:xenial or ubuntu:16.04 both will accomplish exactly the same goal.
# Install dependencies
RUN apt update && apt upgrade -y
This is a very standard sudo apt update and upgrade
# Handle the tzdata prompt
RUN ln -fs /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
RUN DEBIAN_FRONTEND=noninteractive apt install -y --no-install-recommends tzdata
There is an issue where one of the applications we are trying to install in the next section relies on tzdata. When installing tzdata, the user is prompted to specify their time zone location. For me this is America/Los_Angeles. Edit this as you need.
# Put everything we want to install here
RUN apt install -y gawk wget git-core diffstat unzip texinfo \
gcc-multilib build-essential chrpath socat cpio python \
python3 python3-pip python3-pexpect xz-utils debianutils \
iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev \
pylint3 xterm rsync curl locales apt-utils sudo vim bash-completion screen
All the applications required for a successful Yocto Project build with NXP are installed here. Feel free to add more applications should you need them.
# Clean up
RUN apt clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Just tidy up the system and remove temporary files so they do not cause our Docker Image to take up too much space.
# Replace dash with bash (Ubuntu uses dash as /bin/sh for system shells)
# https://wiki.ubuntu.com/DashAsBinSh
# Note that using && below to chain commands is essential to avoid failure
RUN rm /bin/sh && ln -s bash /bin/sh
This DockerFile runs as sudo, so the default shell is dash and not bash. Compared to Bash, Dash is missing the source command which is needed for Docker.
# Install repo utility
RUN curl -o /usr/local/bin/repo https://storage.googleapis.com/git-repo-downloads/repo
RUN chmod a+x /usr/local/bin/repo
This is identical to how we set up repo without Docker
# Set up locales
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
Locale is normally configured when we install Ubuntu on our workstation, but this feature is not done for us in the ubuntu image we are pulling from. If we do not configure locales, Yocto Project will give us errors.
# Create user called build (because we can not run Yocto Project as root)
# Change uid and gid, to match my account on my workstation
# Username is not all that important
ENV UID=1000
ENV GID=1000
ENV USER=build
RUN groupadd -g $GID $USER && \
useradd -u $UID -g $GID -ms /bin/bash $USER && \
usermod -a -G sudo $USER && \
usermod -a -G users $USER
This is where we create a user to run Yocto Project. It is important to set the UID and GID to match your workstation user (described above). The actual username is not that important and does not need to match your workstation username. I like to use "build" because it is generic.
# Change username and work directory
USER $USER
WORKDIR /home/$USER
This is the point in the script where we change the user from root to build. It is important to do this before running the following commands.
# Create the docker directory
RUN mkdir /home/$USER/docker
Create a directory for building Yocto Project. It can be called anything. The most important thing is to remember what it is called for mounting it to the standard workstation filesystem. This will be covered in the next section.
# Configure git
RUN git config --global user.name "Flint Weller"
RUN git config --global user.email "flint.weller@hackster.io"
Just like we have to do for Yocto Project normally, we must provide some simple git configurations.
Deploy a Docker container by running the DockerFileIn this step we are going to create a new Docker Image by running our DockerFile. We only need to run this command once for any DockerFile. The end result will be a Docker Image that we can use to create many Docker Containers.
Docker build syntax
SYNTAX: docker build --file <DockerFile> --tag <ImageName> .
<DockerFile> This is the location/filename for the DockerFile we created above.
<ImageName> The is the name we want to give to the Docker Image we will create.
NOTE: Do not forget the "." at the end of the command.
Build a Docker Image by running the DockerFile
[flint@ZBook] $ docker build --file dockerfile-yocto18 --tag yocto18 .
View the newly created Docker Image
[flint@ZBook] $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
yocto18 latest 3b2476ab42c1 2 days ago 865MB
ubuntu bionic 39a8cfeef173 4 weeks ago 63.1MB
Our newly created image is on the first line, called "yocto18". The "ubuntu" image is the one we pulled earlier.
Syntax for creating a Docker Container
We only need to run this command once per Docker Container. When we create the container we are also giving the container a name and we are linking a container directory to a directory on our host (workstation).
SYNTAX: docker run -dit -P --name <DockerContainerName> -v <HostDirectory>:<DockerDirectory> <DockerImageName>
<DockerContainerName> This is the name we want to give to our Docker Container that will start running after we issue the command
<HostDirectory> This is the location on the local workstation where we want to see our Yocto Project files
<DockerDirectory> This is the location within the Docker Container where our Yocto Project files will go
<DockerImageName> The is the name of our Docker Image that we named in the previous command.
Deploy a container with the working directory mounted
[flint@ZBook] $ docker run -dit -P --name imx8-zeus -v /work/docker/zeus:/home/build/docker yocto18
If Docker is successful in creating the container, it will return a long hexadecimal string that I have never found useful.
6f8305650c03a35ae4d4966da67f9582bac35fe8cba6bf463b6351994b641417
The container that we just created has been started but our terminal has not been attached to it yet.
View the newly created container
[flint@ZBook] $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
25e2c87a9f84 yocto18 "bash" 14 seconds ago Up 13 seconds imx8mp-zeus
Attach to this new container
[flint@ZBook] $ docker attach imx8mp-zeus
build@25e2c87a9f84:~$
Notice that we get a Bash $ prompt. We are now able to run commands within our Docker Container
Confirming the Ubuntu version for our Docker ContainerThis is a quick section that demonstrates how our container is different from our host workstation.
Check the Unix name for the environment within our container
build@25e2c87a9f84:~$ uname -a
Linux 25e2c87a9f84 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Check the Unix name for the environment within our host (workstation)
Run this command on a host terminal, separate from the one that is running our Docker Container
[flint@ZBook] $ uname -a
Linux ZBook 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Notice how the container and the host have the same Linux kernel? This is the kernel used in Ubuntu 20.04 and not Ubuntu 18.04. The reason for this is that the container uses the kernel of the host machine, which greatly improves efficiency compared to running a separate kernel within a virtual machine. Some people have criticized Docker for not being 100% like a different system, but in reality the kernel has no real effect on the build process.
Check the OS release for the environment within our container
build@25e2c87a9f84:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
build@25e2c87a9f84:~$
Check the OS release for the environment within our host (workstation)
Run this command on a host terminal, separate from the one that is running our Docker Container
[flint@ZBook] $ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Notice how the information is different. Although the kernel may be the same, everything else within the container should match an Ubuntu 18.04 environment.
Building Yocto Project within our new Docker ContainerThis section will look very familiar to those who have seen my other Hackster projects on building Yocto Project. The commands I am listing here are identical to how we would build Yocto Project on our workstation without Docker.
Documentation on using Docker Containers for Yocto Project builds
- Using Docker Containers for Yocto Builds
- Using Docker as a Yocto build system container
- 5 Steps to Compile Yocto Using Docker Containers
- Using Docker Containers for Reproducible Yocto Builds
Configure Repo and source the NXP imx-linux Yocto environment
Starting out in the Docker "build" home directory. Change directory into "docker" which is where we want to perform our work
build@25e2c87a9f84:~$ ls
docker
build@25e2c87a9f84:~$ cd docker
build@25e2c87a9f84:~/docker$
Repo command definitions
<branch name>
This is the generational version
imx-linux-zeus is the version we are using for this project
All releases with the 5.4 kernel will use imx-linux-zeus
<release manifest>
This is the NXP release version on a 6 month cadence
imx-5.4.70-2.3.2.xml is the latest and most likely the last release using Yocto Project version 3.0 "Zeus"
NOTE: If you find newer versions on the release manifest, they should work with these instructions
Repo command syntax
$ repo init -u
https://source.codeaurora.org/external/imx/imx-manifest
-b <branch name> [ -m <release manifest>]
Initialize Repo
build@25e2c87a9f84:~/docker$ repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-zeus -m imx-5.4.70-2.3.2.xml
I recommend you type "y" for yes when asked about the color display.
Enable color display in this user account (y/N)?
repo has been initialized in /home/build/docker
Use repo sync to download the files
build@25e2c87a9f84:~/docker$ repo sync
Receiving objects: 100% (158102/158102), 14.93 MiB | 34.43 MiB/s, done.
Resolving deltas: 100% (103740/103740), done.
Fetching: 100% (14/14), done in 53.288s
Garbage collecting: 100% (14/14), done in 0.018s
Checking out: 100% (14/14), done in 0.703s
repo sync has finished successfully.
View the contents of the directory
build@25e2c87a9f84:~/docker$ ls
imx-setup-release.sh README README-IMXBSP setup-environment sources
You can also open a second terminal window, so that you can view the files outside of the Docker environment
[flint@ZBook] $ pwd
/work/imx8mp/zeus/imx-5.4.70-2.3.2
[flint@ZBook] $ ls
imx-setup-release.sh README README-IMXBSP setup-environment sources
NOTE: We are able to view the same files and directory structure between Docker and the standard workstation environment because we previously created this Docker container to link the two together.
imx-setup-release definitions
MACHINE=<machine>
use EVK names for <machine> listed in the NXP Yocto Project Users Guide, section 5.1 "Build configurations"
DISTRO=fsl-imx-<backend> where <backend> refers to the graphics type:
xwayland = Wayland with X11 support - default distro
wayland = Wayland only
fb = Framebuffer (not supported for imx8)
NOTE: Each build folder can only support a single DISTRO
imx-setup-release command syntax
$ [MACHINE=<machine>] [DISTRO=fsl-imx-<backend>] source ./imx-setup-release.sh -b bld-<machine>-<backend>
NOTE: The filename bld-<machine>-<backend> is my personal recommendation and you are not required to follow this format
Set up the Yocto build environment using imx-setup-release
$ EULA=1 MACHINE=imx8mpevk DISTRO=fsl-imx-xwayland source imx-setup-release.sh -b bld-imx8mpevk-xwayland
Bitbake Definitions
<recipe> These are pre-defined recipes specified in Yocto Project Users Guide, section 5.2
imx-image-core = core image with basic graphics and no multimedia. This is a smaller image. It is also the image NXP uses for their daily build tests.
imx-image-multimedia = image with multimedia and graphics. Builds an image with a GUI but without any Qt content.
imx-image-full = image with multimedia and machine learning and Qt. Builds an open source QT 5 image with ML features.
Bitbake command syntax
$ bitbake <recipe>
Initiate building the full image
$ bitbake imx-image-full
Depending on your workstation performance, this may take a few hours or an entire weekend.
To exit this Docker Container
<CTRL> <D>
To enter the Docker container again, we have to attach to it.
[flint@ZBook] $ docker attach imx8mp-zeus
If Docker complains that you cannot attach to a stopped container, then we must start it first.
If the container is stopped, then we must first restart it before attaching to it
[flint@ZBook] $ docker restart imx8mp-zeus
imx8mp-zeus
[flint@ZBook] $ docker attach imx8mp-zeus
build@25e2c87a9f84:~$
Run imx-setup-release
Bitbake will not run if the environment is not configured. So if you close the present shell (terminal) then you will lose the environment set up by imx-setup-release.sh
To set up our environment again, first make sure that your PWD is the Yocto base directory you set up earlier
build@25e2c87a9f84:~$ ls
docker
build@25e2c87a9f84:~$ cd docker
build@25e2c87a9f84:~/docker$
And then run the setup-environment script
build@25e2c87a9f84:~/docker$ source setup-environment bld-imx8mpevk-xwayland
Welcome to Freescale Community BSP
...
You can now run 'bitbake <target>'
...
Your configuration files at bld-imx8mpevk-xwayland have not been touched.
How to Sudo from our Docker ContainerBy default, docker containers run as root and do not have a password on the root account. This is extremely convenient for most Docker situations, but as discussed earlier - Yocto Project wants to run in a non-superuser account; this is why we created the (also passwordless) "build" account.
Our Docker Container works great until we need to run a sudo operation from the build account. I am unaware of any method to successfully sudo or su from a user account when root is passwordless. Some people fix this issue by assigning a password to the root account, but I consider this to go against the passwordless nature of Docker.
The method that I use to achieve sudo is to exit the container, and then to attach to the container again while specifying the root account.
Start
My instructions start from the container bash prompt, in the build account
build@25e2c87a9f84:~$ whoami
build
Exit the container
Type <CTRL> <D>
<CTRL> <D>
build@25e2c87a9f84:~$ exit
Confirm container status
The Docker Container has stopped (exited), lets confirm this
[flint@ZBook] $ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
25e2c87a9f84 yocto18 "bash" 44 hours ago Exited (127) 1 second ago imx8mp-zeus
Start the container
[flint@ZBook] $ docker start imx8mp-zeus
imx8mp-zeus
Attach to the container
This is the command that attaches to the container "imx8mp-zeus" as user "root" and runs the "/bin/bash" shell for us
[flint@ZBook] $ docker exec -it -u root imx8mp-zeus /bin/bash
root@25e2c87a9f84:/home/build# whoami
root
Now that we are logged in as root, we can enter all the commands that require superuser privileges.
Temporarily accessing the user account "build" from root
If you need to gain temporary access to the build account, you can use the "su <UserName>" command. To return to the root account, just type "exit
"
root@25e2c87a9f84:/home/build# whoami
root
root@25e2c87a9f84:/home/build# su build
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
build@25e2c87a9f84:~$ whoami
build
build@25e2c87a9f84:~$ exit
exit
root@25e2c87a9f84:/home/build#
When you have completed the work that required superuser access, I recommend exiting the container and then attaching to it in the usual manner - which will automatically log in to the build account
Exit the container
Type <CTRL> <D>
<CTRL> <D>
build@25e2c87a9f84:~$ exit
Confirm the container status
The Docker Container has stopped (exited), lets confirm this
[flint@ZBook] $ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
25e2c87a9f84 yocto18 "bash" 44 hours ago Exited (0) 29 seconds ago imx8mp-zeus
Start the container
[flint@ZBook] $ docker start imx8mp-zeus
imx8mp-zeus
Attach to the container
[flint@ZBook] $ docker attach imx8mp-zeus
build@25e2c87a9f84:~$
Notice that we get a Bash $ prompt. We are now able to run commands within our Docker Container
Useful Docker commandsThese are for your reference. I find this list useful.
To view commands
[flint@ZBook] $ docker
Test that docker is working
[flint@ZBook] $ docker info
Run a simple test
[flint@ZBook] $ docker run hello-world
Remove image
[flint@ZBook] $ docker rmi hello-world
Search for ubuntu images on dockerhub
[flint@ZBook] $ docker search ubuntu
Download the ubuntu image
[flint@ZBook] $ docker pull ubuntu
See the images I have
[flint@ZBook] $ docker images
Run a Docker Container
[flint@ZBook] $ docker run -it ubuntu
To list running containers
[flint@ZBook] $ docker ps
To list all containers
[flint@ZBook] $ docker ps -a
To rename a container
[flint@ZBook] $ docker rename 72775a7a50a7 imx8-yocto-zeus
To exit from and stop a container
[flint@ZBook] $ exit
To exit from and stop a container
<CTRL> <D>
To start a stopped container
[flint@ZBook] $ docker start imx8-zeus
To restart a stopped container
[flint@ZBook] $ docker restart imx8-zeus
To enter a container again, attach to it
[flint@ZBook] $ docker container attach imx8-zeus
To remove a container
[flint@ZBook] $ docker rm d01bebc76e09
To stop a running container
[flint@ZBook] $ docker stop d01bebc76e09
Comments