Experimenting with Alternatives for Docker : Podman on Mac OS Big Sur

By now everyone must have seen the news that docker desktop is no longer free for enterprise usage.

My interpretation is that this only covers the docker desktop package and not command line docker so linux users shouldn’t have to care. Windows users with WSL2 should be running docker directly in WSL2 in any case.

Mac users are affected I believe as docker desktop effectively wraps docker commands around docker running in a linux VM.

I started looking for alternatives and chanced upon this article written by the ex-maintainer of minikube proposing minikube as an alternative; but after actually going through the documentation, I realised that minikube will require converting docker-compose scripts to kubernetes helmcharts or an equivalent to work.

Instead I decided to give podman a try as appeared to meet all the criteria to be a docker replacement i.e.

• Can be used directly instead of docker on linux.
• On Big Sur, works with a VM either locally or remotely ; possible to setup a local VM on virtualbox, vmware or hyperkit (the hypervisor used by docker
• Has podman-compose as an equivalent to docker-compose

Since I use a Mac with Big Sur and I already have Vmware Fusion running, I decided to adapt the instructions provided here to get this work

I am using homebrewon mac, and I do recommend that you do as well to simplify installation. Commands described below are run on a terminal. I used iTerm2 with zsh. You could use the standard Mac Terminal.

Also I decided to use vagrantas a way to easily manage VMs using their vmware fusion provider

I decided to use the latest dev version of podman-composeto ensure that it supports the delegated mount option which i do have in a few of my docker-compose.yml files to improve docker io performance on docker desktop for mac; i believe that this flag is now mostly redundant and ignored now though

Setup Steps

Commands to install podman, vagrantand podman-compose are below

brew install podman
brew install vagrant
pip3 install https://github.com/containers/podman-compose/archive/devel.tar.gz

I also ran this command

sudo touch /etc/containers/nodocker

to ensure that this message didn’t keep popping up when i ran podman

Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.

Option 1: Setting up Podman with Podman Machine

I haven’t tried this step myself, but it looks so simple as per the podman documentation that I thought I would mention it here.

Podman 3.3 comes with podman machine (apparently not to be confused with the older podman-machine ) that is well integrated with podman.

As per this link the difference between podman machine and podman-machineis as follows

This is a new feature based on QEMU and CoreOS, unlike the old one (described here) which is based on Docker Machine and Boot2Docker, as was available in Docker Toolbox…

i.e. this also uses a virtual machine under the hood. This is much simpler to setup when compared to using Vmware Fusion (see Option 2)

I have not found much documentation on how QEMU is emulating the hypervisor for the VM — usually QEMU needs to be paired with a hypervisor (e.g. KVM on linux) to get the best performance. On Mac, I believe that the underlying hypervisor will have to be one of the free alternatives — potentially Hyperkit? I will look for more documentation on this.

At this point in time, I am not convinced this is the most performant option on Mac, happy to be proved wrong though. It definitely looks like the simplest option though.

podman machine init
podman machine start

Option 2: Setting up Podman with Vmware Fusion

I tried this option because I am running a licensed Vmware Fusion for windows applications on my mac; I didn’t fancy running two hypervisors like I used to do with Docker Desktop on Hyperkit, and Windows on Vmware.

In order to setup the Vmware provider, one needs to first download and install the vagrant vmware utility from this link.

Once installed run the following

vagrant plugin install vagrant-vmware-desktop

Next we need to create the VM definition in vagrant — called a Vagrantfile

Create ~/.vagrant.d/Vagrantfile — Change memsize and numvcpus below as required.

Vagrant.configure("2") do |config|
  config.vm.box = "generic/fedora33"
  config.vm.provider "vmware_desktop" do |v|
    v.vmx["memsize"] = "8192"
    v.vmx["numvcpus"] = "4"
  end
  config.vm.provision "shell", inline: <<-SHELL
    yum install -y podman
    groupadd -f -r podman
    #systemctl edit podman.socket
    mkdir -p /etc/systemd/system/podman.socket.d
    cat >/etc/systemd/system/podman.socket.d/override.conf <<EOF
[Socket]
SocketMode=0660
SocketUser=root
SocketGroup=podman
EOF
    systemctl daemon-reload
    echo "d /run/podman 0770 root podman" > /etc/tmpfiles.d/podman.conf
    sudo systemd-tmpfiles --create
    systemctl enable podman.socket
    systemctl start podman.socket
    usermod -aG podman $SUDO_USER
  SHELL
end

Then add these environment variables in your shell usually ~/.bash_profile or ~/.zshrc

The CONTAINER_HOST is the main one - it can point to your local vagrant VM (as below) or a remote VM on cloud. the CONTAINER_SSHKEY points to the private key that you can use for passwordless ssh into the VM - local or remote.

I also created aliases to point the docker and docker-compose commands to the podman equivalents so that this would work seamlessly.

export CONTAINER_HOST=ssh://vagrant@127.0.0.1:2222/run/podman/podman.sock
export VAGRANT_CWD=${HOME}/.vagrant.d
export CONTAINER_SSHKEY=${VAGRANT_CWD}/.vagrant/machines/default/vmware_desktop/private_key
#use podman and podman-compose instead of docker and docker-compose
alias docker=podman
alias docker-compose=podman-compose

Start a new shell — ensure that your environment variables are loaded and then start the VM.

vagrant up --provider vmware_desktop

To stop the VM

vagrant halt

Testing

I ran the usual docker commands to test. These seemed to work fine

docker ps -a

Playing around a bit, it does feel that this performs much better than docker desktop did — possibly because of the use of Vmware Fusion as the hypervisor in lieu of hyperkit which docker was using. Time will tell.

I did run into a few issues as listed below. I will update this article as and when I find solutions.

Port forwarding issues & resolution (Option 2 specific)

When running a server e.g. tomcat, I am not able to access the port through the browser on localhost . The browser immediately errors out

This suggests that port forwarding from the VM is not in place. So I tried adding specific ports in the Vagrantfile under the section starting with Vagrant.configure("2") do |config|

I added the following forward for ports 8888 which was being used for port forwarding using the -p argument with podman/docker.

config.vm.network "forwarded_port", guest: 8888, host: 8888

While this didn’t solve the issue, instead of erroring out immediately, the page hangs when accessing the port — which suggests that some sort of port forwarding is in place but still being blocked from accessing the actual application within the container. I am wondering if I should look at firewall settings; another post suggests that having docker installed side by side results in conflict. I have removed docker-desktop and rebooted my laptop but it still doesn’t work.

Then I found a solution as described in this issue

i.e. using the following argument along with the podman run command

--network bridge

fixes this for the immediate container

A more permanent solution is to create $HOME/.config/containers/containers.conf (you will need to create the folders and file inside .config )with the following content

[containers]
rootless_networking = "cni"

Keep in mind this only lets you access the port from the IP of the VM i.e. in my case from http://<VM IP>:<port> To access the service from localhost you also need to add the port forwarding entries for the guest and host port in the VagrantFile as above and restart the VM to get this to work.

Note that changing the Vagrantfile requires a restart using vagrant reload

If you want to test this on the fly as a temporary change use the vagrant ssh command

e.g.

vagrant ssh -- -L 8888:localhost:8888

will port forward port 8888 just like a regular ssh command. Running vagrant ssh without any arguments will also ssh you into the VM — pretty handy when you want to inspect the containers inside directly.

Unable to mount local folders and volumes (Option 2 specific)

I was unable to mount local folders as volumes i.e. of the form -v /LocalPath/To/Folder/Or/File:/Path/Within/Container . Standard volume mounts of -v <volume>:/Path/in/Container work;

Podman is currently running in rootless mode on my mac my logged in user

$ls -atl /usr/local/Cellar/podman/3.3.0/bin/podman-remote
-r-xr-xr-x  1 bharat_rajagopalan@uk.ibm.com  admin  30861344 20 Aug 19:47 /usr/local/Cellar/podman/3.3.0/bin/podman-remote

My local files are owned by the same user and are readable by user, group and other so would have thought this would have been sufficient.

$ls -atl .
-rw-r--r--     1 bharat_rajagopalan@uk.ibm.com  staff    11103  6 Jan  2021 README.md

podman is setup to run as root within the VM as seen in the Vagrantfile above. I did note the following though when running podman info

idMappings:
    gidmap: null
    uidmap: null

I wasn’t sure if I needed to set something up there for this to work.

Then it suddenly struck me that I was overthinking this.

podman for Big Sur is effectively running within VM, the command that you run on your Mac terminal is effectively sshing into the VM and running commands.

So what if -v /LocalPath/To/Folder/Or/File:/Path/Within/Container resulted in podman looking for /LocalPath/To/Folder/Or/File inside the VM rather than on your mac file system?

Then the easiest way to fix this would be to share the local folder directly into the VM in exactly the same path. In my case it was within my homefolder.

So I started looking for solutions on sharing folders with vagrant and hit this link based on which I added the following lines under the sectionVagrant.configure("2") do |config|

The lines basically share my entire home folder with the VM and recreates the exact path within the VM:

config.vm.synced_folder "/Users/bharat", "/Users/bharat", create: :true

This fixed the issue for me.

Issues Installing IBM Sterling Docker Devtoolkit

Most of the basic tests around docker-compose worked, but the big test for me was installing the IBM Sterling docker devtoolkit which makes extensive use of docker and docker-compose

At this point in time, I didn’t succeed — the process generated plenty of errors.

Specifically podman failed to load a container from a image provided as a zip file (not pulled).

My backup option is to run this directly within the VM if it fails.

docker-compose: error: unrecognized arguments: -v — remove-orphans

usage: docker-compose [-h] [-f file] [-p PROJECT_NAME]
                      [--podman-path PODMAN_PATH] [--podman-args args]
                      [--podman-pull-args args] [--podman-push-args args]
                      [--podman-build-args args] [--podman-inspect-args args]
                      [--podman-run-args args] [--podman-start-args args]
                      [--podman-stop-args args] [--podman-rm-args args]
                      [--podman-volume-args args] [--no-ansi] [--no-cleanup]
                      [--dry-run]
                      [-t {1pod,1podfw,hostnet,cntnet,publishall,identity}]
                      {help,version,pull,push,build,up,down,ps,run,exec,start,stop,restart,logs}
                      ...
docker-compose: error: unrecognized arguments: db2server appserver mqserver

The postings on this site are the PoV of the author and do not necessarily represent IBM’s positions, strategies or opinions