Running mips/mipsel images fails

[mayeut]

When trying to run mips/mipsel images, docker can't start the container.

Matt$ docker run -it --rm multiarch/debian-debootstrap:mipsel-jessie /bin/uname -a
panic: standard_init_linux.go:175: exec user process caused "no such file or directory" [recovered]
	panic: standard_init_linux.go:175: exec user process caused "no such file or directory"

goroutine 1 [running, locked to thread]:
panic(0x88f8a0, 0xc820152d90)
	/usr/local/go/src/runtime/panic.go:481 +0x3e6

After a while thinking about an error with the images, I finally found that the no such file or directory was related to the qemu binary.

We can check that the expected one is present in the image:

Matt$ docker run -it --rm multiarch/debian-debootstrap:mipsel-jessie /usr/bin/qemu-mipsel-static /bin/uname -a
Linux b9de7767a6d1 4.4.39-moby #1 SMP Fri Dec 16 07:34:12 UTC 2016 mips GNU/Linux

What happens in fact is that when running docker run --privileged --rm multiarch/qemu-user-static:register, 2 qemu binaries get registered for the mips magic/mask.The same goes for mipsel.

I moved a bit the lines from register.sh to emphasize this:

   echo ':mips:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:'${QEMU_BIN_DIR}'/qemu-mips-static:' > /proc/sys/fs/binfmt_misc/register
echo ':mipsn32:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:'${QEMU_BIN_DIR}'/qemu-mipsn32-static:' > /proc/sys/fs/binfmt_misc/register
   echo ':mipsel:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:'${QEMU_BIN_DIR}'/qemu-mipsel-static:' > /proc/sys/fs/binfmt_misc/register
echo ':mipsn32el:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:'${QEMU_BIN_DIR}'/qemu-mipsn32el-static:' > /proc/sys/fs/binfmt_misc/register

What happens when starting a mips image is that qemu-mipsn32-static is not found.

I worked around that issue (c.f. libjpeg-turbo/libjpeg-turbo#106) by creating a new image that adds another copy of qemu-mipsel-static with the name qemu-mipsn32el-static.

I don't know how this should/could be fixed (well, other than removing/commenting the n32 variants). Adding the real copy of qemu-mipsn32el-static to the image leads to qemu exiting with an error.

[junaruga]

@mayeut sorry for late response.
Can you try to run docker run --rm --privileged multiarch/qemu-user-static --reset -p yes to run standard mips/mipsel container images?
multiarch/* images are not required any more to run the arch's image.

I just searched it on DockerHub, then run below command. The uname command shows x86_64. I do not know if the result is correct.

https://hub.docker.com/search?q=mipsel&type=image

$ docker run --rm -t japaric/mipsel-unknown-linux-gnu uname -a
Linux 09f593170a06 5.1.18-300.fc30.x86_64 #1 SMP Mon Jul 15 15:42:34 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[aallrd]

Hello,
Just to comment that I think I had the same issue today.
I was trying to run a multiarch/debian-debootstrap:mips-wheezy image with Docker on my Mac but could not get it to work properly:

$ docker run -it --rm multiarch/debian-debootstrap:mips-wheezy uname -m
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped

However this was working fine:

$ docker run -it --rm multiarch/debian-debootstrap:mips-wheezy /usr/bin/qemu-mips-static /bin/uname -a
mips

From @mayeut's comment, it seems that the issue is that the mips/mipsel entry files are overwritten by the mipsn32/mipsn32el ones.
Looking at qemu-binfmt-conf.sh, it seems like the mipsn32* magic values are the same than mips* ones.

Since these files are shared between the host and the containers, I tried to remove them:

# access the Linux VM storing/running the containers on Mac
$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
# press enter
# remove the binfmt_misc mipsn32 entry files
$ find /proc/sys/fs/binfmt_misc -type f -name "*mipsn32*" -exec sh -c 'echo -1 > {}' \;
# exit
$ Ctrl-A k
$ y

After that, it was working as expected:

$ docker run -it --rm multiarch/debian-debootstrap:mips-wheezy uname -m
mips

Thank you @mayeut for the issue and writing about your findings :).

[timandy]

I have the same issue.

[jonnor]

@timandy make sure to check the workaround provided by @aallrd - it resolved the issue for me

[jonnor]

I belive this issue has been fixed in upstream qemu with the commit "qemu-binfmt-conf.sh: mips: allow nonzero EI_ABIVERSION, distinguish o32 and n32"
https://gitlab.com/qemu/qemu/-/commit/77d119dd335f910c7f953a265726e3753c69a0bb
https://gitlab.com/qemu-project/qemu/-/issues/843

However maybe a forked or old version of qemu-binfmt-conf.sh is used here? Can someone who know the project advise as to where that file comes from when running register as per the README?