ADB server does not ACK if it is launched inside Linux chrooted / jailed / emulated with the Linuxulator

UNIX FreeBSD Current
1

Hello to everyone.

I'm trying to connect to my mobile phone using adb over wifi using the Linuxulator because I want to install a specific java application that requires Linux to work. I have already used Linux virtualized with bhyve and it worked. But I prefer to save some memory trying to use the Linuxulator instead of starting a vm,if possible. I've already tried to run the app using java installed natively on FreeBSD,but it fails because it wasn't designed for this. So :

[root@noble /]==> adb connect 192.168.1.2:5555

* daemon not running. starting it now on port 5037 *
cannot bind 'tcp:5037'
ADB server didn't ACK
* failed to start daemon *
error: cannot connect to daemon

that's the error that I get. I'm not sure if I can use some trick or if there is anything that I can do because the Linuxulator does not support that in any way. Instead,if I launch it directly in FreeBSD,it works :

[root@marietto /home/marietto]==> adb connect 192.168.1.2:5555

* daemon not running; starting now at tcp:5037
* daemon started successfully
connected to 192.168.1.2:5555

The problem is that the java app does not recognizes that the adb server is working :

[root@noble /home/marietto/Desktop/Files/OS/Linux/Tools/DeskDockServer_1.3.0]==> java -jar DeskDockServer_1.3.0.jar

Program: DeskDockServer 1.3.0
System: Linux 5.15.0, amd64
JRE: 21.0.5+11-Ubuntu-1ubuntu124.04
AdbLocator: Found ADB in env PATH
class com.floriandraschbacher.deskdockserver.Main: 
Using ADB from /usr/bin/adb
ab@3f95b479: Error getting devices: 
ADB server didn't ACK
q$b@31524dfb: Error getting ADB devices: 
java.io.IOException: java.io.IOException: 
ADB server didn't ACK

but it is working :

[root@noble /home/marietto/Desktop/Files/OS/Linux/Tools/DeskDockServer_1.3.0]==> ps ax
      
  PID TTY      STAT   TIME COMMAND
 6089 pts/0    R+     0:00 ps ax
 5979 pts/0    S      0:00 adb -P 5037 fork-server server
 5952 pts/0    S      0:00 /bin/zsh

According with this thread :

this is what I tried to do :

nano /etc/pf.conf

nat on $ext_if from 127.0.0.0/24 to any -> 192.168.1.2
rdr on $ext_if proto tcp from any to any port 5037:5555 -> 127.0.0.255

root@noble:/home/marietto/Desktop/Files/OS/Linux/Tools/DeskDockServer_1.3.0# ./adb connect 

* daemon not running. starting it now on port 5037 *
* daemon started successfully *
** daemon still not running
error: cannot connect to daemon

root@noble:/home/marietto/Desktop/Files/OS/Linux/Tools/DeskDockServer_1.3.0# ps ax

  PID TTY      STAT   TIME COMMAND
 8759 pts/2    R+     0:00 ps ax
 8758 pts/2    S      0:00 adb -P 5037 fork-server server
 8739 pts/2    S      0:00 /bin/bash

but it didn't work :

root@noble:/home/marietto/Desktop/Files/OS/Linux/Tools/DeskDockServer_1.3.0# java -jar DeskDockServer_1.3.0.jar 

2024-11-22 17:33:02.348   Program: DeskDockServer 1.3.0
2024-11-22 17:33:02.348   System: Linux 5.15.0, amd64
2024-11-22 17:33:02.348   JRE: 21.0.5+11-Ubuntu-1ubuntu124.04
2024-11-22 17:33:02.469   AdbLocator: Found ADB in env PATH
2024-11-22 17:33:02.469   class com.floriandraschbacher.deskdockserver.Main: Using ADB from /usr/bin/adb
2024-11-22 17:33:02.508   ab@c703a44: Error getting devices: ADB server didn't ACK
2024-11-22 17:33:02.508   q$b@2b7d5ba: Error getting ADB devices: java.io.IOException: java.io.IOException: 
ADB server didn't ACK127.0.0.255192.168.1.2:5555
2

Hi @marietto

Have you tried to check that the Linuxulator networking stack is properly configured and supports the required TCP binding and routing. Run:

sysctl compat.linux

Let us know how that works out and then we will go from there!

3 
[root@marietto /home/marietto]==> sysctl compat.linux

compat.linux.use_real_ifnames: 0
compat.linux.emul_path: /compat/linux
compat.linux.oss_version: 198144
compat.linux.osrelease: 5.15.0
compat.linux.osname: Linux
compat.linux.setid_allowed: 1
compat.linux.map_sched_prio: 1
compat.linux.preserve_vstatus: 1
compat.linux.ignore_ip_recverr: 1
compat.linux.dummy_rlimits: 0
compat.linux.default_stacksize: 8388608
compat.linux.default_openfiles: 1024
compat.linux.debug: 3
4

Great.

Now, verify that the Linuxulator has the required adb binaries and permissions inside /compat/linux, like this:

ls -l /compat/linux/usr/bin/adb

How does that check out, @marietto ?

5

Its not so easy. This is the whole config :

nano /etc/fstab :

# Device		Mountpoint		  						FStype		Options	   Dump	Pass
	
fdesc			/dev/fd									fdescfs		rw	   0   	0
proc	       	/proc									procfs		rw	   0   	0


nano /etc/rc.conf :

ifconfig_em0="inet 192.168.1.5 netmask 255.255.255.0"
defaultrouter="192.168.1.8"
local_unbound_enable="YES"
pf_enable="YES"
jail_enable="YES"
jail_parallel_start="YES"
ifconfig_lo1_aliases="inet 10.10.0.1-12/24"


nano /boot/loader.conf :

if_tap_load="YES"
if_bridge_load="YES"
bridgestp_load="YES"
tmpfs_load="YES"
fdescfs_load="YES"
linprocfs_load="YES"
linsysfs_load="YES"
nullfs_load="YES"


nano /etc/jail.conf :

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";


nano /etc/jail.conf.d/noble.conf :

noble {
    host.hostname = "noble";
    path = "/mnt/da2p2/Backup/compat/noble";

    # permissions
    allow.raw_sockets;
    exec.clean;
    persist;
    sysvmsg=inherit;
    sysvsem=inherit;
    sysvshm=inherit;
    enforce_statfs=1;

    # permissions
    devfs_ruleset=7;

    # network
    ip4.addr="lo1|10.10.0.5/24";

    # mount
    mount += "devfs            $path/dev      		devfs           rw                      0       0";
    mount += "tmpfs            $path/dev/shm  		tmpfs           rw,size=1g,mode=1777    0       0";
    mount += "fdescfs          $path/dev/fd   		fdescfs         rw,linrdlnk             0       0";
    mount += "linprocfs        $path/proc     		linprocfs       rw                      0       0";
    mount += "linsysfs         $path/sys      		linsysfs        rw                      0       0";
    mount += "/tmp             $path/tmp      		nullfs          rw                      0       0";
    mount += "/var/run/user/   $path/run/user/   	nullfs 		rw 		        0       0";
}


nano start-noble-bash

xhost +
doas service jail stop noble
doas service jail start noble
doas jexec noble /bin/bash


nano /etc/sysctl.conf

net.link.tap.up_on_open=1
net.link.tap.user_open=1
net.inet.ip.forwarding=1
net.inet.ip.random_id=1
net.inet.tcp.sendspace=65536
net.inet.tcp.recvspace=65536


nano /etc/devfs.rules

[system=10]
add path 'dri/*' mode 0666 group operator
add path 'dri/*' mode 0666 group video
add path 'drm/*' mode 0666 group operator
add path 'drm/*' mode 0666 group video
add path 'ttyU*' mode 0660 group operator
add path 'tap*' mode 0660 group operator
add path 'ugen*' mode 0660 group operator

[jail=7]
add include $devfsrules_jail
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide
6

So :


[root@marietto /mnt/da2p2/Backup/compat/noble/usr/bin]==> ls -l /mnt/da2p2/Backup/compat/noble/usr/bin/adb
-rwxr--r--  1 root wheel 1199904 21 nov 15:50 /mnt/da2p2/Backup/compat/noble/usr/bin/adb
7

Yes, it looks like the adb binary path may not match the expected Linuxulator path structure (/compat/linux). The Linuxulator expects binaries to be located under /compat/linux/usr/bin, but your adb binary resides at /mnt/da2p2/Backup/compat/noble/usr/bin/adb.

Simplify the Path

1. Link the Correct Path in Standard Form

Create a symbolic link from the current adb binary to the expected Linuxulator path:

ln -s /mnt/da2p2/Backup/compat/noble/usr/bin/adb /compat/linux/usr/bin/adb

2. Verify the Link

Check if the link was created correctly:

ls -l /compat/linux/usr/bin/adb

You should see something like this:

lrwxrwxrwx  1 root  wheel  ...  /compat/linux/usr/bin/adb -> /mnt/da2p2/Backup/compat/noble/usr/bin/adb

3. Adjust the PATH

Ensure the Linuxulator environment can find the adb binary by adding /compat/linux/usr/bin to the PATH variable:

export PATH=/compat/linux/usr/bin:$PATH

Add this line to your shell configuration file (e.g., ~/.bashrc or ~/.zshrc) to make it permanent.

Update your config file to point to the newly created syslink in /etc/jail.conf.d/noble.conf

You should also test to make sure this config file above is correct, something like this:

jail -c -f /etc/jail.conf.d/noble.conf

I'm have never used this jail system, but often setting things up in a simple way and testing step-by-step can help solve these types of issues.

Otherwise, sorry I could not be more helpful, since I do not have a server with FreeBSD and have not used jail.

8

ok. This is what I did. on /compat,I've created the script called noble :

nano /compat/noble :

#!/bin/sh
#
# PROVIDE: noble
# REQUIRE: archdep mountlate
# KEYWORD: nojail
#
# This is a modified version of /etc/rc.d/linux
# Based on the script by mrclksr:
# https://github.com/mrclksr/linux-browser-installer/blob/main/rc.d/ubuntu.in
#

. /etc/rc.subr

name="noble"
desc="Enable noble chroot, and Linux ABI"
rcvar="noble_enable"
start_cmd="${name}_start"
stop_cmd=":"

noble_start()
{
    local _emul_path _tmpdir

    load_kld -e 'linux(aout|elf)' linux
    case `sysctl -n hw.machine_arch` in
    amd64)
        load_kld -e 'linux64elf' linux64
        ;;
    esac
    if [ -x /mnt/da2p2/Backup/compat/noble/sbin/ldconfigDisabled ]; then
        _tmpdir=`mktemp -d -t linux-ldconfig`
        /mnt/da2p2/Backup/compat/noble/sbin/ldconfig -C ${_tmpdir}/ld.so.cache
        if ! cmp -s ${_tmpdir}/ld.so.cache /mnt/da2p2/Backup/compat/noble/etc/ld.so.cache; then
            cat ${_tmpdir}/ld.so.cache > /mnt/da2p2/Backup/compat/noble/etc/ld.so.cache
        fi
        rm -rf ${_tmpdir}
    fi

    # Linux uses the pre-pts(4) tty naming scheme.
    load_kld pty

    # Handle unbranded ELF executables by defaulting to ELFOSABI_LINUX.
    if [ `sysctl -ni kern.elf64.fallback_brand` -eq "-1" ]; then
        sysctl kern.elf64.fallback_brand=3 > /dev/null
    fi

    if [ `sysctl -ni kern.elf32.fallback_brand` -eq "-1" ]; then
        sysctl kern.elf32.fallback_brand=3 > /dev/null
    fi
    sysctl compat.linux.emul_path=/mnt/da2p2/Backup/compat/noble

    _emul_path=/mnt/da2p2/Backup/compat/noble
}

load_rc_config $name
run_rc_command "$1"

and then,I've modified the script "start-noble-bash" like this one :

xhost +
/compat/./noble onestop  && sysctl compat.linux.emul_path=/mnt/da2p2/Backup/compat/noble && /compat/./noble onestart
doas pfctl -F all -f /etc/pf.conf
doas service jail stop noble
doas service jail start noble
doas jexec noble /bin/bash

At this point I've added these rules to /etc/pf.conf :

rdr pass on em0 proto tcp from any to 10.10.0.5 port 5037 -> 192.168.1.2 port 5037
pass in on em0 proto tcp from any to 10.10.0.5 port 5037

This is what happened when I ran the script "start-noble-bash" :

[root@marietto /compat]==> ./start-noble-bash
               
access control disabled, clients can connect from any host
egrep: warning: egrep is obsolescent; using /usr/local/bin/ggrep -E
egrep: warning: egrep is obsolescent; using /usr/local/bin/ggrep -E
compat.linux.emul_path: /mnt/da2p2/Backup/compat/noble -> /mnt/da2p2/Backup/compat/noble
Ethernet rules cleared
rules cleared
nat cleared
0 tables deleted.
20 states cleared
source tracking entries cleared
pf: statistics cleared
pf: interface flags reset
Stopping jails: noble.
Starting jails: noble.

root@noble:/# adb connect 192.168.1.2:5555
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
** daemon still not running
error: cannot connect to daemon

It didn't work. I suspect that the IP range assigned for the jail on /etc/rc.conf,I mean this :

ifconfig_lo1_aliases="inet 10.10.0.1-12/24"

and on /etc/jail.conf.d/noble.conf :

# network
    ip4.addr="lo1|10.10.0.5/24";

may be wrong. What do you think ? maybe it should be something like 192.168.1.100/24 ?

9

I have even performed another experiment where I haven't used the jail,but I kept pf enabled :

nano /etc/rc.conf :

ifconfig_em0="DHCP"
#ifconfig_vtnet0="DHCP"
ifconfig_em0="inet 192.168.1.5 netmask 255.255.255.0"
defaultrouter="192.168.1.8"
pf_enable="YES"
#jail_enable="YES"
#jail_parallel_start="YES"
#ifconfig_lo1_aliases="inet 10.10.0.1-12/24"

nano /etc/pf.conf :

rdr pass on em0 proto tcp from any to 192.168.1.5 port 5037 -> 192.168.1.2 port 5037
pass in on em0 proto tcp from any to 192.168.1.5 port 5037

and ADB has been able to connect !!!!

root@marietto:/# mount -t proc proc /proc

root@marietto:/# adb connect 192.168.1.2:5555
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
connected to 192.168.1.2:5555

root@marietto:/# uname
Linux

so,there is for sure some error in the jail network configuration settings.

1 Like
10

Hi @marietto

Consider these potential issues with Jail:

Jail Interface Configuration Issues

  1. The jail might not be properly configured to bind to the em0 interface or use the correct IP address.

  2. The jail needs explicit networking rules to allow traffic on specific ports like 5037.

PF Configuration for Jails

  1. PF might not be correctly redirecting traffic to the jail’s interface or IP address.

  2. PF rules need to explicitly account for the jail’s IP address and port bindings.

adb Port Binding

  1. If adb inside the jail cannot bind to tcp:5037, it may be due to overlapping configurations, restrictions, or missing permissions.

These are general suggestions, I guess you have already solved this issue by now @marietto

11

Nope. I didn't solve the issue. I even tried asking to chatgpt. I tried several permutations,but none of them worked. I have no idea why the jail is blocking the connection.

12

I don't know much about this jail setup; but I recall when I was first moving apps to docker containers, getting the port mappings right was tricky at first.

Maybe it's just a matter of getting these networking bindings set up correctly?

13

No idea.

14

I think I found the reason why ADB does not connect when I use the jail. The error :

linux: jid 2 pid 5717 (bash): syscall setfsgid not implemented

happens every time I do :

doas jexec noble

not every time I do :

adb connect

this is my /etc/jail.conf :

# startup/logging
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";
enforce_statfs = 0;

and this is my /etc/jail.conf.d/noble.conf :

noble {
host.hostname = "noble";
path = "/mnt/da2p2/Backup/compat/noble";

# permissions
allow.raw_sockets;
exec.clean;
persist;
sysvmsg=inherit;
sysvsem=inherit;
sysvshm=inherit;
enforce_statfs = 0;

# permissions
devfs_ruleset=7;

# network
ip4.addr="lo0|10.10.0.5/24";
15

The error syscall setfsgid not implemented indicates that the setfsgid system call is being attempted but isn't available or implemented in your jail's environment. This is a common issue when certain Linux-based tools (like ADB) are used in FreeBSD jails, particularly because FreeBSD and Linux have differences in syscall support.

Make sure you enable Linux compatibility in the jail, @marietto

16

Working configuration :

/etc/rc.conf :

ifconfig_em0="inet 192.168.1.5 netmask 255.255.255.0"
defaultrouter="192.168.1.8"
local_unbound_enable="YES"

cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7 tap8 tap9 tap10 tap11 tap12 tap13 tap14 tap15 tap16 tap17 tap18 tap19 tap20 em0 lo1"

ifconfig_bridge0="addm em0 addm tap0 addm tap1 addm tap2 addm tap3 addm tap4 addm tap5 addm tap6 addm tap7 addm tap8 addm tap9 addm tap10 addm tap11 addm tap12 addm tap13 addm tap14 addm tap15 addm tap16 addm tap17 addm tap18 addm tap19 addm tap20"

devfs_system_ruleset="system"
linux_enable="YES"
pf_enable="YES"

#jail_enable="YES"
#jail_parallel_start="YES"


/etc/pf.conf :

int_if="em0" # em0 interface
ext_if="ue0" # ue0 interface (ue0 or tap10)

nat on $int_if from {em0:network} to any -> ($int_if)
nat on $ext_if from {ue0:network} to any -> ($ext_if) (ue0 or tap10)

# forward port 5037
rdr pass on ue0 proto tcp from any to 192.168.1.234 port 5037 -> 192.168.1.2 port 5037
(ue0 or tap10)

# allow port 5037 to the jail
pass in on ue0 proto tcp from any to 192.168.1.234 port 5037
(ue0 or tap10)

/etc/jail.conf.d/noble.conf :


noble {
host.hostname = "noble";
path = "/mnt/da2p2/Backup/compat/noble";

    # permissions
    allow.raw_sockets;
    exec.clean;
    persist;
    sysvmsg=inherit;
    sysvsem=inherit;
    sysvshm=inherit;
    enforce_statfs=1;
    devfs_ruleset=7;

    # network
    ip4.addr="ue0|192.168.1.234/24";
(ue0 or tap10)

    # mount
    mount += "devfs            $path/dev              devfs           rw                      0       0";
    mount += "tmpfs            $path/dev/shm          tmpfs           rw,size=1g,mode=1777    0       0";
    mount += "fdescfs          $path/dev/fd           fdescfs         rw,linrdlnk             0       0";
    mount += "linprocfs        $path/proc             linprocfs       rw                      0       0";
    mount += "linsysfs         $path/sys              linsysfs        rw                      0       0";
    mount += "/tmp             $path/tmp              nullfs          rw                      0       0";
    mount += "/var/run/user/   $path/run/user/        nullfs          rw                      0       0";
    mount += "/var/run/dbus/   $path/run/dbus/        nullfs          rw                      0       0";
}


/etc/devfs.rules :

[system=10]
add path 'dri/*' mode 0666 group operator
add path 'dri/*' mode 0666 group video
add path 'drm/*' mode 0666 group operator
add path 'drm/*' mode 0666 group video
add path 'ttyU*' mode 0660 group operator
add path 'tap*' mode 0660 group operator
add path 'ugen*' mode 0660 group operator

[jail=7]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide


/etc/jail.conf :


# startup/logging
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";

# permissions
allow.raw_sockets;
exec.clean;
mount.devfs;


/compat/start-noble-bash-jail :


xhost +
vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (BE0191500218)/ && d{print d}'`
echo "G-DRIVE USB UFS ; $vmdisk1"

if [ "${vmdisk1}" = "da1" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da1.conf /etc/jail.conf.d/noble.conf
fi
if [ "${vmdisk1}" = "da2" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da2.conf /etc/jail.conf.d/noble.conf
fi
if [ "${vmdisk1}" = "da3" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da3.conf /etc/jail.conf.d/noble.conf
fi
if [ "${vmdisk1}" = "da4" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da4.conf /etc/jail.conf.d/noble.conf
fi
if [ "${vmdisk1}" = "da5" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da5.conf /etc/jail.conf.d/noble.conf
fi
if [ "${vmdisk1}" = "da6" ]; then
cp /etc/jail.conf.d/noble-disks/noble-da6.conf /etc/jail.conf.d/noble.conf
fi


/compat/./noble-linuxulator-start-jail onestop && /compat/./noble-linuxulator-start-jail onestart


doas pfctl -F all -f /etc/pf.conf
doas service jail onestop noble
doas service jail onestart noble
doas jexec noble /bin/bash


/compat/noble-linuxulator-start-jail :


#!/bin/sh
#
# PROVIDE: noble
# REQUIRE: archdep mountlate
# KEYWORD: nojail
#
# This is a modified version of /etc/rc.d/linux
# Based on the script by mrclksr:
# https://github.com/mrclksr/linux-browser-installer/blob/main/rc.d/ubuntu.in
#

. /etc/rc.subr

name="noble"
desc="Enable noble chroot, and Linux ABI"
rcvar="noble_enable"
start_cmd="${name}_start"
stop_cmd=":"

vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (BE0191500218)/ && d{print d}'`
echo "G-DRIVE USB UFS ; $vmdisk1"

noble_start()
{
    local _emul_path _tmpdir

    load_kld -e 'linux(aout|elf)' linux
    case `sysctl -n hw.machine_arch` in
    amd64)
        load_kld -e 'linux64elf' linux64
        ;;
    esac
    if [ -x /mnt/$vmdisk1'p2'/Backup/compat/noble/sbin/ldconfigDisabled ]; then
        _tmpdir=`mktemp -d -t linux-ldconfig`
        /mnt/$vmdisk1'p2'/Backup/compat/noble/sbin/ldconfig -C ${_tmpdir}/ld.so.cache
        if ! cmp -s ${_tmpdir}/ld.so.cache /mnt/$vmdisk1'p2'/Backup/compat/noble/etc/ld.so.cache; then
            cat ${_tmpdir}/ld.so.cache > /mnt/$vmdisk1'p2'/Backup/compat/noble/etc/ld.so.cache
        fi
        rm -rf ${_tmpdir}
    fi

    # Linux uses the pre-pts(4) tty naming scheme.
    load_kld pty

    # Handle unbranded ELF executables by defaulting to ELFOSABI_LINUX.
    if [ `sysctl -ni kern.elf64.fallback_brand` -eq "-1" ]; then
        sysctl kern.elf64.fallback_brand=3 > /dev/null
    fi

    if [ `sysctl -ni kern.elf32.fallback_brand` -eq "-1" ]; then
        sysctl kern.elf32.fallback_brand=3 > /dev/null
    fi
    sysctl compat.linux.emul_path=/mnt/$vmdisk1'p2'/Backup/compat/noble

    _emul_path=/mnt/$vmdisk1'p2'/Backup/compat/noble
}

load_rc_config $name
run_rc_command "$1"