Capturing the process substitution exit code in Bash

Process substitutions in Bash is a very nice feature which in many cases helps to avoid temporary files. One downside of it, however, is the lack of controls for communicating the exit code of the command in a process substitution. Process substitution provides the output of a command via a file descriptor but doesn’t tell whether the output is complete or correct. Did the command produce no output or did it fail before producing any output?

There is a way to capture the exit code of the process substitution using coproc:

(
  coproc { echo Hello World; false; }
  cpid="$COPROC_PID"

  diff -du <(cat <&${COPROC[0]}) <(echo Goodbye World)

  wait "$cpid"

  echo $?
)

outputs:

--- /dev/fd/61	2022-01-18 22:36:10.201792655 +0000
+++ /dev/fd/59	2022-01-18 22:36:10.200792655 +0000
@@ -1 +1 @@
-Hello World
+Goodbye World
1

In the above example, the first process substitution forwards the output of the echo command running in a coprocess. The wait command will wait for the coprocess to finish and will exit with the exit code of the coprocess command (which ends with the false command, so the exit code is going to be 1).

It is very important to save the value of $COPROC_PID as soon as possible because it will not be available once the coprocess command has finished executing.

Also, bear in mind, that Bash (as of version 5) only supports one coprocess at a time, so it is currently not possible to apply this approach to multiple parallel process substitutions.

Similar to background processes with job control disabled, the command executing via coproc will have its standard file descriptors detached from the running terminal. Some programs are picky about that when they want to prompt on TTY (notably pinentry-curses used by gpg-agent, and by pass as a result), and sometimes it is simply desired to allow the coprocess to read from the parent’s STDIN. It is possible to forward the STDIN from the parent process to the coprocess like this:

(
  exec {stdin_fd_copy}<&0

  coproc pass Kubernetes/admin.key <&${stdin_fd_copy}
  cpid="$COPROC_PID"

  kubectl config set-credentials admin \
          --client-key <(cat <&${COPROC[0]}) \
          --client-certificate admin.crt \
          --embed-certs

  wait "$cpid"

  echo $?
)

File descriptor inheritance

$ diff -du <(cat) - <<< "foo"
cat: -: Input/output error
--- /dev/fd/63	2021-07-04 22:02:49.684001000 +0100
+++ -	2021-07-04 22:02:49.686762850 +0100
@@ -0,0 +1 @@
+foo
$ { diff -du <(cat) -; } <<< "foo"
--- /dev/fd/63	2021-07-04 22:03:04.119001000 +0100
+++ -	2021-07-04 22:03:04.121612176 +0100
@@ -1 +0,0 @@
-foo

In the first case only diff gets “foo” as STDIN. In the second case both diff and cat get the same STDIN FD and cat wins the race as it executes earlier.

Keep that in mind when re-directing STDIN for functions.

Get the IP address of the default route interface

Requires iproute2 version 4.14.1 or later:

ip -json route get $(ip -json route show 0.0.0.0/0 | jq -r '.[0]|.gateway') | jq -r '.[0]|.prefsrc'

Nemawashi

Nemawashi (根回し) in Japanese means an informal process of quietly laying the foundation for some proposed change or project, by talking to the people concerned, gathering support and feedback, and so forth. It is considered an important element in any major change, before any formal steps are taken, and successful nemawashi enables changes to be carried out with the consent of all sides.

Wikipedia

Machine-readable LVM report

lvm fullreport --reportformat json

ThinkPad P52s charging/Thunderbolt issues

This morning after waking my ThinkPad P52s (Windows 10 x64) from sleep the following happened:

  1. The “A Thunderbolt controller has experienced a problem and cannot connect to devices or other computers until new firmware is installed” dialog box popped up several times. However, it no longer shows up. I believe (after some googling) this means the controller went into “safe mode” for some reason and will only accept firmware updates now.
  2. Both USB-C ports stopped working properly (peripherals connected to either of those ports are not functioning). I was able to use those ports just fine few days ago.
  3. The 65W Lenovo power adapter is now detected as “15W USB-C power” instead of “AC power”. This results in batteries charging only when power consumption does not exceed 15W (i.e. when laptop is powered off or idle). Most likely due to the same reason the following message is now shown during every reboot: “The connected AC adapter has a lower wattage than the recommended model which was shipped with the system. To boot with the AC adapter, please connect the AC adapter which was shipped with the system. Press Esc to continue."

Thunderbolt controller is enabled in BIOS. “Thunderbolt in BIOS Assist Mode” is disabled.

It looks like Power Delivery is not being negotiated between the power adapter and the laptop due to (malfunctioning?) Thunderbolt controller.

What I’ve tried so far:

  • Plugging power adapter to both USB-C ports with the same result (see 3. above)
  • Updating OS to the latest version using Windows Update
  • Applying all the updates in Lenovo Vantage (including BIOS update to 1.22)
  • Re-installing the latest available version of Thunderbolt Software (17.4.78.500)
  • Re-installing and running the latest version of the Thunderbolt Firmware Update Tool (17.0.0.1-N27TF15W). The tool is not able to detect any Thunderbolt controllers and exits with the following message: “No active Thunderbolt(TM) controller found in the system or Thunderbolt(TM) software is not present in the system. Make sure you have Thunderbolt(TM) software installed. Then connect Thunderbolt(TM) device and retry." I can see it tries to use the “force power” feature to wake the controller without any success.
  • Tried booting into Ubuntu Linux and triggering “force power” myself with echo 1 >/sys/bus/wmi/devices/86CCFD48-205E-4A77-9C48-2021CBEDE341/force_power, but the controller does not show up under /sys/bus/thunderbolt. lspci just hangs for an unknown reason.
  • Disconnected the power source, took out the removable battery and disabled the internal battery in BIOS. Also tried disconnecting the internal battery by inserting a paperclip into the emergency-reset hole on the bottom of the laptop.

Unfortunately I do not have any true Thunderbolt peripherals (only USB 1.0/2.0 with USB-C adapter cables) to plug in an see if that will wake the Thunderbolt controller up.

It is unclear whether it is a hardware issue or some catch 22 with controller not waking up because it wants the firmware update and firmware update not being applied because the controller is not waking up.

UPD: Had the motherboard replaced few days ago which fixed the issue for myself. But it looks like Lenovo has acknowledged the issue and the fix for P52s is pending: https://pcsupport.lenovo.com/gb/en/solutions/ht508988

2019-07-03

RHEL8 is out

ICS stops working after a reboot

There are numerous reports (first, second, third results on Google) on the Internet about an issue with Internet Connection Sharing in Windows 10. Since the 1607 release (or 1703 for some people) ICS stops working after a reboot. The only way to get it working again for me was to disable and then re-enable the sharing. Resetting Windows Firewall settings, suggested by some people, didn’t work.

You can automate the disable/enable steps with a simple Powershell script run as a scheduled task on system startup, like this one:

$m = New-Object -ComObject HNetCfg.HNetShare

$private = $m.EnumEveryConnection |? { $m.NetConnectionProps.Invoke($_).Name -eq "Loopback0" }
$private_config = $m.INetSharingConfigurationForINetConnection.Invoke($private)

$public = $m.EnumEveryConnection |? { $m.NetConnectionProps.Invoke($_).Name -eq "Wi-Fi" }
$public_config = $m.INetSharingConfigurationForINetConnection.Invoke($public)

$private_config.DisableSharing()
$public_config.DisableSharing()

Start-Sleep 1

$public_config.EnableSharing(0)
$private_config.EnableSharing(1)

if you’re after a quick and easy fix. Some users reported Windows Firewall filling up with duplicate rules created on every enable/disable of the ICS though.

There is a better fix, however.

Dissatisfied with all proposed workarounds I’ve tried searching for the reports on this issue on Microsoft TechNet directly. The issue has been there for years, so there are plenty.

One thread mentioned creating the DWORD HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\SharedAccess\EnableRebootPersistConnection registry key, setting it to 1 and setting the Internet Connection Sharing service startup type to “Automatic”. This fixed the problem!

The mentioned registry key was introduced with the 1709 release in KB4054517. See KB4055559 for more information.

Synchronizing multiple subshells in Bash

Imagine you want to diff outputs of two scripts which operate on the same resource (file). Compare contents of the file before and after a transformation in a single pass, for example.

With process substitution, like diff <(cat myfile) <(reformat myfile; cat myfile) both substitutions are started in separate subshells at the same time, so you would need a semaphore, to ensure the second process substitution is started after after the first one - otherwise you might hit a race condition.

One option is to use a named pipe for that, but it involves the creation (and cleanup) of the pipe device file.

Another option is to use a co-process (Bash 4+) for that. cat command running as the co-process would basically act as an anonymous pipe!

echo BEFORE >data.txt

(
    coproc cat

    diff -du \
        <(sleep 5; cat data.txt; echo >&"${COPROC[1]}") \
        <(read <&"${COPROC[0]}"; echo AFTER >data.txt; cat data.txt)
)

(sleep is for demonstration purposes only)

will output

--- /dev/fd/62	2018-11-15 21:41:17.771271807 +0000
+++ /dev/fd/61	2018-11-15 21:41:17.771271807 +0000
@@ -1 +1 @@
-BEFORE
+AFTER