Copy and rename folders and replace text strings in for loop in bash

Dear all

I'm a bash newbie trying to do the following. I currently have a folder called "100-mT", which contains several subfolders. I am looking for a way to perform the following steps in each iteration in a for loop (for idx in 200 300 400 500 600 700 800 900 1000) in the bash environment:

(i) Copy the folder "100-mT", and rename it to "idx-mT" (I would also be interested in knowing how, when idx=1000, the folder name becomes "1-T", but it is secondary)
(ii) Search in each of the subfolders that make up the new folder "idx-mT" for a file called "input", and replace in the text string "sim:applied-field-strength = 100 !mT" with "sim:applied-field-strength = idx !mT" (I would be interested in replacing the aforementioned line, when idx=1000 by "sim:applied-field-strength = 1 !T")

As information that may be useful, the folder "100-mT" has the following subfolders (each new line implies that the folders cited are within the above):

(i) "Rectangular-Sample", "Square-Sample"
(ii) "0-K", "2-K"
(iii) "Dipolar-Hierarchical", "Dipolar-Tensorial"
(iv) "Scaled-DMI", "Unscaled-DMI"
(v) "D3-D1-1", "D3-D1-1with2", "D3-D1-1with4", "D3-D1-1with6", "D3-D1-1with8", "D3-D1-2", "D3-D1-2with2", "D3-D1-2with4", "D3-D1-2with6", "D3-D1-2with8", "D3-D1-3"

Within these last folders are the "input" files that I referred to, which have different lines, line 28 being the aforementioned text entry "sim:applied-field-strength = 100 !mT".

My attempt at the moment is as follows

for idx in 200 300 400 500 600 700 800 900 1000;
do
    # Let's copy, paste, and rename the relevant folder 
    if "$idx"<1000;
    then
       cp -r 100-mT "$idx"-mT;
       echo "$idx-mT";
       cd "$idx-mT";
    else
       cp -r 100-mT 1-T;
       cd "1-T";
    fi
    # Let's create and loop over all subfolders
    for Geometry in Rectangular-Sample Square-Sample;
    do
        echo "$Geometry";
        cd "$Geometry";
        for Temperature in 0-K 2-K;
        do
            echo "$Temperature";
            cd "$Temperature";
            for Dipolar in Dipolar-Hierarchical Dipolar-Tensorial;
            do
                echo "$Dipolar";
                cd "$Dipolar";
                for DMI in Scaled-DMI Unscaled-DMI;
                do
                    echo "$DMI";
                    cd "$DMI";
                    for DMI_Value in D3-D1-1 D3-D1-1with2 D3-D1-1with4 D3-D1-1with6 D3-D1-1with8 D3-D1-2 D3-D1-2with2 D3-D1-2with4 D3-D1-2with6 D3-D1-2with8 D3-D1-3;
                    do
                        echo "$DMI_Value";
                        cd "$DMI_Value";
                        # Let's replace the numerical value of the relevant text string
                        if "$idx"<1000;
                        then
                            sed 's/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = "$idx" !mT' <input >input;
                        else
                            sed 's/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = 1 !T' <input >input;
                        fi
                        cd  ..
                    done
                    cd ..
                done
                cd ..
            done
            cd ..
        done
        cd ..
    done
    cd ..
done

It seems that this approach it is not able to create the "$idx-mT" folders when $idx is lower than 1000. It creates correctly the "1-T" folder, but it does not change the relevant line on the input files thereon.

Any ideas?

Hi @rimaroa

Your script has a few issues, particularly with conditional syntax in Bash and incorrect use of sed.

Suggest you use a tool like shellcheck below to check your basics:

You can also use ChatGPT 4o which will also do a kind of shellcheck but in it's on genAI kinda-way, for example, the overly-friendly collaborative machine intelligence (CMI) suggests:

Untested ChatGPT 4o Suggestions

#!/bin/bash

# Iterate over specified values of idx
for idx in 200 300 400 500 600 700 800 900 1000; do
    # Determine folder name based on idx
    if [ "$idx" -lt 1000 ]; then
        new_folder="${idx}-mT"
    else
        new_folder="1-T"
    fi
    
    # Copy and rename the folder
    cp -r 100-mT "$new_folder"
    echo "Created folder: $new_folder"

    # Search all subdirectories for input files and replace the target line
    find "$new_folder" -type f -name "input" | while read -r input_file; do
        if [ "$idx" -lt 1000 ]; then
            sed -i "s/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = ${idx} !mT/" "$input_file"
        else
            sed -i "s/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = 1 !T/" "$input_file"
        fi
        echo "Updated file: $input_file"
    done
done

Here are the details on the issues.

The if takes a command, so it tries to run "$idx" with input redirected from file 1000.
Correct is POSIX (all standard shells)
if [ "$idx" -lt 1000 ]
or bash (not yet POSIX)
if (( idx<1000 ))

$Geometry is relative to the current directory, and changes the current directory; the next iteration will descend further/deeper. You correct this by a cd .. at the end, but this is on thin ice. E.g. if a cd failed it will go up neverthess, and the loop will continue at the wrong level.
Improvement:

if cd "$Geometry"
then
  ...
  cd ..
fi

Generally this "race condition" will corrupt the file.
A correction is >input.tmp; mv input.tmp input.
sed does this automatically in
sed -i ... input
(Note that mv creates a new inode that breaks a hard-link to the file. Use cp and rm if there are hard-links.)

The shell will not evaluate the "$idx" because it is within a 'string'
A fix is a concatenation 'string'"string"'string':
sed 's/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = '"$idx"' !mT'

Well, this is how 1000 milliTeslas = 1 Tesla, just the same as 1000 millimeters = 1 meter. But it doesn't really matter, as you won't need to worry about any SI units conversion or any other "advanced work" on prefixes - because you know the target values, so you define all target directory names yourself (and copy identical contents of only a single directory to all of them).

Perhaps:

#!/usr/bin/env bash
: || {  # comment this entire line and the line with '}' at the bottom (or remove ': ||'), to make this code block applicable
# this part was used only for sample environment preparation (as per OPs problem description):
if [[ ! -d 100-mT ]]; then
  mkdir -p 100-mT/{Rectangular,Square}-Sample/{0,2}-K/Dipolar-{Hierarchical,Tensorial}/{S,Uns}caled-DMI/D3-D1-{{1,2}{,with{2..8..2}},3}
  printf "%s" 'line '{1..27}$'\n' | tee 100-mT/{Rectangular,Square}-Sample/{0,2}-K/Dipolar-{Hierarchical,Tensorial}/{S,Uns}caled-DMI/D3-D1-{{1,2}{,with{2..8..2}},3}/input
  echo "sim:applied-field-strength = 100 !mT" | tee -a 100-mT/{Rectangular,Square}-Sample/{0,2}-K/Dipolar-{Hierarchical,Tensorial}/{S,Uns}caled-DMI/D3-D1-{{1,2}{,with{2..8..2}},3}/input
  printf "%s" 'line '{29..40}$'\n' | tee -a 100-mT/{Rectangular,Square}-Sample/{0,2}-K/Dipolar-{Hierarchical,Tensorial}/{S,Uns}caled-DMI/D3-D1-{{1,2}{,with{2..8..2}},3}/input
fi
} # comment this entire line and the line with ': || {' at the top (or remove ': ||'), to make this code block applicable

# main part of the solution:
[[ ! -d "100-mT" ]] && exit 1; # there's no source directory here - nothing to copy 

for idx in {100..900..100}-mT 1-T; do
  [[ ! -d "${idx}" ]] && cp -r 100-mT "${idx}";
 # also checks the target directory existence first,
 # to avoid recursive copying into an existing directory
done

for input_file_path in {{1..9}00-mT,1-T}/{Rectangular,Square}-Sample/{0,2}-K/Dipolar-{Hierarchical,Tensorial}/{S,Uns}caled-DMI/D3-D1-{{1,2}{,with{2..8..2}},3}/input; do
  TOP_LEVEL_DIR="${input_file_path%%/*}";
  AMOUNT="${TOP_LEVEL_DIR//[!0-9]/}";
  UNIT="${TOP_LEVEL_DIR//[!a-zA-Z]/}"

  echo "Input file: ${input_file_path} / Directory: '${TOP_LEVEL_DIR}' / Substitute: '${AMOUNT} !${UNIT}'";

  if [[ ! -f "${input_file_path}" ]]; then
    echo "${input_file_path} does not exist!";
  elif [[ ! -w "${input_file_path}" ]] || [[ ! -w "${input_file_path%/*}" ]]; then
    echo "Cannot modify ${input_file_path} - check the file and directory permissions/ownership!";
  else
   sed -i "s/sim:applied-field-strength = 100 !mT/sim:applied-field-strength = ${AMOUNT} !${UNIT}/" "${input_file_path}" || echo "${input_file_path} modification failed!";
  fi
done