Use a bash variable where a file is needed

 · 2 min · torgeir

With bash you can use a variable with newline separated content where a file is needed. Or rather, pass your content as an _unnamed pipe_.

Terminal Bash Pipes Kotlin Gradle

Some time ago I needed a way to build a bunch of select gradle modules, independently, and I discovered the bash syntax <(cmd). It creates an unnamed pipe with the content that is output from the command.

Bash unnamed pipes

I used this to skip a list of newline separated modules from being built, by filtering a list of folders, to keep only those that did not exist in the file I gave to grep -v -f <file>. This way it was not nescessary to clean up intermediary files afterwards.

man grep

-f file, --file=file
        Read  one  or more newline separated patterns from file.  Empty pattern lines match every input line.  Newlines are not considered part of a pattern.
        If file is empty, nothing is matched.
...
-v, --invert-match
        Selected lines are those not matching any of the specified patterns.

By using cat <<EOF to build newline separated content in the variable skip_modules, it can be passed to grep pretending to be a file, using <(echo "$skip_modules"). An unnamed pipe! It looked like this

skip_modules=$(cat <<EOF
folders
to
skip
EOF
)

# build all modules idependently, except excluded ones found in skip_modules
for build in $(find . -name settings.gradle.kts | xargs dirname | grep -v -f <(echo "$skip_modules")); do
  pushd $build
  ./gradlew clean build "$@"
  popd
done

You could have used a gradle composite build!

I could have. I did, once. That looked like this

fun includeBuild(moduleSettingsGradleKts: String) {
    val module =
        moduleSettingsGradleKts
        .replace("${rootDir.path}/", "")
        .replace("/settings.gradle.kts", "")
    println(":: includeBuild($module)")
    includeBuild(module)
}

val sharedModulesPath = "modules"
includeBuild("$rootDir/$sharedModulesPath/settings.gradle.kts")

// automatically create a composite build that includeBuild()-s all top level
// deployables, which have standalone gradle builds
var topLevelModules =
    rootDir.listFiles()!!
    .filter { it.isDirectory }
    .flatMap { directory ->
        directory.walkTopDown()
            .maxDepth(2)
            .filter { it.isFile }
            .filter { it.name == "settings.gradle.kts" }
            .filter { it.parentFile.name != rootProject.name }
            .filter { !it.path.contains(sharedModulesPath) }
            .map { file -> file.path }
    }

topLevelModules.forEach(::includeBuild)

Resources