Skip to content

Make the root dir the invocation dir, not the config dir - #438

Merged
kindermax merged 7 commits into
masterfrom
fix/root-dir-semantics
Aug 10, 2026
Merged

Make the root dir the invocation dir, not the config dir#438
kindermax merged 7 commits into
masterfrom
fix/root-dir-semantics

Conversation

@kindermax

@kindermax kindermax commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Why

0.0.63 changed where commands run, as a side effect of a checksum fix (9859486). Before it, every command silently got Command.WorkDir = filepath.Abs("") — the process cwd — which shadowed Config.WorkDir at runner.go:48. Adding an if cmd.WorkDir != "" guard un-shadowed a field that had been dead since 2020, and commands started running in the config file's directory:

$ lets55 -c data/lets.shafa.yaml validate stg a b
/Users/max/code/github/lets/b:/b
$ lets63 -c data/lets.shafa.yaml validate stg a b
/Users/max/code/github/lets/data/b:/b      # <- changed

Neither version was coherent. Each resolved paths against two different directories within a single command definition:

0.0.62 0.0.63
cmd: cwd invocation dir config dir
env.sh cwd invocation dir invocation dir
checksum: config dir config dir / work_dir
env_file: config dir config dir
work_dir: base invocation dir invocation dir

So on 0.0.62, checksum: [data.txt] and cmd: cat data.txt in the same command could mean two different files. On 0.0.63, cmd: and env.sh disagreed instead.

What

One rule, two halves:

Config assembly resolves against the config file that declares it.
Everything a command reads or runs resolves against that command's working dir — the dir lets was invoked from, or work_dir if set.

mixins: is the only config-assembly directive, and the only thing still relative to the config file — it is an include, so it has to resolve the same way wherever lets runs. cmd:, checksum:, env_file: and env.sh all now share one directory.

The root dir is where you ran lets, whatever config was loaded and wherever it lives:

you run commands run in
cd proj && lets x proj
cd proj && lets -c sub/lets.yaml x proj
cd proj/deep && lets x (config found up the tree) proj/deep
cd proj/deep && lets -c ../lets.yaml x proj/deep
cd proj && lets -c https://…/lets.yaml x proj

--config-dir / LETS_CONFIG_DIR still only steer which config is found. To act on the project rather than your current directory, use $LETS_CONFIG_DIR.

Breaking changes

  • Commands run in the invocation dir, not the config dir (restores 0.0.62 and earlier for the common case).
  • checksum: and env_file: resolve against the command's working dir. env_file previously resolved against the config dir at both scopes, and did not follow work_dir.
  • .lets/ is created in the root dir, so persisted checksums stay paired with the files they were computed from.
  • LETS_CONFIG_DIR for a remote config is now the local cache dir rather than the cwd.
  • A remote config declaring a local mixins path is now an error instead of silently resolving against the cwd — its config dir only ever holds the downloaded yaml.

Structure

  1. 576e8ea split Config.WorkDir into ConfigDir / RootDir — pure refactor, verified behaviour-identical before anything else moved. The conflation of these two meanings in one field is why the bug existed and went unnoticed for six years.
  2. 7401d82 the semantics.
  3. 65d219c docs — new Where commands run section, plus the statements commit 2 invalidated.
  4. 1974130 tests for the changed rules.
  5. 329c4f9 tighten the remote LETS_CONFIG_DIR assertion.
  6. 3e4bb42 track an empty fixture dir.

Testing

163 bats tests pass from a clean clone, unit tests pass, lets lint clean.

New coverage for behaviours that had none: checksum / env_file / env.sh following work_dir and defaulting to the root; mixins resolving against the config dir when the root is elsewhere; a remote config rooting at the cwd; a remote config rejecting a local mixin. tests/root_dir.bats pins each way of pointing lets at a config.

Two existing tests asserted the old behaviour and were rewritten rather than deleted — .lets location, and checksum-from-a-subdir (that one now has real fixture files with different contents in subdir/, so it proves the resolution rather than observing an empty hash).

Writing the remote-mixin test found a real bug: LoadRemote set RemoteSource after decoding, but mixins resolve during decode, so the new rejection never fired. Fixed in 1974130.

Notes for review

  • .lets/ moving to the root dir is the judgement call I am least certain of. The argument for it is checksum integrity — a persisted checksum stored next to a config but computed from a different directory would flip-flop. The cost is stray .lets/ dirs when running from subdirectories. One line in find.go to flip back.
  • work_dir is now relative to where you stand, so work_dir: docs works from the project root and fails from a subdirectory. That is inherent to the model. It also does not expand env vars, so work_dir: ${LETS_CONFIG_DIR}/docs does not work; docs recommend cmd: cd "${LETS_CONFIG_DIR}/docs" && … instead. Adding expansion would be a separate feature.
  • Recursive mixins are disallowed (config.go:342), so a mixin's own mixins: block is never read. Commit 2 passes the mixin's absolute path to NewMixinConfig (clearing an old TODO(maybe bug)), but that has no user-visible effect today.

Summary by Sourcery

Align command execution semantics so that commands run from the directory where lets is invoked, with a clear separation between the config file’s directory and the root working directory.

Bug Fixes:

  • Ensure work_dir, checksum, env_file, and env.sh all resolve consistently against the command’s working directory instead of mixing config and invocation directories.
  • Make remote configs reject local mixin paths explicitly and correctly apply remote mixin handling during load.

Enhancements:

  • Introduce distinct ConfigDir and RootDir concepts and centralize command working directory resolution via Config.CommandWorkDir.
  • Create .lets/ in the root directory so persisted checksums stay with the files they were computed from.
  • Have builtin env and runtime env variables reflect the new root/config directory semantics, including for remote configs.
  • Update mixin resolution to be based on the config directory and to propagate remote/URL-only constraints through nested mixins.

Documentation:

  • Add a "Where commands run" section and update config, env, work_dir, remote config, and changelog docs to describe the new root dir and resolution rules.

Tests:

  • Add and expand bats and Go tests to pin root-dir behaviour, work_dir behaviour, checksum resolution, env_file/env.sh resolution, mixin path handling, and remote config behaviour, including new fixtures for differing file contents.

@sourcery-ai

sourcery-ai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

This PR decouples the config file directory from the command execution directory by introducing explicit RootDir and ConfigDir concepts, making commands consistently resolve all file paths against the invocation/root directory while keeping mixin resolution relative to the config location, updating runtime/env behavior, .lets placement, remote config handling, and tests/docs accordingly.

Sequence diagram for resolving RootDir/ConfigDir and running a command

sequenceDiagram
actor User
participant lets
participant FindConfig
participant loadConfigFromFile
participant Config
participant shellRunner
participant Command
participant Envs
participant EnvFiles

User->>lets: invoke lets (configName, configDirFlag)
lets->>FindConfig: FindConfig(configName, configDirFlag)
FindConfig->>FindConfig: getSearchDir(configName, configDirFlag)
FindConfig->>FindConfig: path.GetFullConfigPath / GetFullConfigPathRecursive
FindConfig-->>lets: PathInfo{AbsPath, ConfigDir, RootDir, DotLetsDir}

lets->>loadConfigFromFile: loadConfigFromFile(AbsPath, RootDir, DotLetsDir, displayName, remoteSource, version)
loadConfigFromFile->>Config: NewConfig(RootDir, AbsPath, DotLetsDir)
loadConfigFromFile->>Config: RemoteSource = remoteSource
loadConfigFromFile-->>lets: *Config

User->>lets: run command
lets->>shellRunner: run(command, cmdScript)
shellRunner->>Config: CommandWorkDir(command)
Config-->>shellRunner: workDir
shellRunner->>Command: GetEnv(*Config, workDir, defaultEnv)
Command->>Envs: Execute(shell, workDir, baseEnv)
Envs->>EnvFiles: Load(workDir, filenameEnv)
EnvFiles-->>Command: env from files
Command-->>shellRunner: resolved env
shellRunner->>shellRunner: osCmd.Dir = workDir
shellRunner-->>User: command runs in RootDir or command.WorkDir
Loading

File-Level Changes

Change Details Files
Introduce RootDir/ConfigDir in config loading and make command execution, checksum, env.sh, and env_file consistently resolve against the command working directory (root dir by default).
  • Replace Config.WorkDir with ConfigDir and RootDir fields and add CommandWorkDir helper
  • Change FindConfig to compute ConfigDir from the config path and RootDir from the process cwd, with .lets following RootDir
  • Update executor runner and checksum handling to use CommandWorkDir instead of Command.WorkDir
  • Refactor env and env_file resolution to take shell/baseDir explicitly so sh and checksum globs run in the command work dir and env_file paths follow it
internal/config/config/config.go
internal/config/find.go
internal/config/workdir.go
internal/executor/runner.go
internal/executor/executor.go
internal/config/config/env.go
internal/config/config/env_file.go
internal/config/config/command.go
Clarify and enforce semantics for remote configs, including mixin resolution and LETS_CONFIG_DIR/BuiltinEnv behavior.
  • Make LoadRemote and loadConfigFromFile set RemoteSource before YAML decode so mixin logic can differentiate remote configs
  • Change BuiltinEnv to use ConfigDir for LETS_CONFIG_DIR even for remote configs and keep LETS_CONFIG as the URL
  • Reject local mixin paths in remote configs and propagate RemoteSource into mixin configs
  • Adjust migration and path collection to use ConfigDir for mixin path discovery
internal/config/load.go
internal/config/config/runtime_env.go
internal/config/config/config.go
internal/config/migrate/migrate.go
internal/config/load_test.go
Add and adjust tests to pin the new root-dir behavior, mixin resolution rules, env/env_file behavior, checksum resolution, and remote config constraints.
  • Add root_dir.bats and fixtures to cover various ways of pointing lets at configs and work_dir resolution
  • Extend command_work_dir tests and fixtures so checksum, env_file, and env.sh are proven to follow work_dir and root
  • Update checksum tests to differentiate root vs subdir content, and adjust .lets location expectations
  • Add load_test cases for RootDir vs ConfigDir and remote config LETS_CONFIG_DIR, plus remote mixin rejection tests
  • Update env and env_file unit tests to use RootDir/baseDir signatures and CommandWorkDir
tests/root_dir.bats
tests/root_dir/lets.yaml
tests/root_dir/sub/lets.yaml
tests/command_work_dir.bats
tests/command_work_dir/lets.yaml
tests/command_work_dir/.env.wd
tests/command_work_dir/input.txt
tests/command_work_dir/project/.env.wd
tests/command_work_dir/project/input.txt
tests/command_checksum.bats
tests/find_config.bats
internal/config/load_test.go
internal/config/config/env_execute_test.go
internal/config/config/env_file_test.go
internal/executor/execute_test.go
tests/mixins.bats
tests/mixins/lets.yaml
tests/mixins/sub/outer.yaml
tests/command_checksum/subdir/foo_1.txt
tests/command_checksum/subdir/foo_2.txt
tests/command_checksum/subdir/bar_1.txt
Update documentation to describe the root-dir model, new work_dir semantics, env resolution, remote config behavior, and changelog entries.
  • Add a 'Where commands run' section explaining RootDir, ConfigDir, and how cmd/checksum/env_file/env.sh resolve
  • Clarify work_dir as relative to the root dir and that it moves all command-touching resolution, with examples
  • Adjust env_file and command env docs to reference the root dir/command work dir instead of the config dir
  • Document remote config behavior, including root-dir equivalence, URL-only mixins, and updated LETS_CONFIG_DIR
  • Add changelog entries for the breaking behavior changes and fixes around work_dir, checksums, .lets, and remote mixins
docs/docs/config.md
docs/docs/env.md
docs/docs/changelog.md
Add a real-world example config fixture to exercise checksum semantics under the new root-dir behavior.
  • Introduce data/lets.shafa.yaml with multiple commands using checksum and persist_checksum to validate behavior
  • Ensure this fixture aligns with the new root-dir and work_dir resolution semantics for checksums and commands
data/lets.shafa.yaml

Possibly linked issues

  • #: PR makes commands run from the invocation directory, fixing the pwd path discrepancy described when using nested configs.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="internal/config/workdir.go" line_range="10-13" />
<code_context>
-	workDir, err := os.Getwd()
+// getSearchDir is where lets starts looking for the config file: the process
+// cwd, or rootDir when the user pinned one via --config-dir / LETS_CONFIG_DIR.
+func getSearchDir(filename string, rootDir string) (string, error) {
+	searchDir, err := os.Getwd()
 	if err != nil {
 		return "", fmt.Errorf("failed to get workdir for config %s: %w", filename, err)
 	}

</code_context>
<issue_to_address>
**suggestion:** Error message still refers to "workdir" while the helper is now getSearchDir, which could be confusing.

Please update the error message to refer to the "search dir" (or whatever term you standardize on) instead of "workdir" so it aligns with getSearchDir’s semantics and avoids confusion when debugging config discovery issues.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +10 to 13
func getSearchDir(filename string, rootDir string) (string, error) {
searchDir, err := os.Getwd()
if err != nil {
return "", fmt.Errorf("failed to get workdir for config %s: %w", filename, err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Error message still refers to "workdir" while the helper is now getSearchDir, which could be confusing.

Please update the error message to refer to the "search dir" (or whatever term you standardize on) instead of "workdir" so it aligns with getSearchDir’s semantics and avoids confusion when debugging config discovery issues.

@kindermax
kindermax force-pushed the fix/root-dir-semantics branch from 03e7ba6 to 92bc86a Compare August 10, 2026 11:16
Config.WorkDir meant two things at once: the directory holding the config
file, and the directory commands run in. Every path-resolving directive read
the same field, so there was no way to tell which meaning a call site wanted.

Split it. ConfigDir is the config file's directory and is read only by mixin
resolution; RootDir is where commands run and is read by everything else.
Mixin configs now carry the absolute path of the file that declares them, so a
nested mixin resolves against its own directory (removes an old TODO).

No behavior change: RootDir is still initialised to the config dir.
The root dir is now the directory lets was invoked from, whatever config it
loaded and wherever that config lives. A config describes commands; it does not
relocate them.

Everything a command reads or runs resolves against one directory - that
command's working dir, which is the root unless the command sets work_dir. That
covers cmd, checksum globs, env_file paths and env.sh. Previously these
disagreed: checksum and env_file resolved against the config dir while cmd ran
in the config dir and env.sh ran in the cwd, so the same filename in one command
definition could mean two directories.

Mixins are the exception and stay relative to the config file that declares
them - they are an include, and must resolve the same way wherever lets runs.

Also:
- .lets follows the root, so persisted checksums stay paired with the files
  they were computed from
- remote configs stop being a special case; their root was already the cwd
- a remote config declaring a local mixin now errors instead of silently
  resolving it against the cwd, since its config dir only holds the cached yaml
- LETS_CONFIG_DIR is the config's real dir for remote configs too (the cache
  dir); the project root is now just $PWD
Add a 'Where commands run' section stating the rule and the six ways of pointing
lets at a config, and rewrite the statements the previous commit invalidated:
work_dir's base, env_file resolution at both scopes, and the remote config note.

Changelog records the behaviour changes as breaking - the 0.0.63 entry described
this only as a checksum fix.
The previous commits changed several behaviours that had no test:

- checksum, env_file and env.sh following work_dir (env_file did the opposite
  before, and env.sh had no defined base at all)
- the same three resolving against the root when no work_dir is set
- mixin paths still resolving against the config dir when the root is elsewhere
- the root dir of a remote config being the cwd, not the cache dir
- a remote config declaring a local mixin being rejected

Writing the last of these found a real bug: LoadRemote assigned RemoteSource
after decoding, so the rejection never fired during mixin resolution. Set it
before decode instead.

Drops the changelog claim about nested mixins resolving against the declaring
file - recursive mixins are disallowed, so it is not observable.
The previous test only checked that ConfigDir differed from RootDir, which
would pass for any wrong-but-different value and never touched the env var a
command receives. Pin both to the cache dir, and pin LETS_CONFIG to the URL.
git does not track empty directories, so tests/root_dir/deep/nested vanished on
a fresh clone and the recursive-discovery test could not cd into it.
The rules were only reachable from a section buried in the config reference,
which is the wrong place for the first question a confused user asks. Give them
their own page in the sidebar: the short answer up front, the three directories
lets distinguishes, what resolves against what, every way of pointing lets at a
config, recipes, and the reasoning behind each choice.

The config reference keeps a short summary and links out. Record the decision
and the rejected alternatives as ADR-0004, and add Root dir and Config dir to
the domain vocabulary - conflating them in one field is what caused this.
@kindermax
kindermax force-pushed the fix/root-dir-semantics branch from bd35cbc to 5ee086d Compare August 10, 2026 15:57
@kindermax
kindermax merged commit 8cee546 into master Aug 10, 2026
7 checks passed
@kindermax
kindermax deleted the fix/root-dir-semantics branch August 10, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant