feat(manifest): socket manifest dotnet + centralized config-name globs (REA-627, REA-628) - #1404
Conversation
…s (REA-627, REA-628) Bring .NET/NuGet to the Socket facts pipeline, at parity with the JVM tools. socket manifest dotnet (REA-627): - New command + auto-manifest detection (top-level .sln/.slnx/.csproj/.fsproj/ .vbproj) and setup-wizard entry; socket.json defaults.manifest.dotnet.*. - Bundled C# tool (socket-facts-dotnet): one MSBuild session — evaluate, in-process restore, read project.assets.json via NuGet.ProjectModel — emitting the shared records TSV. Ships no NuGet/MSBuild runtime assemblies (compile-low/run-high against the locator-selected SDK; net6 floor, RollForward LatestMajor), so it works across installed SDK versions. Restore forces TreatWarningsAsErrors=false so NU19xx security advisories don't abort the run. packages.config supported (flat closure + HintPath DLLs, with pinned-artifact download via the project's configured feeds). Fail-closed: emits failure records that raise the CLI exit code; every emitted artifact path is guaranteed to exist. - --target-frameworks / --exclude-target-frameworks (TFM globs; RID targets match under their base TFM); --dotnet-opts (MSBuild -p: props applied to the whole session). Reachability sidecar emits nuget-tagged ResolvedComponents. - Resolution summary reports scanned target frameworks across N projects, with per-project attribution under --verbose. Centralized config-name glob compilation (REA-628): - Compile --include-configs/--exclude-configs globs to anchored regex sources once in config-glob.mts; the gradle/sbt/maven scripts and the dotnet tool consume the compiled patterns. Removes the per-language globToRegex (Groovy/Scala/Java) and adds a cross-language vector test suite. Gated on coana REA-626 (nuget sidecar consumer): the coana version pin is bumped and this is un-drafted once coana releases.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
|
Caution Review the following alerts detected in dependencies. According to your organization's Security Policy, you must resolve all "Block" alerts before proceeding. It is recommended to resolve "Warn" alerts too. Learn more about Socket for GitHub.
|
|
Jeppe Fredsgaard Blaabjerg (@jfblaa) Would this live better in the coana cli? |
|
Six verified issues, most severe first. Each one is traced to the exact lines on this branch and checked against the v1.x base (and the pinned coana builds where that mattered). Two things that looked like bugs turned out fine — they're at the bottom so nobody re-chases them. 🔴 1. Windows line endings corrupt the last field of every record the C# tool emits
🟠 2. The sidecar schema break is wider than the draft warning says — every reachability scan fails, not just polyglot ones
🟠 3. A tool crash after partial output still exits 0 — a truncated SBOM is reported as success
🟠 4. A failed restore can report success with stale data
🟠 5. A leftover packages.config silently wins over the real PackageReference graph
🟠 6. Polyglot repos overwrite one ecosystem's facts with another's
🟡 Smaller items
🟢 Verified fine: two things that looked like bugs but aren't
Items 1 (Windows line endings) and 3–5 (the fail-closed trio) are the ones to settle before un-drafting — item 1 (Windows line endings) because it's a one-line fix for a whole platform, and items 3 (crash exits 0), 4 (stale-restore success) and 5 (packages.config precedence) because they all break the same promise the tool's design leans on: a wrong scan never exits green. Item 2 (sidecar scope) just needs the PR body corrected so branch testers know what to expect. |
The six JVM producers (gradle/maven/sbt × facts/workspaces) each carried their own NIO PathMatcher-based exclude-glob implementation, including a "zero-depth variant expansion" workaround for a semantics mismatch between NIO's `**` and the CLI's micromatch. Following the same pattern already used for --include-configs/--exclude-configs (PR #1404), the glob is now compiled to a portable regex pattern source once in exclude-paths-glob.mts, transported via the existing -D/-P property mechanism, and each producer just Pattern.compile()s what it receives. Gradle/sbt/maven all share java.util.regex, so one dialect covers every producer. Verified end-to-end against real gradle, maven, and sbt invocations (facts and workspaces variants) with an actual --exclude-paths value.
* Add socket manifest dynamic-sbom-inference
Recursively discovers independent gradle/sbt/maven build roots and
generates a Socket facts SBOM for each, without re-invoking the manifest
script on subproject/reactor-module directories a parent build root
already covers. Maven reactors in particular have a pom.xml per module,
so naive marker-file discovery would over-invoke; coverage is tracked
per ecosystem using the facts SBOM's own projects[].subprojectDir, which
every build-tool producer already reports.
This is a standalone command rather than an --auto-manifest extension,
so it can force stricter behavior later without touching auto's
existing contract. Scoped narrowly for this first PR: discovery plus
generation only, one global socket.json config applied to every
discovered root (no per-build-root cascade), no new CLI flags beyond
--exclude-paths/--verbose.
Refs REA-685, REA-553
* Fix misleading --exclude-paths description on manifest commands
--exclude-paths on socket manifest auto/gradle/kotlin/maven/scala/
dynamic-sbom-inference reused the flag description from the scan/reach
flag definitions verbatim, which talks about "the scan" and full
application reachability analysis. None of these commands scan or run
reachability analysis themselves, so the wording was confusing when
viewed via --help on any of them standalone.
Adds a manifest-scoped excludePathsFlag with wording specific to
manifest/facts generation and switches all six commands to it.
* Add per-ecosystem javaHome config for gradle/maven/sbt manifest generation
Different projects in a repo may need different JDKs for their build
tool. Adds a javaHome field to defaults.manifest.{gradle,maven,sbt} in
socket.json, configurable via the socket manifest setup wizard, and
threads it through to the actual build-tool invocation (overrides
JAVA_HOME for that spawn only, everything else about the environment is
left untouched).
Applies to socket manifest gradle/kotlin/maven/scala, socket manifest
auto, and socket manifest dynamic-sbom-inference. Scoped to the Socket
facts generation path only, not the legacy --pom conversion path. No
new CLI flag - socket.json/the setup wizard is the only configuration
surface for now.
* Resolve socket.json per build root, and support env var refs in javaHome
socket manifest dynamic-sbom-inference previously read socket.json once
at the overall recursion root and applied that single config to every
discovered build root. It now resolves each build root's own nearest
socket.json (walking up from that root, bounded at the recursion root,
nearest wins - no merging), so different projects in the same repo can
carry their own settings instead of being forced onto one shared config.
Also adds $VAR/${VAR} expansion for javaHome, resolved against the CLI
process's own environment. A hardcoded absolute JDK path only works on
whoever's machine wrote it; referencing an env var each developer sets
themselves (e.g. $JAVA11_HOME) makes a shared socket.json portable
across machines. Fails closed with a clear message if the referenced
variable isn't set, rather than silently passing a broken path to the
build tool.
* Deep-merge the socket.json cascade instead of nearest-wins
readSocketJsonCascade (renamed from readOrDefaultSocketJsonUpTo) now
merges defaults.manifest.<ecosystem> field-by-field across every
ancestor between a build root and the recursion root, nearest winning
per field, instead of one file replacing the root config wholesale. A
subproject can now override just javaHome while still inheriting the
root's excludeConfigs/bin/etc., rather than having to restate the whole
config. Ecosystems the override doesn't mention are left untouched.
Verified against the sandbox tree: a root socket.json setting
excludeConfigs plus a nested one setting only javaHome both applied
together for that build root.
* Warn (but proceed) when a resolved config sets facts: false
dynamic-sbom-inference always generates Socket facts SBOMs, never
pom.xml. If a build root's cascaded socket.json sets facts: false
(the pom-mode opt-out other manifest commands honor), that's ignored
here rather than skipping the project or silently doing nothing -
but it's a real, deliberate setting the user made, so it's surfaced
as a warning rather than silently overridden.
* Revert facts:false to skip, add a dedicated ignored config
dynamic-sbom-inference previously forced Socket facts generation over
an explicit defaults.manifest.<ecosystem>.facts: false (pom mode),
just warning about it. Reverted: facts: false now skips the project
again, same as before that change, since this command has no pom-mode
equivalent to fall back to.
Also adds a separate ignored: true field (gradle/maven/sbt) scoped
specifically to dynamic-sbom-inference, for projects that should be
skipped during recursive generation regardless of their facts/pom
preference for other commands. Kept as its own boolean rather than
folding into facts, since facts already means something specific to
other manifest commands and overloading it would require touching
every consumer for a value only this command understands.
New skippedIgnored outcome status covers both reasons, distinguished
via the warning message.
* Consolidate ignored into disabled instead of a separate field
A cascaded (not just root-level) disabled: true produces identical
per-root behavior to the ignored field just added, so keeping both
was redundant. Removed ignored; disabled now does double duty:
root-only ecosystem-wide gating for auto/gradle/etc. (unchanged), plus
a cascaded per-build-root skip specifically for dynamic-sbom-inference.
Renamed the skippedIgnored outcome status to skippedDisabled to match.
* Add explicit clear sentinel, lightweight workspace discovery, and recursive setup for dynamic-sbom-inference
- socket.json manifest fields now accept `null` as an explicit "clear the
inherited value" sentinel, distinct from leaving a field unset; the setup
wizard writes it when a previously-set value is cleared instead of just
deleting the key.
- Add a lightweight per-ecosystem workspace enumeration path for
gradle/sbt/maven (new standalone scripts, kept fully separate from the
existing facts-generation scripts) that discovers a build's subprojects
without running dependency resolution.
- Add a hidden `socket manifest setup --dynamic-sbom-inference` mode:
configures root-level defaults per ecosystem, then recursively marks
`disabled: true` on build roots matching `--exclude-paths`, leaving
everything else untouched.
- Speed up `dynamic-sbom-inference`'s handling of a disabled build root with
many nested candidates by reusing the nearest already-resolved disabled
ancestor instead of re-walking the whole config cascade for each one, and
only logging the root cause instead of once per nested candidate.
* Write --exclude-paths disable at the exclusion root, not per project
Previously each excluded build root got its own disabled:true write,
relying on cascade to skip descendants already covered by an ancestor
write. That only worked when the excluded ancestor happened to be a
build root itself; a non-project directory containing multiple sibling
projects would leave later siblings enabled.
Instead, group excluded projects by the shallowest directory that
actually matches --exclude-paths and write disabled:true there once,
covering every ecosystem and sibling/nested project beneath it
regardless of whether that directory is a build root of its own.
* Abort recursive facts generation when a build root's workspace layout is unknown
A build root whose facts generation fails (a build-tool crash or a
blocking resolution failure) never produces its projects[] list, so
there's no way to tell whether a later candidate underneath it is
already covered by that root or a genuinely independent project.
Continuing to process further candidates in that state risked
misclassifying subprojects and piling on doomed attempts against a
build already known to be broken.
Fail closed instead: abort the entire recursive walk as soon as one
build root's workspace layout can't be determined, rather than
continuing to sibling and nested candidates.
* Make recursive setup actually configure discovered build roots
The recursive wizard (`socket manifest setup --dynamic-sbom-inference`)
previously only ever disabled build roots matching --exclude-paths;
every other discovered root was left completely untouched, with no
way to set its bin/JDK/opts short of running the plain single-project
wizard on it directly. Every non-excluded candidate now gets an
interactive configure-or-inherit-defaults prompt (in discovery order,
parent before child), seeded with its cascaded effective value so
accepting every prompt unchanged preserves whatever it already
inherits. Disabling a specific candidate is intentionally not offered
here - that stays --exclude-paths' job, so a whole excluded subtree
still collapses into a single write.
Also fixes a real bug surfaced along the way: askForBin pre-filled the
hardcoded tool fallback (mvn/./gradlew/sbt) as the prompt's shown
value, so accepting the default was indistinguishable from explicitly
typing it and got written to socket.json for no reason. The shown
default is now only ever a prior explicit value; the fallback is
mentioned as a hint instead. A matching guard drops any ecosystem
section that ends up empty so it doesn't count as configured or
trigger a write.
Finally, the root step now detects which ecosystems are actually
present at cwd (reusing the same check the plain wizard uses) and
asks about detected ecosystems first, phrased accordingly, before
offering to configure undetected ones "anyway" for subprojects that
might need them.
* Fix null-clear and exclude-paths bugs in the recursive setup wizard
Fixes two correctness bugs found in an audit pass: leaving a config prompt
blank when the field was already explicitly cleared (null) deleted the key
instead of preserving the clear, silently reverting it to inheriting an
ancestor's value; and --exclude-paths never reached the wizard's workspace
enumeration, so excluding a broken reactor member didn't stop the wizard
from still trying to resolve it and aborting the whole walk.
Also tightens several UX rough edges in the recursive wizard: drops a
redundant write confirmation inconsistent with the rest of the flow, fixes
a tally line that double-counted re-enabled candidates and never reported
disabled ones, and softens wording that overclaimed knowledge the wizard
doesn't actually have yet (a build root at cwd, a fixed --exclude-paths
prompt count). Trims the surrounding comments down to non-obvious rationale
only, per repo comment-style guidelines.
* Trim verbose comments across the dynamic-sbom-inference feature
Cuts several multi-line comment blocks down to their non-obvious why,
matching the repo's comment-style guidelines.
* Address code review findings on the dynamic-sbom-inference PR
- A root-disabled ecosystem was dropped from the build-tool scan entirely,
so a nested socket.json could never re-enable it. Moved the strip-before-
scan helper (previously wizard-only) to discover-manifest-roots.mts and
applied it in generateRecursiveManifests too, letting the existing
per-directory cascade check be the sole arbiter of skip vs. include.
- runManifestFacts's progress line used logger.log (stdout), polluting
--json output with non-JSON lines ahead of the payload. Switched to
logger.info (stderr), matching every other status line in that function.
- `socket manifest setup --dynamic-sbom-inference` forwarded --exclude-paths
without validation, unlike every sibling manifest command. Added the same
assertValidExcludePaths call.
- Recursive generation inferred a build-root failure from whether
process.exitCode changed during a call, which misclassifies a real
failure as empty if the exit code was already non-zero beforehand. Gave
runManifestFacts an explicit null (failure) vs. undefined (empty)
return so callers don't have to infer it from global state.
- Verbose error logging in enumerate-workspaces/run-manifest-facts
string-coerced the caught error directly; switched to the existing
getErrorMessageOr helper.
* Refresh CLI banner snapshots after merging v1.x's version bump
The merge brought package.json's version from 1.1.150-prerelease to a
clean release version, so these inline snapshots no longer matched.
* Use 1.1.153-prerelease pending the stuck 1.1.152 release
1.1.152 never actually published (still 404s on the registry, staged
dist-tag is still 1.1.151), so bump to the next-version-hint convention
used elsewhere in this repo's release history and refresh the CLI banner
snapshots to match.
* Simplify dynamic-sbom-inference's summary output
Drop reactor-member "skippedCovered" lines from the per-line table -
they're implied by their parent's line already showing up, and the
aggregate count in the summary still reports them. Also drop the
"across N build root(s)" total from the summary line: it counted every
candidate directory visited, including reactor members that aren't
independent build roots, which overstated how many actually exist.
* Reduce the summary line to just the generated count
Drop failed/skipped/empty from the tally: a failure aborts the whole
walk immediately rather than accumulating (and is already reported via
its own fail message), and the disabled/covered/empty buckets count
candidate directories rather than independent build roots, so a total
there is just as misleading as the "N build root(s)" figure already
removed.
* Trim verbose comment in output-manifest-dynamic-sbom-inference
* Make the CLI-banner-version test normalization suffix-aware
Root cause of the flaky snapshot mismatches on this PR: whether the CLI
itself redacts its version banner (VITEST baked in at build time) or
prints the real one and leaves redaction to this test helper depends on
env propagation into the build step, not just the test run - so the same
source can produce either "<redacted>" or the real "vX.Y.Z-prerelease"
depending on how it was built. normalizeBanner's regex only stripped a
bare "vX.Y.Z", leaving a trailing prerelease suffix dangling in one case
but not the other. Broadened it to match a trailing prerelease/build
suffix and to be idempotent on an already-redacted value, so both cases
normalize identically. Refreshed the now-correct snapshots.
* Fix realpath/boundary mismatch and a silent-zero-projects gap
generateRecursiveManifests read candidate dirs realpath-resolved
(findBuildToolCandidates already does this) but passed the raw cwd as
the socket.json cascade boundary and as the anchor for re-anchoring
--exclude-paths. Whenever cwd contains a symlink (macOS /tmp ->
/private/tmp, several CI layouts), the boundary comparison never
matched: the cascade walked all the way to the filesystem root instead
of stopping at cwd, and --exclude-paths silently stopped reaching the
build tool invocation. Resolve cwd once and use it consistently for
both. Also realpath-resolve a project's subprojectDir before adding it
to the covered set, in both generate-recursive-manifests.mts and
setup-recursive-manifest-config.mts, so a symlinked reactor member is
correctly recognized as covered instead of escaping and being
reinvoked as an independent root.
enumerateWorkspaces treated a clean exit with zero projects as success,
but every real enumeration reports at least the build's own root
project, so zero projects always means the task never ran (e.g. an
extension jar built before the workspace-enumeration participant
existed). Drop the exit-code condition so this is always a failure.
* Address remaining non-blocking review findings
- Hide `socket manifest dynamic-sbom-inference` - its name collides with
the unrelated, root-only --dynamic-sbom-inference flag on scan
create/reach (different semantics: this one is recursive per-root).
Keep it internal until that naming collision is resolved.
- --json --verbose emitted invalid JSON: the verbose debug preamble and
run-manifest-facts' verbose resolution-detail logging both wrote to
stdout ahead of the JSON payload. This is the only manifest command
with --json, so the combination was newly reachable. Gated the
preamble on !json and switched the detail logging to stderr.
- Fixed two stale "socket manifest setup --recursive" comments left over
from an earlier flag name.
- renderTable no longer prints a bare blank line when there are no
non-covered outcomes to show.
* Make configureCandidate write a minimal diff instead of snapshotting the cascade
Every field shown to the sub-wizard (including ones inherited from an
ancestor, never touched by the user) was written verbatim into the
candidate's own file, permanently pinning that value against future
changes to the ancestor - converting inheritance into a one-time
snapshot despite the field-level cascade being the point. Now diffs the
final seed against an ancestor-only baseline (the cascade computed from
dir's parent, excluding dir's own file) and only writes fields that
actually differ, so an untouched field keeps inheriting live.
* Add compat-matrix coverage for the workspace-enumeration producers
socket-workspaces.init.gradle, socket-workspaces.plugin.scala, and
CoanaWorkspacesLifecycleParticipant/SocketWorkspacesRecordsEngine had no
automated coverage at all - the local compat matrix only exercised the
facts scripts, so a registration or compile error in the workspaces
siblings would surface only as silent degradation (the setup wizard's
reactor-coverage pruning treating a real reactor as if it had no
members). Adds a smoke-test-workspaces.sh per ecosystem, wired into the
same per-version matrix run-compat.sh already runs for the facts
scripts, asserting each variant emits exactly a meta record plus the
expected project record(s) and nothing else (no node/root/file records,
confirming no dependency resolution happens). Verified locally against
the currently installed gradle/maven/sbt.
* Scope the fail-closed abort to the failing ecosystem, not the whole walk
Coverage (`covered`/`disabledRoots`) is tracked per ecosystem, so a
gradle failure carries no information about maven or sbt's own
classification - yet aborting all remaining ecosystems meant an
unrelated one could be blocked from ever starting, with nothing in the
output explaining why. Narrowed the abort to the failing ecosystem's
own loop, and added an 'aborted' status for that ecosystem's
still-untried candidates so they show up explicitly instead of being
silently absent from the output.
* expandEnvVarRefs: report every missing var and add a $$ escape
Only the first missing variable was ever reported, so a value
referencing two unset vars needed two runs to discover both. Also had
no way to represent a literal $WORD - $$ now expands to a literal $, so
whatever follows it is left untouched. Added a dedicated test file;
this function had none.
* Centralize --exclude-paths glob compilation into one TypeScript module
The six JVM producers (gradle/maven/sbt × facts/workspaces) each carried
their own NIO PathMatcher-based exclude-glob implementation, including a
"zero-depth variant expansion" workaround for a semantics mismatch between
NIO's `**` and the CLI's micromatch. Following the same pattern already
used for --include-configs/--exclude-configs (PR #1404), the glob is now
compiled to a portable regex pattern source once in
exclude-paths-glob.mts, transported via the existing -D/-P property
mechanism, and each producer just Pattern.compile()s what it receives.
Gradle/sbt/maven all share java.util.regex, so one dialect covers every
producer.
Verified end-to-end against real gradle, maven, and sbt invocations
(facts and workspaces variants) with an actual --exclude-paths value.
2 participants
Closes REA-627. Closes REA-628.
Warning
Draft — do not merge until coana PR (REA-626) is released and the pinned coana version is bumped in this PR. The reachability sidecar emits nuget-tagged entries that the currently-released coana rejects (strict schema); a polyglot .NET+JVM
--reachscan would break until coana ships REA-626. The SBOM-only path (no--reach) is coana-independent.What
Brings .NET/NuGet to the Socket facts pipeline, at parity with the JVM tools (the .NET analog of REA-613). Previously
nugetwas only a package-scoring ecosystem — no manifest generation, no auto-detection, no reachability; the only path was a manualsocket manifest cdxgen -t dotnet.socket manifest dotnet(REA-627).sln/.slnx/.csproj/.fsproj/.vbproj) + setup-wizard entry;socket.jsondefaults.manifest.dotnet.*.scripts/dotnet-tool/): one MSBuild session — evaluate → in-process restore → readproject.assets.jsonviaNuGet.ProjectModel— emitting the shared records TSV.RollForward LatestMajor), so it works across installed SDKs and avoids the ref/def assembly clash that breaks multi-SDK hosts.TreatWarningsAsErrors=falsesoNU19xxsecurity-advisory warnings don't abort the run.developmentDependency→dev + HintPath DLLs; downloads missing pinned artifacts via the project's configured feeds/credential providers).failure/unscannablerecords that raise the CLI exit code; every emitted artifact path is guaranteed to exist.--target-frameworks/--exclude-target-frameworks(TFM globs; RID targets match under their base TFM);--dotnet-opts(MSBuild-p:props applied to the whole session).ResolvedComponents (consumed by coana REA-626).--verbose).Centralized config-name glob compilation (REA-628)
--include-configs/--exclude-configsglobs → anchored regex sources once inconfig-glob.mts; the gradle/sbt/maven scripts and the dotnet tool all consume the compiled patterns. Removes the per-languageglobToRegex(Groovy/Scala/Java) and adds a cross-language vector test suite. Rides in this PR because the .NET tool is a fourth consumer of the shared patterns.Testing
Unit tests (config-glob vectors, dotnet records→assemble, detection, auto-manifest branch, command snapshots); accuracy sweep vs. native-restore ground truth across 17 real .NET repos (~99.6% avg recall, exact real-package versions, packages.config 100%).
pnpm checkclean.Follow-ups before un-drafting