Skip to content

Conversation

@henrymercer
Copy link
Contributor

@henrymercer henrymercer commented Jan 26, 2026

  • Right now, this just skips computing baseline file coverage, but as follow up, we'll change this to also skip running the CodeQL queries to determine actual file coverage.
  • We don't plan to roll this out externally for now. codeql database interpret-results currently expects baseline information to always be present, and displays a log message "The database provided was made with a CLI version before 2.11.2 which did not record file baseline information, so file baseline information will be absent." when it is missing. We'll address this and put this feature behind a tools feature flag. At this point we'll also add a changelog entry. I'm proposing merging this now to make more incremental code changes and to allow us to test this internally.
  • There are two items of drive-by cleanup: removing an unused CLI function, and updating the version of the caniuse-lite package to avoid a pesky warning during builds.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Workflow types:

  • Advanced setup - Impacts users who have custom CodeQL workflows.
  • Managed - Impacts users with dynamic workflows (Default Setup, CCR, ...).

Products:

  • Code Scanning - The changes impact analyses when analysis-kinds: code-scanning.
  • Code Quality - The changes impact analyses when analysis-kinds: code-quality.
  • CCR - The changes impact analyses for Copilot Code Reviews.

Environments:

  • Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.

How did/will you validate this change?

  • Test repository - This change will be tested on a test repository before merging.

See https://github.com/github/codeql-action/actions/runs/21356271329

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Feature flags - All new or changed code paths can be fully disabled with corresponding feature flags.

How will you know if something goes wrong after this change is released?

  • Telemetry - I rely on existing telemetry or have made changes to the telemetry.
    • Alerts - New or existing monitors will trip if something goes wrong with this change.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

@github-actions github-actions bot added the size/S Should be easy to review label Jan 26, 2026
@henrymercer henrymercer force-pushed the henrymercer/feature-skip-file-coverage-info-prs branch from 8fc92e6 to 18c2cfc Compare January 26, 2026 11:42
@henrymercer henrymercer marked this pull request as ready for review January 26, 2026 11:42
@henrymercer henrymercer requested a review from a team as a code owner January 26, 2026 11:42
Copilot AI review requested due to automatic review settings January 26, 2026 11:42
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a feature flag that allows skipping baseline file coverage computation for PR analyses (currently gated for internal GitHub usage), wires that through the configuration, and cleans up an unused CLI method, along with a small dependency update.

Changes:

  • Add Feature.SkipFileCoverageOnPrs and thread a new enableFileCoverageInformation boolean through InitConfigInputs and Config, defaulting to true and computed in init-action based on debug mode, PR detection, repository owner, and the new feature flag.
  • Update databaseInitCluster in codeql.ts to conditionally pass baseline-related CLI flags based on config.enableFileCoverageInformation, and log an informational message in analyze.ts when file coverage is disabled for a run.
  • Remove the unused databasePrintBaseline method from the CodeQL interface and stub, adjust tests and test utilities accordingly, and bump caniuse-lite in package-lock.json.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/testing-utils.ts Extends createTestConfig to set enableFileCoverageInformation: true by default so tests construct a complete Config including the new flag.
src/init-action.ts Imports isAnalyzingPullRequest, computes debugMode once, and derives enableFileCoverageInformation from debug mode, PR vs non-PR, repository owner, and Feature.SkipFileCoverageOnPrs before passing it into initConfig.
src/feature-flags.ts Adds the SkipFileCoverageOnPrs feature enum and its featureConfig entry, backed by the CODEQL_ACTION_SKIP_FILE_COVERAGE_ON_PRS env var and currently without a minimum CLI version.
src/config-utils.ts Extends Config and InitConfigInputs with enableFileCoverageInformation and ensures it is carried through initActionState to the returned Config.
src/config-utils.test.ts Updates createTestInitConfigInputs to include enableFileCoverageInformation: true, keeping tests aligned with the extended InitConfigInputs shape.
src/codeql.ts Removes the databasePrintBaseline method from the CodeQL interface and stub, and makes databaseInitCluster conditionally include baseline-related CLI flags via a new baselineFilesOptions array and ignoringOptions for extra env options.
src/analyze.ts Guards logging of analysisSummary / qualityAnalysisSummary against empty strings and logs a clear informational message when config.enableFileCoverageInformation is false.
src/analyze.test.ts Simplifies the createStubCodeQL usage for the status report test by dropping the now-removed databasePrintBaseline stub.
package-lock.json Bumps caniuse-lite to 1.0.30001766 (including integrity, URL, and license metadata) to eliminate build warnings.
lib/upload-sarif-action.js Updates generated JS to reflect the new feature flag and baseline option wiring used by the SARIF upload action.
lib/upload-sarif-action-post.js Regenerates JS to include the SkipFileCoverageOnPrs feature config for the upload post action.
lib/upload-lib.js Regenerated library JS mirroring the new feature flag and conditional baseline CLI options.
lib/start-proxy-action.js Generated JS updated to include the new SkipFileCoverageOnPrs entry in featureConfig.
lib/start-proxy-action-post.js Generated JS updated similarly for the proxy post action’s feature configuration.
lib/setup-codeql-action.js Regenerated setup action JS to mirror the new baseline option handling and feature flag definition.
lib/resolve-environment-action.js Generated JS updated with the new feature flag entry.
lib/init-action.js Generated JS reflecting the new enableFileCoverageInformation computation and propagation into initConfig, plus conditional baseline flags.
lib/init-action-post.js Regenerated JS including the SkipFileCoverageOnPrs feature config and baseline option handling.
lib/autobuild-action.js Generated JS updated to add the SkipFileCoverageOnPrs feature to the shared featureConfig.
lib/analyze-action.js Generated JS mirroring the updated runQueries logging and use of config.enableFileCoverageInformation.
lib/analyze-action-post.js Generated JS updated with the new feature flag config and baseline option wiring for the analyze post action.

@henrymercer henrymercer self-assigned this Jan 26, 2026
Copy link
Member

@mbg mbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! A few comments, mainly about test coverage and some awkward wording in a log message.

Should we also add some telemetry here to measure what impact this has?

Comment on lines +370 to +378
// Always enable file coverage information in debug mode
debugMode ||
// We're most interested in speeding up PRs, and we want to keep
// submitting file coverage information for the default branch since
// it is used to populate the status page.
!isAnalyzingPullRequest() ||
// For now, restrict this feature to the GitHub org
repositoryNwo.owner !== "github" ||
!(await features.getValue(Feature.SkipFileCoverageOnPrs)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move this to a separate function and add some unit tests for it?

}
if (!config.enableFileCoverageInformation) {
logger.info(
"File coverage information is disabled for this PR analysis for performance reasons. " +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the wording of this could be interpreted as being specific to a given PR, rather than PRs in general. It could lead a user to ask why it is disabled for their PR specifically. Can we word this differently?

For example, if we (eventually) plan to do this for all pull requests:

Suggested change
"File coverage information is disabled for this PR analysis for performance reasons. " +
"File coverage information is disabled for pull requests. " +

if (!config.enableFileCoverageInformation) {
logger.info(
"File coverage information is disabled for this PR analysis for performance reasons. " +
"It will still be enabled for analyses triggered by a push or a schedule.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The future tense here suggests that this is something we will implement in the future, but that wouldn't happen right now.

Additionally, "by a push" is misleading, because file coverage information would not be generated if a user pushed a commit to a PR branch. You're talking about push events. Talking about the events may be confusing for dynamic workflows though, where these rules don't apply in the same way.

Suggested change
"It will still be enabled for analyses triggered by a push or a schedule.",
"It is available for analyses on the main branch.",

const baselineFilesOptions = config.enableFileCoverageInformation
? [
"--calculate-language-specific-baseline",
"--sublanguage-file-coverage",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is supported by all versions of GHES that we support now (GitHub Enterprise Server v3.12.0+ according to the CLI). A few questions about this:

  • I suppose that once we have a tools feature for enableFileCoverageInformation, that it implies the availability of --sublanguage-file-coverage so we don't need any other checks here.
  • Are there any downstream implications of this option that we need to / should account for?
  • Any tests that need updating, or should be added?

...extraArgs,
...getExtraOptionsFromEnv(["database", "init"], {
ignoringOptions: ["--overwrite"],
ignoringOptions: ["--overwrite", ...baselineFilesOptions],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a comment here for why these options in particular cannot be overridden?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Should be easy to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants