Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

censorProjectWarnings in monorepo workspace spago.yaml #1316

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 110 additions & 5 deletions CHANGELOG.md

Large diffs are not rendered by default.

22 changes: 17 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -975,13 +975,16 @@ $ node -e "import('./index.js').then(m => console.log(m.main))"
```

Spago does not wrap the entirety of the bundler's API (esbuild for JS builds), so it's possible to pass arguments through to it. E.g. to exclude an NPM package from the bundle you can pass the `--external` flag to esbuild:

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My editor auto-formatted this markdown on save - can revert if preferred.

- either through the command line, with the `--bundler-args` flag, i.e. `--bundler-args "--external:better-sqlite3"`.
- or by adding it to the configuration file:
```yaml
package:
bundle:
extra_args:
- "--external:better-sqlite3"
```

### Enable source maps

When bundling, you can include `--source-maps` to generate a final source map for your bundle.
Expand Down Expand Up @@ -1083,6 +1086,7 @@ Library authors will often build "ecosystems" of small interdependent packages t
If you're using the registry solver then this is not an issue, but if your project is based on a package set, then that will not contain your newly published package, since well, you just published it!

You should be able to add the newly released version to your build plan by adding it to the `extraPackages` section ([see here](https://github.com/purescript/spago/issues/1215)):

```yaml
package:
name: next-library-to-publish
Expand All @@ -1095,6 +1099,7 @@ workspace:
extraPackages:
newly-published-library: 0.1.0
```

> [!NOTE]\
> This only works when the package you add to `extraPackages` has been published to the registry. Adding a git dependency will produce an error, as publishing to the Registry only admits build plans that only contain packages coming from the Registry.

Expand All @@ -1117,9 +1122,10 @@ If you are the owner of a package and you want to transfer it to another user, y
about the new location of the repository, so that the new owner will be able to publish new versions of the package.

The transfer procedure is automated by Spago commands, and goes as follows:
* Add your (or the new owner's) SSH public key to the `spago.yaml` through `spago auth` if they are not there already (see previous section)
* Transfer the repository to the new owner using the hosting platform's transfer mechanism (e.g. GitHub's transfer feature)
* Depending on whose key is present in the `owners` field, either you or the new owner will update the `publish.location` field in the `spago.yaml`, and call `spago registry transfer` to initiate the transfer. If all goes well you'll now be able to publish a new version from the new location.

- Add your (or the new owner's) SSH public key to the `spago.yaml` through `spago auth` if they are not there already (see previous section)
- Transfer the repository to the new owner using the hosting platform's transfer mechanism (e.g. GitHub's transfer feature)
- Depending on whose key is present in the `owners` field, either you or the new owner will update the `publish.location` field in the `spago.yaml`, and call `spago registry transfer` to initiate the transfer. If all goes well you'll now be able to publish a new version from the new location.

### Know which `purs` commands are run under the hood

Expand Down Expand Up @@ -1373,7 +1379,7 @@ workspace:
# Value 1: "all" - All warnings are censored
all

# Value 2: `NonEmptyArray (Either String { byPrefix :: String })`
# Value 2: `Array (Either String { byPrefix :: String })`
# - String values:
# censor warnings if the code matches this code
# - { byPrefix } values:
Expand All @@ -1385,6 +1391,11 @@ workspace:
- byPrefix: >
"Data.Map"'s `Semigroup instance`

# Specify whether to censor warnings coming from the compiler
# for files in workspace projects and tests
# Optional - takes the same values as censorLibraryWarnings above
censorProjectWarnings: all

# Specify whether to show statistics at the end of the compilation,
# and how verbose they should be.
# Can be 'no-stats', 'compact-stats' (default), or 'verbose-stats',
Expand Down Expand Up @@ -1563,18 +1574,19 @@ You can enable it manually by adding a `lock: true` field to the `workspace` sec
### File System Paths used in Spago

Run `spago ls paths` to see all paths used by Spago. But in general, Spago utilizes two main directories for every project:

- the local cache directory
- the global cache directory

The local cache directory is located at `<project-directory>/.spago` and its location cannot be changed.

The global cache directory's location depends on your OS. Its location can be changed by configuring the corresponding environment variable, if it is used:

- Mac: `~/Library/Caches/spago-nodejs`. The location cannot be changed.
- Linux: `${XDG_CACHE_HOME}/spago-nodejs`, or if `XDG_CACHE_HOME` is not set, `~/.cache/spago-nodejs`. See [`XDG_CACHE_HOME`](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
- Windows: `%LOCALAPPDATA%\spago-nodejs\Cache`, or if `$LOCALAPPDATA%` is not set, `C:\Users\USERNAME\AppData\Local\spago-nodejs\Cache`.
- NixOS: `${XDG_RUNTIME_DIR}/spago-nodejs`. See [`XDG_RUNTIME_DIR`](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).


## FAQ

### Why can't `spago` also install my npm dependencies?
Expand Down
9 changes: 5 additions & 4 deletions core/src/Config.purs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ module Spago.Core.Config
import Spago.Core.Prelude

import Codec.JSON.DecodeError as CJ.DecodeError
import Data.Array.NonEmpty as NonEmptyArray
import Data.Codec as Codec
import Data.Codec.JSON as CJ
import Data.Codec.JSON.Record as CJ.Record
Expand Down Expand Up @@ -349,19 +348,21 @@ workspaceConfigCodec = CJ.named "WorkspaceConfig" $ CJS.objectStrict
type WorkspaceBuildOptionsInput =
{ output :: Maybe RawFilePath
, censorLibraryWarnings :: Maybe CensorBuildWarnings
, censorProjectWarnings :: Maybe CensorBuildWarnings
, statVerbosity :: Maybe StatVerbosity
}

buildOptionsCodec :: CJ.Codec WorkspaceBuildOptionsInput
buildOptionsCodec = CJ.named "WorkspaceBuildOptionsInput" $ CJS.objectStrict
$ CJS.recordPropOptional @"output" CJ.string
$ CJS.recordPropOptional @"censorLibraryWarnings" censorBuildWarningsCodec
$ CJS.recordPropOptional @"censorProjectWarnings" censorBuildWarningsCodec
$ CJS.recordPropOptional @"statVerbosity" statVerbosityCodec
$ CJS.record

data CensorBuildWarnings
= CensorAllWarnings
| CensorSpecificWarnings (NonEmptyArray WarningCensorTest)
| CensorSpecificWarnings (Array WarningCensorTest)

derive instance Eq CensorBuildWarnings

Expand All @@ -375,7 +376,7 @@ censorBuildWarningsCodec = Codec.codec' decode encode
where
encode = case _ of
CensorAllWarnings -> CJ.encode CJ.string "all"
CensorSpecificWarnings censorTests -> CJ.encode (CJ.array warningCensorTestCodec) $ NonEmptyArray.toArray censorTests
CensorSpecificWarnings censorTests -> CJ.encode (CJ.array warningCensorTestCodec) censorTests

decode json = decodeNoneOrAll <|> decodeSpecific
where
Expand All @@ -385,7 +386,7 @@ censorBuildWarningsCodec = Codec.codec' decode encode

decodeSpecific = CensorSpecificWarnings <$> do
arr <- Codec.decode (CJ.array warningCensorTestCodec) json
except $ Either.note (CJ.DecodeError.basic "Expected array of warning codes") $ NonEmptyArray.fromArray arr
except $ Right arr

data WarningCensorTest
= ByCode String
Expand Down
1 change: 1 addition & 0 deletions src/Spago/Command/Build.purs
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ run opts = do
, selectedPackages: NEA.toArray selectedPackages
, psaCliFlags: { strict: strictWarnings, statVerbosity: workspace.buildOptions.statVerbosity }
, censorLibWarnings: workspace.buildOptions.censorLibWarnings
, censorProjectWarnings: workspace.buildOptions.censorProjectWarnings
}
let
psaArgs =
Expand Down
2 changes: 2 additions & 0 deletions src/Spago/Config.purs
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ type Workspace =
type WorkspaceBuildOptions =
{ output :: Maybe LocalPath
, censorLibWarnings :: Maybe Core.CensorBuildWarnings
, censorProjectWarnings :: Maybe Core.CensorBuildWarnings
, statVerbosity :: Maybe Core.StatVerbosity
}

Expand Down Expand Up @@ -454,6 +455,7 @@ readWorkspace { maybeSelectedPackage, pureBuild, migrateConfig } = do
buildOptions =
{ output: workspace.buildOpts >>= _.output <#> \o -> withForwardSlashes $ rootPath </> o
, censorLibWarnings: _.censorLibraryWarnings =<< workspace.buildOpts
, censorProjectWarnings: _.censorProjectWarnings =<< workspace.buildOpts
, statVerbosity: _.statVerbosity =<< workspace.buildOpts
}

Expand Down
16 changes: 9 additions & 7 deletions src/Spago/Psa.purs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import Control.Alternative as Alternative
import Control.Monad.Except.Trans (ExceptT(..), runExceptT)
import Control.Monad.Trans.Class (lift)
import Data.Array as Array
import Data.Array.NonEmpty as NonEmptyArray
import Data.Codec.JSON as CJ
import Data.Either (blush)
import Data.Map as Map
Expand Down Expand Up @@ -106,12 +105,13 @@ toPathDecisions
, selectedPackages :: Array WorkspacePackage
, psaCliFlags :: PsaOutputOptions
, censorLibWarnings :: Maybe Core.CensorBuildWarnings
, censorProjectWarnings :: Maybe Core.CensorBuildWarnings
}
-> Array (Effect (Array (LocalPath -> Maybe PathDecision)))
toPathDecisions { rootPath, allDependencies, selectedPackages, psaCliFlags, censorLibWarnings } =
toPathDecisions { rootPath, allDependencies, selectedPackages, psaCliFlags, censorLibWarnings, censorProjectWarnings } =
projectDecisions <> dependencyDecisions
where
projectDecisions = selectedPackages <#> \selected -> toWorkspacePackagePathDecision { selected, psaCliFlags }
projectDecisions = selectedPackages <#> \selected -> toWorkspacePackagePathDecision { selected, psaCliFlags, censorProjectWarnings }

dependencyDecisions =
map toDependencyDecision
Expand All @@ -129,6 +129,7 @@ toPathDecisions { rootPath, allDependencies, selectedPackages, psaCliFlags, cens
toWorkspacePackagePathDecision
{ selected: p
, psaCliFlags
, censorProjectWarnings
}
_ -> do
let pkgLocation = Tuple.uncurry (Config.getLocalPackageLocation rootPath) dep
Expand All @@ -144,23 +145,24 @@ toPathDecisions { rootPath, allDependencies, selectedPackages, psaCliFlags, cens
toWorkspacePackagePathDecision
:: { selected :: WorkspacePackage
, psaCliFlags :: PsaOutputOptions
, censorProjectWarnings :: Maybe Core.CensorBuildWarnings
}
-> Effect (Array (LocalPath -> Maybe PathDecision))
toWorkspacePackagePathDecision { selected: { path, package }, psaCliFlags } = do
toWorkspacePackagePathDecision { selected: { path, package }, psaCliFlags, censorProjectWarnings } = do
let srcPath = path </> "src"
let testPath = path </> "test"
pure
[ toPathDecision
{ pathIsFromPackage: (srcPath `Path.isPrefixOf` _)
, pathType: IsSrc
, strict: fromMaybe false $ psaCliFlags.strict <|> (package.build >>= _.strict)
, censorWarnings: package.build >>= _.censorProjectWarnings
, censorWarnings: (package.build >>= _.censorProjectWarnings) <|> censorProjectWarnings
}
, toPathDecision
{ pathIsFromPackage: (testPath `Path.isPrefixOf` _)
, pathType: IsSrc
, strict: fromMaybe false $ psaCliFlags.strict <|> (package.test >>= _.strict)
, censorWarnings: package.test >>= _.censorTestWarnings
, censorWarnings: (package.test >>= _.censorTestWarnings) <|> censorProjectWarnings
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to discuss I looked into pulling this out into a separate param like censorTestWarnings, but maybe it doesn't make sense having it under buildOpts as this is akin to build: in the package spago.yaml, maybe we'd need testOpts as censorTestWarnings is under test:.

If you get me

}
]

Expand Down Expand Up @@ -193,4 +195,4 @@ shouldPrintWarning = case _ of
-- We return `true` to print the warning.
-- If an element was found (i.e. `Just` is returned), then one of the tests succeeded,
-- so we should not print the warning and return false here.
\code msg -> isNothing $ NonEmptyArray.find (\f -> f code msg) tests
\code msg -> isNothing $ Array.find (\f -> f code msg) tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package:
name: package-a
dependencies:
- prelude
build:
strict: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
module Src.PACKAGE.A where

import Prelude

packageName :: String -> String
packageName foo =
"package" <> "package-a"
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package:
name: package-b
dependencies:
- package-a
- prelude
build:
strict: true
censor_project_warnings: [] # override workspace
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
module Src.PACKAGE.B where

import Prelude

packageName :: _
packageName foo =
"package" <> "package-b"
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package:
name: package-c
dependencies:
- package-a
- package-b
- prelude
build:
strict: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
module Src.PACKAGE.C where

import Prelude

packageName :: _
packageName foo =
"package" <> "package-c"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
workspace:
buildOpts:
censor_project_warnings: all
package_set:
registry: 0.0.1
extra_packages: {}
10 changes: 10 additions & 0 deletions test/Spago/Build/Monorepo.purs
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,16 @@ spec = Spec.describe "monorepo" do
shouldNotHaveWarning = assertWarning paths false
spago [ "build" ] >>= check { stdout: mempty, stderr: shouldNotHaveWarning, result: isRight }

Spec.it "build fails when 'strict: true' because warnings censored at the workspace level were overridden" \{ spago, fixture, testCwd } -> do
FS.copyTree { src: fixture "monorepo/strict-true-workspace-censored-warnings", dst: Path.toGlobal testCwd }
let
errs =
[ "[ERROR 1/2 WildcardInferredType] " <> escapePathInErrMsg [ "package-b", "src", "Main.purs:5:16" ]
, "[ERROR 2/2 UnusedName] " <> escapePathInErrMsg [ "package-b", "src", "Main.purs:6:13" ]
]
hasUnusedWarningError = assertWarning errs true
spago [ "build" ] >>= check { stdout: mempty, stderr: hasUnusedWarningError, result: isLeft }

Spec.it "build fails when 'strict: true' and warnings were not censored" \{ spago, fixture, testCwd } -> do
FS.copyTree { src: fixture "monorepo/strict-true-uncensored-warnings", dst: Path.toGlobal testCwd }
let
Expand Down