Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pixi global reinstall flag #2274

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
195d2d4
feat: pixi global manifest (#1802)
Hofer-Julian Aug 19, 2024
c3bb1da
Merge branch 'main' into feature/pixi-global
Hofer-Julian Aug 19, 2024
a4ddbd2
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 2, 2024
05946ff
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 2, 2024
c021fbb
feat: add `global sync` (#1835)
Hofer-Julian Sep 3, 2024
df961cf
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 3, 2024
98a74c3
fix: integration tests for pixi global (#1972)
Hofer-Julian Sep 3, 2024
a31b9b6
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 3, 2024
9f1a027
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 5, 2024
4ebec3c
Merge branch 'main' into feature/pixi-global
nichmor Sep 10, 2024
9ccadc1
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 17, 2024
a006941
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 17, 2024
95779df
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 19, 2024
e04e20e
feat: add migration path for new pixi global (#1975)
Hofer-Julian Sep 20, 2024
ebda8b1
fix: Remove `allow(unused` and fix resulting warnings (#2091)
Hofer-Julian Sep 20, 2024
6358be9
test: for `local_environment_matches_spec` (#2093)
Hofer-Julian Sep 20, 2024
e943408
feat: Give better info message for `pixi global sync` (#2100)
Hofer-Julian Sep 20, 2024
09c4868
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 20, 2024
2a4c451
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 23, 2024
c17fd80
feat: add pixi global expose command (#2030)
nichmor Sep 24, 2024
efa6d51
fix: cleanup initial manifest creation (#2124)
ruben-arts Sep 24, 2024
aaaf70d
feat: `fs_err` in `pixi global` (#2130)
ruben-arts Sep 24, 2024
4eef1a1
refactor: Remove error messages that aren't needed anymore
Hofer-Julian Sep 25, 2024
501da3b
Merge branch 'main' into feature/pixi-global
Hofer-Julian Sep 25, 2024
38a2a7e
Fix: ci pixi global (#2135)
ruben-arts Sep 25, 2024
33bab49
fix: executable name from path (#2136)
ruben-arts Sep 25, 2024
05cdc2c
Fix integration tests
Hofer-Julian Sep 25, 2024
4d099b6
fix: executable parsing on windows (#2142)
ruben-arts Sep 26, 2024
462f131
refactor: split package_from_conda_meta for easier reuse (#2140)
ruben-arts Sep 26, 2024
05e82c3
feat: add `pixi global install` (#2110)
Hofer-Julian Sep 30, 2024
88491a1
Merge branch 'main' into feature/pixi-global
ruben-arts Sep 30, 2024
8a8c699
refactor: remove `tokio-stream` dependency (#2154)
Hofer-Julian Sep 30, 2024
49f3b28
fix: `pixi global install` needs to adapt the manifest
Hofer-Julian Sep 30, 2024
2504345
fix: install multiple packages at once
ruben-arts Sep 30, 2024
b1197e0
feat: pixi global list (#2138)
ruben-arts Sep 30, 2024
ac95dfe
fix: global ensure manifest dir exists (#2163)
Hofer-Julian Oct 1, 2024
72e8845
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 1, 2024
73e69a6
refactor: use sync environment logic for `pixi g expose` (#2156)
ruben-arts Oct 1, 2024
4727325
Merge branch 'main' into feature/pixi-global
Hofer-Julian Oct 1, 2024
e3cf90c
Linting
Hofer-Julian Oct 1, 2024
4be8146
fix: also remove binaries of pruned environments (#2176)
ruben-arts Oct 2, 2024
68d33ec
refactor: only update one channel in `update_channel.nu` (#2178)
Hofer-Julian Oct 2, 2024
4272d94
fix: going from matchspec to pixispec and into toml value (#2173)
ruben-arts Oct 2, 2024
31bc76b
feat: add `global uninstall` (#2179)
ruben-arts Oct 2, 2024
62efae9
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 2, 2024
9ac5a27
fix: only revert environments we actually modified (#2182)
Hofer-Julian Oct 2, 2024
3e32fbb
feat: add inline tables for `exposed` and `dependencies` (#2170)
Hofer-Julian Oct 2, 2024
d958e2d
feat: always expose the executable with the name of the dependency (#…
ruben-arts Oct 3, 2024
d584a9d
feat: pixi global add
ruben-arts Oct 3, 2024
b311623
refactor: `pixi global add` follow up (#2200)
Hofer-Julian Oct 4, 2024
f1f81dd
fix: update expose cli (#2201)
Hofer-Julian Oct 4, 2024
e7c8e2b
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 7, 2024
92ff1c9
feat: add `pixi global update` (#2183)
ruben-arts Oct 7, 2024
5fc47fc
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 7, 2024
c5eb033
fix: channels creation on migrate (#2215)
ruben-arts Oct 8, 2024
dc7ae1a
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 8, 2024
ae8ce9f
fix: doc comments
Hofer-Julian Oct 8, 2024
8a2a28b
Allow `add_channel` to be unused
Hofer-Julian Oct 8, 2024
fbe0458
fix: improve global migration (#2234)
Hofer-Julian Oct 9, 2024
184f0c5
feat: improve sync reporting (#2202)
Hofer-Julian Oct 9, 2024
0d17051
feat: improve expose (#2240)
Hofer-Julian Oct 9, 2024
876ae69
feat: add version to global manifest (#2232)
Hofer-Julian Oct 9, 2024
b6bb9ae
fix: only uninstall single specified environments, only cleanup on sy…
ruben-arts Oct 10, 2024
70d4954
Merge branch 'main' into feature/pixi-global
ruben-arts Oct 10, 2024
519a27c
feat: add `pixi global edit` (#2219)
ruben-arts Oct 10, 2024
6c70e39
feat: improve global install (#2244)
Hofer-Julian Oct 10, 2024
21c1470
fix: remove broken exec (#2248)
Hofer-Julian Oct 10, 2024
d38f723
docs: first iteration on the documentation of the global feature (#2169)
ruben-arts Oct 10, 2024
7acc946
feat: `pixi global remove` (#2226)
ruben-arts Oct 11, 2024
6b2adad
feat: improve debugablity global (#2241)
ruben-arts Oct 11, 2024
a2d7a91
Merge branch 'main' into feature/pixi-global
Hofer-Julian Oct 11, 2024
ed87eaf
feat: Re-add `pixi global upgrade` (#2254)
Hofer-Julian Oct 11, 2024
5f8c44e
fix: `global install/uninstall` allow `_` in packages (#2259)
Hofer-Julian Oct 14, 2024
88e205a
fix: unique exposed name detection (#2258)
Hofer-Julian Oct 14, 2024
564cffa
fix: edit cli on windows and for both config and global (#2265)
ruben-arts Oct 14, 2024
b9d9099
docs: update `pixi global` docs (#2270)
Hofer-Julian Oct 14, 2024
110b8d5
feat: add --force-reinstall
nichmor Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ distribution-types = { git = "https://github.com/astral-sh/uv", tag = "0.4.0" }
dunce = "1.0.4"
fd-lock = "4.0.2"
flate2 = "1.0.28"
fs-err = { version = "2.11.0" }
fs_extra = "1.3.0"
futures = "0.3.30"
http = "1.1.0"
Expand Down Expand Up @@ -228,6 +229,7 @@ rattler_repodata_gateway = { workspace = true, features = [
rattler_shell = { workspace = true, features = ["sysinfo"] }
rattler_solve = { workspace = true, features = ["resolvo", "serde"] }

fs-err = { workspace = true, features = ["tokio"] }
pixi_config = { workspace = true, features = ["rattler_repodata_gateway"] }
pixi_consts = { workspace = true }
pixi_default_versions = { workspace = true }
Expand Down Expand Up @@ -302,6 +304,7 @@ strip = false

[dev-dependencies]
async-trait = { workspace = true }
fake = "2.9.2"
http = { workspace = true }
insta = { workspace = true, features = ["yaml", "glob"] }
rstest = { workspace = true }
Expand Down
3 changes: 3 additions & 0 deletions crates/pixi_consts/src/consts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ pub const TASK_CACHE_DIR: &str = "task-cache-v0";
pub const PIXI_UV_INSTALLER: &str = "uv-pixi";
pub const CONDA_PACKAGE_CACHE_DIR: &str = rattler_cache::PACKAGE_CACHE_DIR;
pub const CONDA_REPODATA_CACHE_DIR: &str = rattler_cache::REPODATA_CACHE_DIR;
// TODO: move to rattler
pub const CONDA_META_DIR: &str = "conda-meta";
pub const PYPI_CACHE_DIR: &str = "uv-cache";
pub const CONDA_PYPI_MAPPING_CACHE_DIR: &str = "conda-pypi-mapping";
pub const CACHED_ENVS_DIR: &str = "cached-envs-v0";
Expand All @@ -39,6 +41,7 @@ lazy_static! {
pub static ref TASK_STYLE: Style = Style::new().blue();
pub static ref PLATFORM_STYLE: Style = Style::new().yellow();
pub static ref ENVIRONMENT_STYLE: Style = Style::new().magenta();
pub static ref EXPOSED_NAME_STYLE: Style = Style::new().yellow();
pub static ref FEATURE_STYLE: Style = Style::new().cyan();
pub static ref SOLVE_GROUP_STYLE: Style = Style::new().cyan();
pub static ref DEFAULT_PYPI_INDEX_URL: Url = Url::parse("https://pypi.org/simple").unwrap();
Expand Down
65 changes: 62 additions & 3 deletions crates/pixi_manifest/src/channel.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,41 @@
use std::str::FromStr;

use itertools::Itertools;
use rattler_conda_types::NamedChannelOrUrl;
use serde::{de::Error, Deserialize, Deserializer};
use serde::{de::Error, Deserialize, Deserializer, Serialize, Serializer};
use serde_with::serde_as;
use toml_edit::{Table, Value};

/// A channel with an optional priority.
/// If the priority is not specified, it is assumed to be 0.
/// The higher the priority, the more important the channel is.
#[serde_as]
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize)]
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize)]
pub struct PrioritizedChannel {
pub channel: NamedChannelOrUrl,
pub priority: Option<i32>,
}

impl PrioritizedChannel {
/// The prioritized channels contain a priority, sort on this priority.
/// Higher priority comes first. [-10, 1, 0 ,2] -> [2, 1, 0, -10]
pub fn sort_channels_by_priority<'a, I>(
channels: I,
) -> impl Iterator<Item = &'a NamedChannelOrUrl>
where
I: IntoIterator<Item = &'a crate::PrioritizedChannel>,
{
channels
.into_iter()
.sorted_by(|a, b| {
let a = a.priority.unwrap_or(0);
let b = b.priority.unwrap_or(0);
b.cmp(&a)
})
.map(|prioritized_channel| &prioritized_channel.channel)
}
}

impl From<NamedChannelOrUrl> for PrioritizedChannel {
fn from(value: NamedChannelOrUrl) -> Self {
Self {
Expand Down Expand Up @@ -64,6 +85,19 @@ impl TomlPrioritizedChannelStrOrMap {
}
}

impl From<PrioritizedChannel> for TomlPrioritizedChannelStrOrMap {
fn from(channel: PrioritizedChannel) -> Self {
if let Some(priority) = channel.priority {
TomlPrioritizedChannelStrOrMap::Map(PrioritizedChannel {
channel: channel.channel,
priority: Some(priority),
})
} else {
TomlPrioritizedChannelStrOrMap::Str(channel.channel)
}
}
}

impl<'de> Deserialize<'de> for TomlPrioritizedChannelStrOrMap {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
Expand All @@ -81,8 +115,20 @@ impl<'de> Deserialize<'de> for TomlPrioritizedChannelStrOrMap {
}
}

impl Serialize for TomlPrioritizedChannelStrOrMap {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
match self {
TomlPrioritizedChannelStrOrMap::Map(map) => map.serialize(serializer),
TomlPrioritizedChannelStrOrMap::Str(str) => str.serialize(serializer),
}
}
}

/// Helper so that we can deserialize
/// [`crate::project::manifest::serde::PrioritizedChannel`] from a string or a
/// [`crate::channel::PrioritizedChannel`] from a string or a
/// map.
impl<'de> serde_with::DeserializeAs<'de, PrioritizedChannel> for TomlPrioritizedChannelStrOrMap {
fn deserialize_as<D>(deserializer: D) -> Result<PrioritizedChannel, D::Error>
Expand All @@ -93,3 +139,16 @@ impl<'de> serde_with::DeserializeAs<'de, PrioritizedChannel> for TomlPrioritized
Ok(prioritized_channel.into_prioritized_channel())
}
}

/// Helper so that we can serialize
/// [`crate::channel::PrioritizedChannel`] to a string or a
/// map.
impl serde_with::SerializeAs<PrioritizedChannel> for TomlPrioritizedChannelStrOrMap {
fn serialize_as<S>(source: &PrioritizedChannel, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let toml_prioritized_channel: TomlPrioritizedChannelStrOrMap = source.clone().into();
toml_prioritized_channel.serialize(serializer)
}
}
24 changes: 6 additions & 18 deletions crates/pixi_manifest/src/features_ext.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use indexmap::IndexSet;
use rattler_conda_types::{NamedChannelOrUrl, Platform};
use rattler_solve::ChannelPriority;

use crate::{HasManifestRef, SpecType};
use crate::{HasManifestRef, PrioritizedChannel, SpecType};

use crate::has_features_iter::HasFeaturesIter;
use crate::{pypi::pypi_options::PypiOptions, SystemRequirements};
Expand Down Expand Up @@ -35,24 +35,12 @@ pub trait FeaturesExt<'source>: HasManifestRef<'source> + HasFeaturesIter<'sourc
fn channels(&self) -> IndexSet<&'source NamedChannelOrUrl> {
// Collect all the channels from the features in one set,
// deduplicate them and sort them on feature index, default feature comes last.
let channels: IndexSet<_> = self
.features()
.flat_map(|feature| match &feature.channels {
Some(channels) => channels,
None => &self.manifest().parsed.project.channels,
})
.collect();
let channels = self.features().flat_map(|feature| match &feature.channels {
Some(channels) => channels,
None => &self.manifest().parsed.project.channels,
});

// The prioritized channels contain a priority, sort on this priority.
// Higher priority comes first. [-10, 1, 0 ,2] -> [2, 1, 0, -10]
channels
.sorted_by(|a, b| {
let a = a.priority.unwrap_or(0);
let b = b.priority.unwrap_or(0);
b.cmp(&a)
})
.map(|prioritized_channel| &prioritized_channel.channel)
.collect()
PrioritizedChannel::sort_channels_by_priority(channels).collect()
}

/// Returns the channel priority, error on multiple values, return None if no value is set.
Expand Down
6 changes: 4 additions & 2 deletions crates/pixi_manifest/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@ mod validation;
pub use dependencies::{CondaDependencies, Dependencies, PyPiDependencies};

pub use manifests::manifest::{Manifest, ManifestKind};
pub use manifests::TomlManifest;

pub use crate::environments::Environments;
pub use crate::parsed_manifest::ParsedManifest;
pub use crate::parsed_manifest::{deserialize_package_map, ParsedManifest};
pub use crate::solve_group::{SolveGroup, SolveGroups};
pub use activation::Activation;
pub use channel::PrioritizedChannel;
pub use channel::{PrioritizedChannel, TomlPrioritizedChannelStrOrMap};
pub use environment::{Environment, EnvironmentName};
pub use error::TomlError;
pub use feature::{Feature, FeatureName};
use itertools::Itertools;
pub use metadata::ProjectMetadata;
Expand Down
58 changes: 57 additions & 1 deletion crates/pixi_manifest/src/manifests/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::fmt;

use toml_edit::{self, Array, Item, Table, TableLike, Value};

pub mod project;
Expand All @@ -20,10 +22,15 @@ impl TomlManifest {
Self(document)
}

/// Get or insert a top-level item
pub fn get_or_insert<'a>(&'a mut self, key: &str, item: Item) -> &'a Item {
self.0.entry(key).or_insert(item)
}

/// Retrieve a mutable reference to a target table `table_name`
/// in dotted form (e.g. `table1.table2`) from the root of the document.
/// If the table is not found, it is inserted into the document.
fn get_or_insert_nested_table<'a>(
pub fn get_or_insert_nested_table<'a>(
&'a mut self,
table_name: &str,
) -> Result<&'a mut dyn TableLike, TomlError> {
Expand All @@ -45,6 +52,49 @@ impl TomlManifest {
Ok(current_table)
}

/// Inserts a value into a certain table
/// If the most inner table doesn't exist, an inline table will be created.
/// If it already exists, the formatting of the table will be preserved
pub fn insert_into_inline_table<'a>(
&'a mut self,
table_name: &str,
key: &str,
value: Value,
) -> Result<&'a mut dyn TableLike, TomlError> {
let mut parts: Vec<&str> = table_name.split('.').collect();

let last = parts.pop();

let mut current_table = self.0.as_table_mut() as &mut dyn TableLike;

for part in parts {
let entry = current_table.entry(part);
let item = entry.or_insert(Item::Table(Table::new()));
if let Some(table) = item.as_table_mut() {
// Avoid creating empty tables
table.set_implicit(true);
}
current_table = item
.as_table_like_mut()
.ok_or_else(|| TomlError::table_error(part, table_name))?;
}

// Add dependency as inline table if it doesn't exist
if let Some(last) = last {
if let Some(dependency) = current_table.get_mut(last) {
dependency
.as_table_like_mut()
.map(|table| table.insert(key, Item::Value(value)));
} else {
let mut dependency = toml_edit::InlineTable::new();
dependency.insert(key, value);
current_table.insert(last, toml_edit::value(dependency));
}
}

Ok(current_table)
}

/// Retrieves a mutable reference to a target array `array_name`
/// in table `table_name` in dotted form (e.g. `table1.table2.array`).
///
Expand Down Expand Up @@ -78,6 +128,12 @@ impl TomlManifest {
}
}

impl fmt::Display for TomlManifest {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}

#[cfg(test)]
mod tests {
use std::str::FromStr;
Expand Down
1 change: 1 addition & 0 deletions crates/pixi_utils/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,4 @@ url = { workspace = true }

[dev-dependencies]
insta = { workspace = true }
rstest = { workspace = true }
Loading
Loading