Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial preparation for supporting writing to any/all cdf targets #435

Merged
merged 34 commits into from
Jul 28, 2023

Conversation

toondaey
Copy link
Contributor

No description provided.

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #435 (62b4864) into master (06f41b7) will increase coverage by 0.03%.
The diff coverage is 87.38%.

@@            Coverage Diff             @@
##           master     #435      +/-   ##
==========================================
+ Coverage   83.72%   83.75%   +0.03%     
==========================================
  Files          98      108      +10     
  Lines       16064    16453     +389     
  Branches     2489     2492       +3     
==========================================
+ Hits        13449    13780     +331     
- Misses       1870     1921      +51     
- Partials      745      752       +7     
Files Changed Coverage Δ
Extractor/NodeSources/NodeSetSource.cs 86.85% <0.00%> (ø)
Extractor/NodeSources/UANodeSource.cs 94.73% <0.00%> (ø)
Extractor/Nodes/UADataType.cs 94.08% <0.00%> (ø)
Extractor/Pushers/FDM/DMSValueConverter.cs 61.22% <ø> (ø)
Extractor/Pushers/Writers/Interfaces/IRawWriter.cs 0.00% <0.00%> (ø)
ExtractorLauncher/ExtractorStarter.cs 57.84% <20.75%> (-7.77%) ⬇️
Extractor/Pushers/FDM/FDMWriter.cs 78.53% <50.00%> (ø)
Extractor/Pushers/Writers/BaseTimeseriesWriter.cs 80.00% <80.00%> (ø)
...tractor/Pushers/Writers/MinimalTimeseriesWriter.cs 90.90% <90.90%> (ø)
Extractor/Pushers/CDFPusher.cs 91.98% <90.97%> (+0.79%) ⬆️
... and 14 more

... and 1 file with indirect coverage changes

@toondaey toondaey force-pushed the support-writing-to-all/dog-1760 branch from b7b5102 to fa3bd31 Compare June 29, 2023 12:20
@toondaey toondaey force-pushed the support-writing-to-all/dog-1760 branch from 0d51110 to 5722538 Compare July 5, 2023 11:30
@toondaey toondaey marked this pull request as ready for review July 6, 2023 12:01
@toondaey toondaey requested a review from a team as a code owner July 6, 2023 12:01
@toondaey toondaey changed the title proposed Initial preparation for supporting writing to any/all cdf targets Jul 6, 2023
@toondaey toondaey force-pushed the support-writing-to-all/dog-1760 branch from d71ee13 to 70fe544 Compare July 9, 2023 09:53
{
IRawWriter Raw { get; }
ITimeseriesWriter Timeseries { get; }
ITimeseriesWriter MinimalTimeseries { get; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this, and use an abstraction instead, so the Timeseries field is either a MinimalTimeseriesWriter, or FullTimeseriesWriter.

var tss = ids.Select(id => tsMap[id]);
var creates = tss.Select(
ts =>
ts.ToTimeseries(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split the ToTimeseries method as well, and call it from subclasses of some BaseTimeseriesWriter class.

Extractor/Pushers/Writers/WriterUtils.cs Show resolved Hide resolved
}
};
}
if (config.Cognite?.FlexibleDataModels != null)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need this mapping, because this isn't public.

@@ -57,6 +59,100 @@
}
}
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these blocks entirely.

ExtractorLauncher/ExtractorStarter.cs Show resolved Hide resolved
{
public bool Assets { get; set; } = true;
public bool Timeseries { get; set; } = true;
public bool Relationships { get; set; } = true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These shouldn't be true by default

/// <summary>
/// Clean metadata targets config
/// </summary>
public CleanMetadataTargetConfig? CleanMetadata { get; set; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just call this clean, and raw

/// <summary>
/// FDM destination config
/// </summary>
public FdmDestinationConfig? FlexibleDataModels { get; set; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be data-models

@toondaey toondaey requested a review from einarmo July 25, 2023 19:11
"assets-table": {
"type": "string",
"description": "Raw table to use for assets"
"raw-metadata": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You changed the name of these in config, need to update here too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think you need to update the example config file.

@toondaey toondaey requested a review from einarmo July 27, 2023 11:08
@toondaey toondaey merged commit 89904ce into master Jul 28, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants