Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add External Source (and External Event) Attribute Validation #1560

Open
wants to merge 64 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
5be7bbd
Prototype of creating external source/event types thru Gateway
JosephVolosin Nov 8, 2024
0a452b3
Prototype WIP
JosephVolosin Nov 8, 2024
12b2811
WIP Prototype of visualizing attributes
JosephVolosin Nov 15, 2024
ea7a54a
update to play better with gateway and source validation
Nov 15, 2024
cc8326f
Prototype cleanup, re-adding permissions, removing code that is now i…
JosephVolosin Nov 15, 2024
35e6ac7
Lint/format cleanup
JosephVolosin Nov 15, 2024
e691c60
fix Uploading... bug
Nov 15, 2024
f8e05eb
additional error handling
Nov 18, 2024
aaa2c8c
remove allowed event types from source type creation
Nov 18, 2024
617ba69
finish error handling
Nov 18, 2024
68949eb
complete error handling
Nov 18, 2024
321f44e
Rework JsonSchema to translate to ValueSchema for UI
JosephVolosin Nov 18, 2024
eebc359
Cleanup
JosephVolosin Nov 18, 2024
c78a997
Use <Parameters> in Management modal
JosephVolosin Nov 18, 2024
4fa7490
Remove extra handling for errors
JosephVolosin Nov 18, 2024
0964e6c
Close creation modal after successful creation
JosephVolosin Nov 18, 2024
1b2b3f6
Add attributes to expected test output, fix formatting
JosephVolosin Nov 19, 2024
79a6e54
Error handling fixes & e2e test updates
JosephVolosin Nov 19, 2024
319864f
Improve handling for non-existent types
JosephVolosin Nov 20, 2024
09ffa66
Alter failure toast messages
JosephVolosin Nov 20, 2024
8c9a639
Lint fix
JosephVolosin Nov 20, 2024
4a6d087
Refactor createExternalSource to send parsed JSON as form data
JosephVolosin Nov 21, 2024
57125b5
update effects for new source type uploading system
Nov 22, 2024
4209f71
WIP - Update modal for creating types from a single schema
JosephVolosin Nov 22, 2024
0fa0f97
Rework Create & Manage modals into one, fix bug with ExternalEventForm
JosephVolosin Nov 25, 2024
f46bea9
Move external type management modal to sub-page of External Sources
JosephVolosin Nov 26, 2024
3a46a88
Update e2e tests for new 'type' page
JosephVolosin Nov 26, 2024
bf7f630
Updating derivation examples for e2e tests
JosephVolosin Nov 26, 2024
72c54a3
--amend
JosephVolosin Nov 26, 2024
5264a98
Add support for arrays as series
JosephVolosin Nov 27, 2024
da4f756
Fix issue with auto-selecting source after upload
JosephVolosin Nov 27, 2024
b06ad74
Formatting
JosephVolosin Nov 27, 2024
2891bde
Always show all 3 tables in type manager
JosephVolosin Dec 3, 2024
e4ff3e3
Linting & test fixes
JosephVolosin Dec 3, 2024
02bf1f0
Formatting on ExternalTypeManager, use Panels like Dictionaries page …
JosephVolosin Dec 4, 2024
b8771bc
Update formatting for Arrays in attribute schemas, add associated sou…
JosephVolosin Dec 4, 2024
4ab9871
Formatting
JosephVolosin Dec 4, 2024
3ef7033
Properly translate nested JsonSchema-typed items to ValueSchema
JosephVolosin Dec 4, 2024
7b78b2c
Add missing deselectSource
JosephVolosin Dec 5, 2024
454414f
E2E test fixes
JosephVolosin Dec 12, 2024
646e437
Remove out-of-date comment
JosephVolosin Dec 13, 2024
ff43925
Fix typo; names => name
JosephVolosin Dec 13, 2024
a323752
update to allow automatic creation of types and allow either/or event…
Dec 30, 2024
6bd17dd
show required attributes
Jan 3, 2025
f17233d
Fix bug with not displaying objects/structs nested within arrays/series
JosephVolosin Jan 3, 2025
8610047
update tests
Jan 3, 2025
dc1b154
handle sources with some defined and some undefined types more clearl…
JosephVolosin Nov 19, 2024
cb35b5a
Formatting fix
JosephVolosin Jan 8, 2025
60c8fc8
Add null condition for source/event types to be created
JosephVolosin Jan 8, 2025
726751d
Remove console.logs
JosephVolosin Jan 8, 2025
af94f72
Fix comparison between event attribute keys during upload
JosephVolosin Jan 10, 2025
126b1cd
Format fix
JosephVolosin Jan 10, 2025
3d4cd95
Fix linting
JosephVolosin Jan 23, 2025
bd86c3a
Fix svelte-check fail
JosephVolosin Jan 27, 2025
bcfcd7d
Convert ExternalTypeManager DataGrids to BulkActionDataGrids
JosephVolosin Jan 28, 2025
8d987dc
Show 'No Attributes Found' if no Event attributes are present
JosephVolosin Jan 29, 2025
f296a94
Add option to ignore valueSource for Parameters
JosephVolosin Jan 29, 2025
d382ee8
Remove 'view' button from ExternalTypeManager (and view fixes)
JosephVolosin Jan 29, 2025
48bc831
Condense if/elif/else structure for header/body sections
JosephVolosin Jan 29, 2025
04fd923
Show 'No Attributes Found' in definition if no Event attributes are p…
JosephVolosin Jan 29, 2025
065ebb7
Add missing typography class
JosephVolosin Jan 29, 2025
c15bef9
Remove right-formatting from attribute definition
JosephVolosin Jan 29, 2025
4213b2a
Fix formatting
JosephVolosin Jan 29, 2025
6851bb3
Fix e2e tests
JosephVolosin Jan 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions e2e-tests/data/Schema_Example_Derivation.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
{
"event_types": {
"DerivationA": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationB": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationC": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationD": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
}
},
"source_types": {
"DerivationTest": {
"type": "object",
"required": [
"exampleAttribute"
],
"properties": {
"exampleAttribute": {
"type": "string"
}
}
}
}
}
46 changes: 46 additions & 0 deletions e2e-tests/data/Schema_Example_Source.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"event_types": {
"ExampleEvent": {
"title": "ExampleEvent",
"description": "Schema for the attributes of the ExampleEvent External Event Type.",
"type": "object",
"properties": {
"example1": {
"type": "string"
},
"example2": {
"type": "number"
},
"optional": {
"type": "string"
}
},
"required": [
"example1",
"example2"
]
}
},
"source_types": {
"Example External Source": {
"title": "Example External Source",
"description": "Schema for the attributes of the Example External Source External Source Type.",
"type": "object",
"properties": {
"optional": {
"type": "string"
},
"version": {
"type": "number"
},
"spacecraft": {
"type": "string"
}
},
"required": [
"version",
"spacecraft"
]
}
}
}
11 changes: 4 additions & 7 deletions e2e-tests/data/example-external-source.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-002T00:00:00Z"
},
"metadata": {
"attributes": {
"version": 1,
"spacecraft": [
"sc1",
"sc2"
]
"spacecraft": "sc1"
}
},
"events": [
Expand All @@ -21,10 +18,10 @@
"event_type": "ExampleEvent",
"start_time": "2022-001T12:00:00Z",
"duration": "00:00:00",
"properties": {
"attributes": {
"example1": "value",
"example2": 1
}
}
]
}
}
21 changes: 21 additions & 0 deletions e2e-tests/data/example-external-source_no-attr.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"source": {
"key": "ExampleExternalSource:example-external-source_no-attr.json",
"source_type": "Empty External Source",
"valid_at": "2024-001T00:00:00Z",
"period": {
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-002T00:00:00Z"
},
"attributes": {}
},
"events": [
{
"key": "EmptyEvent:1/sc/sc1:1:X/1",
"event_type": "EmptyEvent",
"start_time": "2022-001T12:00:00Z",
"duration": "00:00:00",
"attributes": {}
}
]
}
37 changes: 16 additions & 21 deletions e2e-tests/data/external-event-derivation-1.json
Original file line number Diff line number Diff line change
@@ -1,53 +1,48 @@
{
"source": {
"key": "external-event-derivation-1.json",
"source_type": "Derivation Test",
"source_type": "DerivationTest",
"valid_at": "2022-018T00:00:00Z",
"period": {
"start_time": "2022-005T00:00:00Z",
"end_time": "2022-011T00:00:00Z"
},
"metadata": {}
"attributes": {
"exampleAttribute": "TestString"
}
},
"events": [
{
"key": "2",
"event_type": "DerivationD",
"start_time": "2022-005T23:00:00Z",
"duration": "01:10:00",
"properties": {
"rules": [
3,
4
],
"notes": "subsumed by test 01, even though end lies outside of 01, also replaced by test 01 by key",
"should_present": false
"attributes": {
"rules": "3, 4",
"notes": "subsumed by test 01, even though end lies outside of 01, also replaced by test 01 by key",
"should_present": false
}
},
{
"key": "7",
"event_type": "DerivationC",
"start_time": "2022-009T23:00:00Z",
"duration": "02:00:00",
"properties": {
"rules": [
3
],
"notes": "subsumed by test 02, even though end lies outside of 01, because start time during 01",
"should_present": false
"attributes": {
"rules": "3",
"notes": "subsumed by test 02, even though end lies outside of 01, because start time during 01",
"should_present": false
}
},
{
"key": "8",
"event_type": "DerivationB",
"start_time": "2022-010T11:00:00Z",
"duration": "01:05:00",
"properties": {
"rules": [
1
],
"notes": "after everything, subsumed by nothing despite being from oldest file",
"should_present": true
"attributes": {
"rules": "1",
"notes": "after everything, subsumed by nothing despite being from oldest file",
"should_present": true
}
}
]
Expand Down
46 changes: 20 additions & 26 deletions e2e-tests/data/external-event-derivation-2.json
Original file line number Diff line number Diff line change
@@ -1,65 +1,59 @@
{
"source": {
"key": "external-event-derivation-2.json",
"source_type": "Derivation Test",
"source_type": "DerivationTest",
"valid_at": "2022-019T00:00:00Z",
"period": {
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-007T00:00:00Z"
},
"metadata": {}
"attributes": {
"exampleAttribute": "TestString"
}
},
"events": [
{
"key": "1",
"event_type": "DerivationA",
"start_time": "2022-001T00:00:00Z",
"duration": "02:10:00",
"properties": {
"rules": [
1
],
"notes": "before everything, subsumed by nothing",
"should_present": true
"attributes": {
"rules": "1",
"notes": "before everything, subsumed by nothing",
"should_present": true
}
},
{
"key": "2",
"event_type": "DerivationA",
"start_time": "2022-001T12:00:00Z",
"duration": "02:10:00",
"properties": {
"rules": [
4
],
"notes": "overwritten by key in later file, even with type change",
"should_present": false
"attributes": {
"rules": "4",
"notes": "overwritten by key in later file, even with type change",
"should_present": false
}
},
{
"key": "3",
"event_type": "DerivationB",
"start_time": "2022-002T23:00:00Z",
"duration": "03:00:00",
"properties": {
"rules": [
2
],
"notes": "starts before next file though occurs during next file, still included",
"should_present": true
"attributes": {
"rules": "2",
"notes": "starts before next file though occurs during next file, still included",
"should_present": true
}
},
{
"key": "4",
"event_type": "DerivationB",
"start_time": "2022-005T21:00:00Z",
"duration": "03:00:00",
"properties": {
"rules": [
3
],
"notes": "start subsumed by 02, not included in final result",
"should_present": false
"attributes": {
"rules": "3",
"notes": "start subsumed by 02, not included in final result",
"should_present": false
}
}
]
Expand Down
Loading
Loading