Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Correlation feature #394

Merged
merged 73 commits into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
24bb27f
Correlation route and page setup
Nov 8, 2024
fdd6389
Added logTable toolbar
praveen5959 Nov 13, 2024
66b8237
Added sidebar and collapsable config
praveen5959 Nov 13, 2024
f37f28c
Added sidebar and collapsable config
praveen5959 Nov 13, 2024
544b400
Added new UX few items
praveen5959 Nov 14, 2024
baf8901
Added new desing flow for correlation
praveen5959 Nov 14, 2024
84ccdb8
correlation
praveen5959 Nov 15, 2024
edc3b90
Created Correlation store and new UX changes
praveen5959 Nov 19, 2024
c532cf2
Created Correlation store and new UX changes
praveen5959 Nov 19, 2024
ea155f3
Merge pull request #3 from praveen5959/main
praveen5959 Nov 19, 2024
71637c3
Moved useCorrelationQueryLogs to react-query
praveen5959 Nov 19, 2024
aae5b1e
A seperate hook for schema call
praveen5959 Nov 20, 2024
85a7254
Added stream-prefix
praveen5959 Nov 21, 2024
c4688e8
Moved to two stream approach
praveen5959 Nov 22, 2024
9a6eef3
Added correltation query logic
praveen5959 Dec 3, 2024
2a90341
Moved Querier to new component
praveen5959 Dec 4, 2024
c63df2a
fix: Add or update OFFSET clause in query (#376)
praveen5959 Nov 27, 2024
d8ffd31
fix: handle null userRoles in stream metadata fetching logic (#383)
praveen5959 Nov 27, 2024
b63380f
feat: improve UX and DX in delete and reset modals (#382)
pranavgoel29 Nov 28, 2024
721da8a
Moved Querier to new component
praveen5959 Dec 4, 2024
3c2dc2c
Added event lint graph to correlation
praveen5959 Dec 5, 2024
313e5a0
Added multi line graph to events
praveen5959 Dec 6, 2024
d650324
feat: collapsible explore page sidebar. (#378)
praveen5959 Dec 6, 2024
5cba882
feat: move timeRange to AppStore (#384)
praveen5959 Dec 6, 2024
9328b52
Rebased
praveen5959 Dec 6, 2024
38b876b
Rebased
praveen5959 Dec 6, 2024
bb53abd
Moved Querier to new component
praveen5959 Dec 4, 2024
10b3e76
Rebased to main
praveen5959 Dec 6, 2024
bfa07c1
Fixed extra schema call issue
praveen5959 Dec 8, 2024
4f6205c
Fixed the Event Graph queries
praveen5959 Dec 10, 2024
a98e889
Fixed the SQL query with alias
praveen5959 Dec 11, 2024
1f6d3ab
UI CHanges
praveen5959 Dec 19, 2024
3bcb3f4
Fixed SQL query
praveen5959 Dec 19, 2024
cddf346
Added Legend color to ChartTooltip
praveen5959 Dec 19, 2024
9ee3269
Fixed the EventGraph issue
praveen5959 Dec 19, 2024
827b26a
Fixed FieldItem Width issue
praveen5959 Dec 19, 2024
df67dec
Added loaders and removed dead code
praveen5959 Dec 20, 2024
202b024
Handled delete stream row bug
praveen5959 Dec 20, 2024
eb233e4
Resolved merge conflicts
praveen5959 Dec 20, 2024
5c98cb1
Added new CorrelationICon for navbar
praveen5959 Dec 20, 2024
751ae35
Added stepper for placeholder
praveen5959 Dec 20, 2024
4f3ec2a
Resolved lint issues
praveen5959 Dec 20, 2024
b5497ee
Fixed the after correltaion col addition bug
praveen5959 Dec 21, 2024
177431a
Fixed the deep nest warning in Log Table
praveen5959 Dec 21, 2024
f676bb0
Fixed the timeRange and other data view issues
praveen5959 Dec 23, 2024
6ece5d5
Fixed the pagination issues
praveen5959 Dec 24, 2024
bf5f3a7
Added loading state on schema call
praveen5959 Dec 24, 2024
01657d3
Merge branch 'main' of https://github.com/praveen5959/console into co…
praveen5959 Dec 24, 2024
2d5d674
Refactored CorrelationProvider
praveen5959 Dec 26, 2024
cba8817
Moved FieldItem to seperate component
praveen5959 Dec 26, 2024
bc44b05
Removed lodash dep from Correlation index.tsx
praveen5959 Dec 26, 2024
85a7fc1
Merge branch 'main' of https://github.com/praveen5959/console into co…
praveen5959 Dec 26, 2024
4dd31d3
Added Tooltips to the field item
praveen5959 Dec 26, 2024
7f19d6d
Big fixes reagarding delete field & Joins
praveen5959 Dec 26, 2024
a994d64
Fixed the scroll area CSS
praveen5959 Dec 26, 2024
5f42766
FIxed Empty rows case and loading states
praveen5959 Dec 26, 2024
06e61b0
Merge branch 'main' of https://github.com/praveen5959/console into co…
praveen5959 Dec 26, 2024
05fc7a0
fix: Pagination pageData
praveen5959 Dec 31, 2024
137f95f
fix: Totalpages count on pagination fixed
praveen5959 Jan 1, 2025
c2e4ebb
Bug fixes (timeRange page count, Share Btn)
praveen5959 Jan 6, 2025
f9f3a5e
Merge branch 'main' of https://github.com/praveen5959/console into co…
praveen5959 Jan 6, 2025
2bce8ae
Removed unused files
praveen5959 Jan 10, 2025
801ce01
Removed empty file
praveen5959 Jan 10, 2025
006298b
Resolved merge conflicts
praveen5959 Jan 10, 2025
28a4351
Removed QueryEngine Logic
praveen5959 Jan 10, 2025
e2ff453
Improved the CorelationIcon logic
praveen5959 Jan 10, 2025
35e8a88
fix: Correlate button disable states fixed
praveen5959 Jan 12, 2025
667f685
Removed logs
praveen5959 Jan 12, 2025
12da404
Merge branch 'main' of https://github.com/praveen5959/console into co…
praveen5959 Jan 13, 2025
0aedb67
Bug fixes
praveen5959 Jan 13, 2025
4ce85b9
fix: Correlate btn and explore page changes
praveen5959 Jan 13, 2025
d592b75
Merge branch 'correlation' of https://github.com/praveen5959/console …
praveen5959 Jan 13, 2025
5b57f6c
Lint fixes
praveen5959 Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/@types/parseable/api/query.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { QueryEngineType } from '@/@types/parseable/api/about';
export type LogsQuery = {
queryEngine?: QueryEngineType;
streamName: string;
streamName?: string;
startTime: Date;
endTime: Date;
access: string[] | null;
Expand Down
24 changes: 23 additions & 1 deletion src/api/query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { LOG_QUERY_URL } from './constants';
import { Log, LogsQuery, LogsResponseWithHeaders } from '@/@types/parseable/api/query';
import { QueryEngineType } from '@/@types/parseable/api/about';
import timeRangeUtils from '@/utils/timeRangeUtils';
import { QueryBuilder } from '@/utils/queryBuilder';
import { CorrelationQueryBuilder, QueryBuilder } from '@/utils/queryBuilder';

const { formatDateAsCastType } = timeRangeUtils;
type QueryEngine = QueryEngineType;
Expand All @@ -16,6 +16,16 @@ type QueryLogs = {
pageOffset: number;
};

type CorrelationLogs = {
streamNames: string[];
startTime: Date;
endTime: Date;
limit: number;
queryEngine?: QueryEngine;
correlationCondition?: string;
selectedFields?: string[];
};

// to optimize query performace, it has been decided to round off the time at the given level
const optimizeTime = (date: Date) => {
const tempDate = new Date(date);
Expand Down Expand Up @@ -56,6 +66,18 @@ export const getQueryLogsWithHeaders = (logsQuery: QueryLogs) => {
return Axios().post<LogsResponseWithHeaders>(endPoint, formQueryOpts(logsQuery), {});
};

export const getCorrelationQueryLogsWithHeaders = (logsQuery: CorrelationLogs) => {
const queryBuilder = new CorrelationQueryBuilder(logsQuery);
const endPoint = LOG_QUERY_URL({ fields: true }, queryBuilder.getResourcePath());
return Axios().post<LogsResponseWithHeaders>(endPoint, queryBuilder.getCorrelationQuery(), {});
};

export const getStreamDataWithHeaders = (logsQuery: CorrelationLogs) => {
const queryBuilder = new CorrelationQueryBuilder(logsQuery);
const endPoint = LOG_QUERY_URL({ fields: true }, queryBuilder.getResourcePath());
return Axios().post<LogsResponseWithHeaders>(endPoint, queryBuilder.getQuery(), {});
};

// ------ Custom sql query

const makeCustomQueryRequestData = (logsQuery: LogsQuery, query: string) => {
Expand Down
21 changes: 21 additions & 0 deletions src/assets/images/correlation_placeholder.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
30 changes: 30 additions & 0 deletions src/components/Navbar/components/CorrelationIcon.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { forwardRef } from 'react';

export const CorrelationIcon = forwardRef<
SVGSVGElement,
{
stroke?: string;
strokeWidth?: number;
height?: string;
width?: string;
}
>(({ stroke, strokeWidth, height, width }, ref) => (
<svg ref={ref} height={height} width={width} viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M13.3333 17.3333L14.6667 18.6667C15.0203 19.0203 15.4999 19.219 16 19.219C16.5001 19.219 16.9797 19.0203 17.3333 18.6667L22.6667 13.3333C23.0203 12.9797 23.219 12.5001 23.219 12C23.219 11.4999 23.0203 11.0203 22.6667 10.6667L17.3333 5.33333C16.9797 4.97971 16.5001 4.78105 16 4.78105C15.4999 4.78105 15.0203 4.97971 14.6667 5.33333L9.33333 10.6667C8.97971 11.0203 8.78105 11.4999 8.78105 12C8.78105 12.5001 8.97971 12.9797 9.33333 13.3333L10.6667 14.6667"
stroke={stroke}
strokeWidth={strokeWidth}
strokeLinecap="round"
strokeLinejoin="round"
/>
<path
d="M10.6667 6.66667L9.33333 5.33333C8.97971 4.97971 8.5001 4.78105 8 4.78105C7.4999 4.78105 7.02029 4.97971 6.66667 5.33333L1.33333 10.6667C0.979711 11.0203 0.781049 11.4999 0.781049 12C0.781049 12.5001 0.979711 12.9797 1.33333 13.3333L6.66667 18.6667C7.02029 19.0203 7.4999 19.219 8 19.219C8.5001 19.219 8.97971 19.0203 9.33333 18.6667L14.6667 13.3333C15.0203 12.9797 15.219 12.5001 15.219 12C15.219 11.4999 15.0203 11.0203 14.6667 10.6667L13.3333 9.33333"
stroke={stroke}
strokeWidth={strokeWidth}
strokeLinecap="round"
strokeLinejoin="round"
/>
</svg>
));

CorrelationIcon.displayName = 'CorrelationIcon';
27 changes: 25 additions & 2 deletions src/components/Navbar/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,14 @@ import { FC, useCallback, useEffect } from 'react';
import { useLocation, useParams } from 'react-router-dom';
import { useNavigate } from 'react-router-dom';
import { useDisclosure } from '@mantine/hooks';
import { HOME_ROUTE, CLUSTER_ROUTE, USERS_MANAGEMENT_ROUTE, STREAM_ROUTE, DASHBOARDS_ROUTE } from '@/constants/routes';
import {
HOME_ROUTE,
CLUSTER_ROUTE,
USERS_MANAGEMENT_ROUTE,
STREAM_ROUTE,
DASHBOARDS_ROUTE,
CORRELATION_ROUTE,
} from '@/constants/routes';
import InfoModal from './infoModal';
import { getStreamsSepcificAccess, getUserSepcificStreams } from './rolesHandler';
import Cookies from 'js-cookie';
Expand All @@ -26,6 +33,7 @@ import UserModal from './UserModal';
import { signOutHandler } from '@/utils';
import { appStoreReducers, useAppStore } from '@/layouts/MainLayout/providers/AppProvider';
import _ from 'lodash';
import { CorrelationIcon } from './components/CorrelationIcon';

const { setUserRoles, setUserSpecificStreams, setUserAccessMap, changeStream, setStreamSpecificUserAccess } =
appStoreReducers;
Expand All @@ -49,6 +57,12 @@ const navItems = [
path: '/explore',
route: STREAM_ROUTE,
},
{
icon: '',
label: 'Correlation',
path: '/correlation',
route: CORRELATION_ROUTE,
},
];

const previlagedActions = [
Expand Down Expand Up @@ -167,7 +181,16 @@ const Navbar: FC = () => {
onClick={() => navigateToPage(navItem.route)}
key={index}>
<Tooltip label={navItem.label} position="right">
<navItem.icon stroke={isActiveItem ? 1.4 : 1.2} size={'1.2rem'} />
{navItem.route === CORRELATION_ROUTE ? (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

active route can be handled with the predefined functions (same as other routes are handled)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handled it differently now

<CorrelationIcon
stroke={isActiveItem ? '#000000' : '#858e96'}
strokeWidth={isActiveItem ? 1.4 : 1.2}
height="1.2rem"
width="1.2rem"
/>
) : (
<navItem.icon stroke={isActiveItem ? 1.4 : 1.2} size="1.2rem" />
)}
</Tooltip>
</Stack>
);
Expand Down
110 changes: 110 additions & 0 deletions src/components/SideBar/Sidebar.tsx
praveen5959 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import { Stack, Tooltip } from '@mantine/core';
import classes from './styles/SideBar.module.css';
import { IconBolt, IconFilterSearch, IconSettings2 } from '@tabler/icons-react';
import { useCallback } from 'react';
import { useNavigate, useParams } from 'react-router-dom';
import { useAppStore } from '@/layouts/MainLayout/providers/AppProvider';
import { STREAM_VIEWS } from '@/constants/routes';
import _ from 'lodash';

type MenuItemProps = {
setCurrentView: (view: string) => void;
currentView: string;
};

const AllLogsButton = (props: MenuItemProps) => {
const viewName = 'explore';
const isActive = props.currentView === viewName;
const additionalClassNames = `${isActive ? classes.activeMenuItem : ''}`;
return (
<Stack
onClick={() => props.setCurrentView(viewName)}
style={{ padding: '4px 0', alignItems: 'center' }}
className={classes.menuItemContainer}>
<Tooltip label="Explore" position="right">
<Stack className={additionalClassNames} style={{ padding: '4px 4px' }}>
<IconFilterSearch
stroke={isActive ? 1.4 : 1.2}
size="1.2rem"
className={classes.icon}
{...(isActive && { color: 'black' })}
/>
</Stack>
</Tooltip>
</Stack>
);
};

const ConfigButton = (props: MenuItemProps) => {
const viewName = 'manage';
const isActive = props.currentView === viewName;
const additionalClassNames = `${isActive ? classes.activeMenuItem : ''}`;
return (
<Stack
onClick={() => props.setCurrentView(viewName)}
style={{ padding: '4px 0', alignItems: 'center' }}
className={classes.menuItemContainer}>
<Tooltip label="Manage" position="right">
<Stack className={additionalClassNames} style={{ padding: '4px 4px' }}>
<IconSettings2
stroke={isActive ? 1.4 : 1.2}
size="1.2rem"
className={classes.icon}
{...(isActive && { color: 'black' })}
/>
</Stack>
</Tooltip>
</Stack>
);
};

const LiveTailMenu = (props: MenuItemProps) => {
const viewName = 'live-tail';
const isActive = props.currentView === viewName;
const additionalClassNames = `${isActive ? classes.activeMenuItem : ''}`;
return (
<Stack
onClick={() => props.setCurrentView(viewName)}
className={classes.menuItemContainer}
style={{ padding: '4px 0', alignItems: 'center' }}>
<Tooltip label="Live Tail" position="right">
<Stack className={additionalClassNames} style={{ padding: '4px 4px' }}>
<IconBolt
stroke={isActive ? 1.4 : 1.2}
size="1.2rem"
className={classes.icon}
{...(isActive && { color: 'black' })}
/>
</Stack>
</Tooltip>
</Stack>
);
};

const SideBar = () => {
const [currentStream] = useAppStore((store) => store.currentStream);
const [isStandAloneMode] = useAppStore((store) => store.isStandAloneMode);
const { view } = useParams();
const navigate = useNavigate();

const setCurrentView = useCallback(
(view: string) => {
if (_.includes(STREAM_VIEWS, view)) {
navigate(`/${currentStream}/${view}`);
}
},
[currentStream],
);

return (
<Stack className={classes.container} style={{ gap: 0 }}>
<Stack gap={0}>
<AllLogsButton setCurrentView={setCurrentView} currentView={view || ''} />
{isStandAloneMode && <LiveTailMenu setCurrentView={setCurrentView} currentView={view || ''} />}
<ConfigButton setCurrentView={setCurrentView} currentView={view || ''} />
</Stack>
</Stack>
);
};

export default SideBar;
52 changes: 52 additions & 0 deletions src/components/SideBar/styles/SideBar.module.css
praveen5959 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
.container {
flex: 1;
gap: 1rem;
border-right: rem(1px) solid light-dark(var(--mantine-color-gray-3), var(--mantine-color-dark-4));
}

.menuItemContainer {
cursor: pointer;
}

.activeMenuItem {
padding: 4px;
background-color: var(--mantine-color-gray-1);
border-radius: 0.2rem;
}

.menuIconContainer {
background-color: transparent;
}

.menuLabel {
font-size: 0.7rem;
font-weight: 500;
padding-right: 4px;
}

.streamDropdownContainer {
/* margin-top: 0.5rem; */
/* padding: 0 1.25rem; */
position: relative;
}

.sideBarToggleContainer {
position: absolute;
left: 93%;
top: 30%;
align-items: center;
justify-content: center;
z-index: 99;
}

.sideBarToggleIconContainer {
padding: 2rem;
border-radius: 50%;
background-color: white;
border: 1px solid var(--mantine-color-gray-2);
cursor: pointer;
}

.icon {
color: var(--mantine-color-gray-6);
}
2 changes: 2 additions & 0 deletions src/constants/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ export const OIDC_NOT_CONFIGURED_ROUTE = '/oidc-not-configured';
export const CLUSTER_ROUTE = '/cluster';
export const STREAM_ROUTE = '/:streamName/:view?';
export const DASHBOARDS_ROUTE = '/dashboards';
export const CORRELATION_ROUTE = '/correlation';

export const STREAM_VIEWS = ['explore', 'manage', 'live-tail'];

Expand All @@ -27,4 +28,5 @@ export const PATHS = {
cluster: '/cluster',
manage: '/:streamName/:view?',
dashboards: '/dashboards',
correlation: '/correlation',
} as { [key: string]: string };
Loading
Loading