diff --git a/docs/en/html-reporter-setup.md b/docs/en/html-reporter-setup.md index 1eecbe230..87ff87785 100644 --- a/docs/en/html-reporter-setup.md +++ b/docs/en/html-reporter-setup.md @@ -311,41 +311,27 @@ customScripts: [ ### yandexMetrika -This parameter allows you to add [Yandex.Metrika][yandex-metrika] to the report. The parameter is set as an object with the key `counterNumber`. As the key value, you must specify the Yandex.Metrica counter number (see "[How to create a counter][how-to-create-counter]"). The number should be set as a Number, not a String. +By default, anonymous html-reporter interface usage information is collected for us to analyze usage patterns and improve UX. We collect such info as html-reporter loading speed, how often certain UI features are used (e.g. sorting tests) or clicks on UI elements. NO information about your project or tests is ever tracked. -Also, in the Yandex.Metrika interface, go to the _"Counter"_ tab in the settings section, click _"Copy"_ and paste the counter code into the [customScripts](#customscripts) field. +If you want to opt out, choose any of the options below: -With the help of metrics, you can find out how developers interact with your report and what kind of problems they face. - -The report supports the following [goals of metrics][yandex-metrika-goals]: - -* **ACCEPT_SCREENSHOT**—there was a click on the _Accept_ button to accept a screenshot; -* **ACCEPT_OPENED_SCREENSHOTS**—there was a click on the _Accept opened_ button to accept screenshots from open tests. - -Example of setting up Yandex.Metrika in one of the projects: - -```javascript -module.exports = { - plugins: { - 'html-reporter/hermione': { - customScripts: [ - function(){(function(m,e,t,r,i,k,a){m[i]=m[i]||function(){(m[i].a=m[i].a||[]).push(arguments)}; m[i].l=1*new Date();k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)}) (window, document, "script", "https://mc.yandex.ru/metrika/tag.js", "ym"); ym(56782912, "init", { clickmap:true, trackLinks:true, accurateTrackBounce:true, webvisor:true })}, - - // other scripts... - ], - yandexMetrika: { - counterNumber: 1234567 +- Edit your config: + ```javascript + module.exports = { + plugins: { + 'html-reporter/testplane': { + yandexMetrika: { + enabled: false + }, + // other html-reporter settings... }, - - // other plugin settings... + // other Testplane plugins... }, - - // other hermione plugins... - }, - - // other hermione settings... -}; -``` + // other Testplane settings... + }; + ``` +- Using environment variables: `html_reporter_yandex_metrika_enabled=false` or simply `NO_ANALYTICS=true` +- Using CLI arguments: `--html-reporter-yandex_metrika_enabled=false` ### Passing parameters via the CLI diff --git a/docs/ru/html-reporter-setup.md b/docs/ru/html-reporter-setup.md index 2427c65d8..f65b9002d 100644 --- a/docs/ru/html-reporter-setup.md +++ b/docs/ru/html-reporter-setup.md @@ -312,41 +312,27 @@ customScripts: [ ### yandexMetrika -Данный параметр позволяет добавить в отчет [Яндекс.Метрику][yandex-metrika]. Параметр задается в виде объекта с ключом `counterNumber` _(номер счетчика)_. В качестве значения ключа необходимо указать номер счетчика Яндекс.Метрики (см. «[Как создать счетчик][how-to-create-counter]»). Номер должен задаваться как число _(Number)_, а не строка. +По умолчанию выполняется сбор анонимных сведений об использовании интерфейса отчета в целях анализа и улучшения UX. Собираются такие сведения, как скорость загрузки отчета, частота использования некоторых функций (например, сортировка тестов) и клики по элементам управления. Сведения о вашем проекте или содержимом тестов НЕ собираются ни при каких обстоятельствах. -Также в интерфейсе Яндекс.Метрики необходимо перейти в разделе настроек на вкладку _«Счетчик»_, нажать кнопку _«Скопировать»_ и вставить код счетчика в поле [customScripts](#customscripts). +Если вы не хотите делиться аналитикой с нами, вы можете отключить это любым из способов: -С помощью метрики вы сможете узнать как разработчики взаимодействуют с вашим отчетом и с какого рода проблемами они сталкиваются. - -Отчет поддерживает следующие [цели для метрики][yandex-metrika-goals]: - -* **ACCEPT_SCREENSHOT** — было нажатие на кнопку _Accept_ для принятия скриншота; -* **ACCEPT_OPENED_SCREENSHOTS** — было нажатие на кнопку _Accept opened_ для принятия скриншотов из открытых тестов. - -Пример настройки _Яндекс.Метрики_ в одном из проектов: - -```javascript -module.exports = { - plugins: { - 'html-reporter/hermione': { - customScripts: [ - function(){(function(m,e,t,r,i,k,a){m[i]=m[i]||function(){(m[i].a=m[i].a||[]).push(arguments)}; m[i].l=1*new Date();k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)}) (window, document, "script", "https://mc.yandex.ru/metrika/tag.js", "ym"); ym(56782912, "init", { clickmap:true, trackLinks:true, accurateTrackBounce:true, webvisor:true })}, - - // другие скрипты... - ], - yandexMetrika: { - counterNumber: 1234567 +- В конфиге + ```javascript + module.exports = { + plugins: { + 'html-reporter/testplane': { + yandexMetrika: { + enabled: false + }, + // другие настройки html-reporter... }, - - // другие настройки плагина... + // другие плагины Testplane... }, - - // другие плагины гермионы... - }, - - // другие настройки гермионы... -}; -``` + // другие настройки Testplane... + }; + ``` +- С помощью переменных окружения: `html_reporter_yandex_metrika_enabled=false` или просто `NO_ANALYTICS=true` +- С помощью аргументов CLI: `--html-reporter-yandex_metrika_enabled=false` ### Передача параметров через CLI diff --git a/lib/config/index.ts b/lib/config/index.ts index ade2e0d05..69473d134 100644 --- a/lib/config/index.ts +++ b/lib/config/index.ts @@ -28,7 +28,7 @@ const assertType = (name: string, validationFn: (value: unknown) => value is }; const assertString = (name: string): AssertionFn => assertType(name, _.isString, 'string'); const assertBoolean = (name: string): AssertionFn => assertType(name, _.isBoolean, 'boolean'); -const assertNumber = (name: string): AssertionFn => assertType(name, _.isNumber, 'number'); +export const assertNumber = (name: string): AssertionFn => assertType(name, _.isNumber, 'number'); const assertPlainObject = (name: string): AssertionFn> => assertType(name, isPlainObject, 'plain object'); const assertSaveFormat = (saveFormat: unknown): asserts saveFormat is SaveFormat => { @@ -206,11 +206,27 @@ const getParser = (): ReturnType> => { validate: assertArrayOf('functions', 'customScripts', _.isFunction) }), yandexMetrika: section({ + enabled: option({ + defaultValue: () => { + return !(process.env.NO_ANALYTICS && JSON.parse(process.env.NO_ANALYTICS)); + }, + parseEnv: JSON.parse, + parseCli: JSON.parse, + validate: assertBoolean('yandexMetrika.enabled'), + map: (value: boolean) => { + if (process.env.NO_ANALYTICS && JSON.parse(process.env.NO_ANALYTICS)) { + return false; + } + + return value; + } + }), counterNumber: option({ + isDeprecated: true, defaultValue: configDefaults.yandexMetrika.counterNumber, parseEnv: Number, parseCli: Number, - validate: (value) => _.isNull(value) || assertNumber('yandexMetrika.counterNumber')(value) + map: () => configDefaults.yandexMetrika.counterNumber }) }), pluginsEnabled: option({ diff --git a/lib/constants/defaults.ts b/lib/constants/defaults.ts index 0fd4b17ac..ea6e28a41 100644 --- a/lib/constants/defaults.ts +++ b/lib/constants/defaults.ts @@ -22,7 +22,7 @@ export const configDefaults: StoreReporterConfig = { saveErrorDetails: false, saveFormat: SaveFormat.SQLITE, yandexMetrika: { - counterNumber: null + counterNumber: 99267510 }, staticImageAccepter: { enabled: false, diff --git a/lib/static/components/controls/report-info.jsx b/lib/static/components/controls/report-info.jsx index 1fece14a5..bdad5300a 100644 --- a/lib/static/components/controls/report-info.jsx +++ b/lib/static/components/controls/report-info.jsx @@ -7,8 +7,10 @@ import {isEmpty} from 'lodash'; import {version} from '../../../../package.json'; import useLocalStorage from '@/static/hooks/useLocalStorage'; import {LocalStorageKey, UiMode} from '@/constants/local-storage'; +import {useAnalytics} from '@/static/new-ui/hooks/useAnalytics'; function ReportInfo(props) { + const analytics = useAnalytics(); const {gui, timestamp} = props; const lang = isEmpty(navigator.languages) ? navigator.language : navigator.languages[0]; const date = new Date(timestamp).toLocaleString(lang); @@ -16,6 +18,7 @@ function ReportInfo(props) { const [, setUiMode] = useLocalStorage(LocalStorageKey.UIMode, UiMode.New); const onNewUiButtonClick = () => { + analytics?.trackFeatureUsage({featureName: 'Switch to new UI'}); setUiMode(UiMode.New); const targetUrl = new URL(window.location.href); diff --git a/lib/static/components/gui.jsx b/lib/static/components/gui.jsx index 6c4340673..966f4b92c 100644 --- a/lib/static/components/gui.jsx +++ b/lib/static/components/gui.jsx @@ -15,6 +15,7 @@ import {ClientEvents} from '../../gui/constants/client-events'; import FaviconChanger from './favicon-changer'; import ExtensionPoint from './extension-point'; import BottomProgressBar from './bottom-progress-bar'; +import {MetrikaScript} from '@/static/new-ui/components/MetrikaScript'; class Gui extends Component { static propTypes = { @@ -81,6 +82,7 @@ class Gui extends Component { + {notificationElem} diff --git a/lib/static/components/report.jsx b/lib/static/components/report.jsx index 4b9104451..e45b545c9 100644 --- a/lib/static/components/report.jsx +++ b/lib/static/components/report.jsx @@ -14,6 +14,7 @@ import {CustomScripts} from '../new-ui/components/CustomScripts'; import FaviconChanger from './favicon-changer'; import ExtensionPoint from './extension-point'; import BottomProgressBar from './bottom-progress-bar'; +import {MetrikaScript} from '@/static/new-ui/components/MetrikaScript'; class Report extends Component { static propTypes = { @@ -50,6 +51,7 @@ class Report extends Component { + {notificationElem} diff --git a/lib/static/modules/middlewares/metrika.ts b/lib/static/modules/middlewares/metrika.ts index 6788f2b84..71fb6e065 100644 --- a/lib/static/modules/middlewares/metrika.ts +++ b/lib/static/modules/middlewares/metrika.ts @@ -32,7 +32,8 @@ export function getMetrikaMiddleware(analytics: YandexMetrika): Middleware<{}, S analytics.setVisitParams({ [action.type]: Date.now() - startLoadTime, initView: state.view, - testsCount + testsCount, + isNewUi: Boolean(state?.app?.isNewUi) }); return result; @@ -82,6 +83,32 @@ export function getMetrikaMiddleware(analytics: YandexMetrika): Middleware<{}, S return next(action); } + case actionNames.GROUP_TESTS_SET_CURRENT_EXPRESSION: { + analytics.trackFeatureUsage({featureName: action.type, groupByKey: action.payload.expressionIds[0]}); + + return next(action); + } + + case actionNames.SORT_TESTS_SET_CURRENT_EXPRESSION: { + analytics.trackFeatureUsage({ + featureName: action.type, + sortByKey: action.payload.expressionIds[0], + sortDirection: store.getState().app.sortTestsData.currentDirection + }); + + return next(action); + } + + case actionNames.SORT_TESTS_SET_DIRECTION: { + analytics.trackFeatureUsage({ + featureName: action.type, + sortByKey: store.getState().app.sortTestsData.currentExpressionIds[0], + sortDirection: action.payload.direction + }); + + return next(action); + } + case actionNames.OPEN_MODAL: case actionNames.CLOSE_MODAL: { const modalId = get(action, 'payload.id', action.type); diff --git a/lib/static/modules/store.js b/lib/static/modules/store.js index f92dcbc3c..7977376a6 100644 --- a/lib/static/modules/store.js +++ b/lib/static/modules/store.js @@ -21,11 +21,11 @@ if (process.env.NODE_ENV !== 'production') { } const metrikaConfig = (window.data || {}).config?.yandexMetrika; -const areAnalyticsEnabled = metrikaConfig?.enabled && metrikaConfig?.counterId; +const areAnalyticsEnabled = metrikaConfig?.enabled && metrikaConfig?.counterNumber; const isYaMetrikaAvailable = window.ym && typeof window.ym === 'function'; if (areAnalyticsEnabled && isYaMetrikaAvailable) { - const metrika = new YandexMetrika(); + const metrika = new YandexMetrika(areAnalyticsEnabled && isYaMetrikaAvailable); const metrikaMiddleware = getMetrikaMiddleware(metrika); middlewares.push(metrikaMiddleware); diff --git a/lib/static/modules/yandex-metrika.ts b/lib/static/modules/yandex-metrika.ts index fde9f0cb7..0229a5781 100644 --- a/lib/static/modules/yandex-metrika.ts +++ b/lib/static/modules/yandex-metrika.ts @@ -1,3 +1,5 @@ +import {SortDirection} from '@/static/new-ui/types/store'; + enum YandexMetrikaMethod { ReachGoal = 'reachGoal', Params = 'params', @@ -24,10 +26,29 @@ interface ScreenshotAcceptData { acceptedImagesCount: number; } -interface FeatureUsageData { +interface BasicFeatureUsageData { featureName: string; } +interface GroupByFeatureUsageData extends BasicFeatureUsageData { + groupByKey: string; +} + +interface SortByFeatureUsageData extends BasicFeatureUsageData { + sortByKey: string; + sortDirection: SortDirection; +} + +interface ChangeTreeViewModeFeatureUsageData extends BasicFeatureUsageData { + treeViewMode: string; +} + +type FeatureUsageData = + | BasicFeatureUsageData + | GroupByFeatureUsageData + | SortByFeatureUsageData + | ChangeTreeViewModeFeatureUsageData; + export class YandexMetrika { protected readonly _isEnabled: boolean; protected readonly _counterNumber: number; diff --git a/lib/static/new-ui/app/App.tsx b/lib/static/new-ui/app/App.tsx index 6c44b9fe1..b540cba4f 100644 --- a/lib/static/new-ui/app/App.tsx +++ b/lib/static/new-ui/app/App.tsx @@ -16,6 +16,7 @@ import store from '../../modules/store'; import {CustomScripts} from '@/static/new-ui/components/CustomScripts'; import {State} from '@/static/new-ui/types/store'; import {AnalyticsProvider} from '@/static/new-ui/providers/analytics'; +import {MetrikaScript} from '@/static/new-ui/components/MetrikaScript'; export function App(): ReactNode { const pages = [ @@ -36,6 +37,7 @@ export function App(): ReactNode { + diff --git a/lib/static/new-ui/components/AttemptPicker/index.tsx b/lib/static/new-ui/components/AttemptPicker/index.tsx index e7706b98c..49991a37a 100644 --- a/lib/static/new-ui/components/AttemptPicker/index.tsx +++ b/lib/static/new-ui/components/AttemptPicker/index.tsx @@ -9,6 +9,7 @@ import {Button, Icon, Spin} from '@gravity-ui/uikit'; import {RunTestsFeature} from '@/constants'; import {thunkRunTest} from '@/static/modules/actions'; import {getCurrentBrowser, getCurrentResultId} from '@/static/new-ui/features/suites/selectors'; +import {useAnalytics} from '@/static/new-ui/hooks/useAnalytics'; interface AttemptPickerProps { onChange?: (browserId: string, resultId: string, attemptIndex: number) => unknown; @@ -23,6 +24,7 @@ interface AttemptPickerInternalProps extends AttemptPickerProps { function AttemptPickerInternal(props: AttemptPickerInternalProps): ReactNode { const {resultIds, currentResultId} = props; + const analytics = useAnalytics(); const dispatch = useDispatch(); const currentBrowser = useSelector(getCurrentBrowser); const isRunTestsAvailable = useSelector(state => state.app.availableFeatures) @@ -39,6 +41,7 @@ function AttemptPickerInternal(props: AttemptPickerInternalProps): ReactNode { const onRetryTestHandler = (): void => { if (currentBrowser) { + analytics?.trackFeatureUsage({featureName: 'Attempt picker: retry test button click'}); dispatch(thunkRunTest({test: {testName: currentBrowser.parentId, browserName: currentBrowser.name}})); } }; diff --git a/lib/static/new-ui/components/MainLayout/Footer.tsx b/lib/static/new-ui/components/MainLayout/Footer.tsx index 3d44ade68..244f9e682 100644 --- a/lib/static/new-ui/components/MainLayout/Footer.tsx +++ b/lib/static/new-ui/components/MainLayout/Footer.tsx @@ -55,6 +55,7 @@ export function Footer(props: FooterProps): ReactNode { title: 'Info', onItemClick: props.onFooterItemClick, current: isInfoCurrent, + qa: 'footer-item-info', itemWrapper: (params, makeItem) => makeItem({ ...params, icon: ({ id: item.url, title: item.title, icon: item.icon, current: Boolean(matchPath(`${item.url.replace(/\/$/, '')}/*`, location.pathname)), - onItemClick: () => navigate(item.url) + onItemClick: (): void => { + analytics?.trackFeatureUsage({featureName: `Go to ${item.url} page`}); + navigate(item.url); + } })); const isInitialized = useSelector(getIsInitialized); @@ -47,7 +52,12 @@ export function MainLayout(props: MainLayoutProps): ReactNode { const [visiblePanel, setVisiblePanel] = useState(null); const onFooterItemClick = (item: GravityMenuItem): void => { - visiblePanel === item.id ? setVisiblePanel(null) : setVisiblePanel(item.id as PanelId); + if (visiblePanel === item.id) { + setVisiblePanel(null); + } else { + setVisiblePanel(item.id as PanelId); + analytics?.trackFeatureUsage({featureName: `Open ${item.id} panel`}); + } }; return state.config.yandexMetrika.enabled); + const ref = useRef(null); + + useEffect(() => { + if (!areAnalyticsEnabled) { + return; + } + + const s = document.createElement('script'); + s.type = 'text/javascript'; + s.innerHTML = ` + (function(m,e,t,r,i,k,a){m[i]=m[i]||function(){(m[i].a=m[i].a||[]).push(arguments)}; + m[i].l=1*new Date(); + for (var j = 0; j < document.scripts.length; j++) {if (document.scripts[j].src === r) { return; }} + k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)}) + (window, document, "script", "https://mc.yandex.ru/metrika/tag.js", "ym"); + + ym(99267510, "init", { + webvisor:false + clickmap:false, + trackLinks:false, + accurateTrackBounce:false + });`; + ref.current?.appendChild(s); + }, [areAnalyticsEnabled]); + + if (!areAnalyticsEnabled) { + return null; + } + + return
; +} diff --git a/lib/static/new-ui/components/SettingsPanel/index.tsx b/lib/static/new-ui/components/SettingsPanel/index.tsx index e9bfdf3f7..9f2770464 100644 --- a/lib/static/new-ui/components/SettingsPanel/index.tsx +++ b/lib/static/new-ui/components/SettingsPanel/index.tsx @@ -10,9 +10,11 @@ import useLocalStorage from '@/static/hooks/useLocalStorage'; import styles from './index.module.css'; import {AsidePanel} from '@/static/new-ui/components/AsidePanel'; import {PanelSection} from '@/static/new-ui/components/PanelSection'; +import {useAnalytics} from '@/static/new-ui/hooks/useAnalytics'; export function SettingsPanel(): ReactNode { const dispatch = useDispatch(); + const analytics = useAnalytics(); const baseHost = useSelector(state => state.view.baseHost); const [, setUiMode] = useLocalStorage(LocalStorageKey.UIMode, UiMode.New); @@ -22,6 +24,7 @@ export function SettingsPanel(): ReactNode { }; const onOldUiButtonClick = (): void => { + analytics?.trackFeatureUsage({featureName: 'Switch to old UI'}); setUiMode(UiMode.Old); window.location.pathname = window.location.pathname.replace(/\/new-ui(\.html)?$/, (_match, ending) => ending ? '/index.html' : '/'); diff --git a/lib/static/new-ui/features/suites/components/TreeActionsToolbar/index.tsx b/lib/static/new-ui/features/suites/components/TreeActionsToolbar/index.tsx index 96d37213b..e2c0b30d0 100644 --- a/lib/static/new-ui/features/suites/components/TreeActionsToolbar/index.tsx +++ b/lib/static/new-ui/features/suites/components/TreeActionsToolbar/index.tsx @@ -51,6 +51,8 @@ interface TreeActionsToolbarProps { onHighlightCurrentTest?: () => void; } +const ANALYTICS_PREFIX = 'Tree actions toolbar:'; + export function TreeActionsToolbar(props: TreeActionsToolbarProps): ReactNode { const dispatch = useDispatch(); const analytics = useAnalytics(); @@ -122,6 +124,7 @@ export function TreeActionsToolbar(props: TreeActionsToolbarProps): ReactNode { }; const handleRun = (): void => { + analytics?.trackFeatureUsage({featureName: `${ANALYTICS_PREFIX} run tests`}); if (isSelectedAtLeastOne) { dispatch(thunkRunTests({tests: selectedTests})); } else { @@ -137,6 +140,7 @@ export function TreeActionsToolbar(props: TreeActionsToolbarProps): ReactNode { const acceptableImageIds = activeImages .filter(image => isScreenRevertable({image, gui: isGuiMode, isLastResult: true, isStaticImageAccepterEnabled})) .map(image => image.id); + analytics?.trackFeatureUsage({featureName: `${ANALYTICS_PREFIX} revert screenshots`}); if (isStaticImageAccepterEnabled) { dispatch(staticAccepterUnstageScreenshot(acceptableImageIds)); @@ -149,6 +153,7 @@ export function TreeActionsToolbar(props: TreeActionsToolbarProps): ReactNode { const acceptableImageIds = activeImages .filter(image => isAcceptable(image)) .map(image => image.id); + analytics?.trackFeatureUsage({featureName: `${ANALYTICS_PREFIX} accept screenshots`}); analytics?.trackScreenshotsAccept({acceptedImagesCount: acceptableImageIds.length}); if (isStaticImageAccepterEnabled) { @@ -159,7 +164,9 @@ export function TreeActionsToolbar(props: TreeActionsToolbarProps): ReactNode { }; const handleToggleTreeView = (): void => { - dispatch(setTreeViewMode({treeViewMode: treeViewMode === TreeViewMode.Tree ? TreeViewMode.List : TreeViewMode.Tree})); + const newTreeViewMode = treeViewMode === TreeViewMode.Tree ? TreeViewMode.List : TreeViewMode.Tree; + analytics?.trackFeatureUsage({featureName: `${ANALYTICS_PREFIX} change tree view mode`, treeViewMode: newTreeViewMode}); + dispatch(setTreeViewMode({treeViewMode: newTreeViewMode})); }; const selectedOrVisible = isSelectedAtLeastOne ? 'selected' : 'visible'; diff --git a/lib/static/new-ui/types/store.ts b/lib/static/new-ui/types/store.ts index 61ddf3c7a..e9e09bfb8 100644 --- a/lib/static/new-ui/types/store.ts +++ b/lib/static/new-ui/types/store.ts @@ -233,8 +233,8 @@ export interface SortByExpression { } export enum TreeViewMode { - Tree, - List + Tree = 'tree', + List = 'list' } export interface State { diff --git a/lib/static/new-ui/utils/analytics.ts b/lib/static/new-ui/utils/analytics.ts index 8e11dfd83..377ac0fab 100644 --- a/lib/static/new-ui/utils/analytics.ts +++ b/lib/static/new-ui/utils/analytics.ts @@ -8,10 +8,8 @@ declare global { export const getAreAnalyticsEnabled = (): boolean => { const metrikaConfig = (window.data || {}).config?.yandexMetrika; - const areAnalyticsEnabled = metrikaConfig?.enabled && metrikaConfig?.counterNumber; - const isYaMetrikaAvailable = window.ym && typeof window.ym === 'function'; - return Boolean(areAnalyticsEnabled && isYaMetrikaAvailable); + return Boolean(metrikaConfig?.enabled && metrikaConfig?.counterNumber); }; export const getCounterId = (): number => { diff --git a/package-lock.json b/package-lock.json index 4662b48bf..f7e88db67 100644 --- a/package-lock.json +++ b/package-lock.json @@ -6,7 +6,7 @@ "packages": { "": { "name": "html-reporter", - "version": "10.12.1", + "version": "10.13.0", "license": "MIT", "workspaces": [ "test/func/fixtures/*", @@ -30,7 +30,7 @@ "fast-glob": "^3.2.12", "filesize": "^8.0.6", "fs-extra": "^7.0.1", - "gemini-configparser": "^1.0.0", + "gemini-configparser": "1.4.2", "http-codes": "1.0.0", "image-size": "^1.0.2", "inquirer": "^8.2.0", @@ -6011,6 +6011,10 @@ "ajv": "^8.8.2" } }, + "node_modules/analytics": { + "resolved": "test/func/fixtures/analytics", + "link": true + }, "node_modules/ansi-colors": { "version": "3.2.4", "resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-3.2.4.tgz", @@ -13827,9 +13831,9 @@ } }, "node_modules/gemini-configparser": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.1.tgz", - "integrity": "sha512-vfaj/nMgyHrcruEyk9BApLLqWusuQbdH0+awSTCrpTMam1XoM1NDYJEz/iEW7NZjUEl+Bh/tYM9lLPlvwCi1kA==", + "version": "1.4.2", + "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.2.tgz", + "integrity": "sha512-UgCSMUdWpordb1UgbUuJCMCv8fqdkOhu23k8QyaTdUzp4urREwjMCzCRBJtFR7B/sPtBxsJSef/b3CZJNUuBvw==", "dependencies": { "lodash": "^4.17.4" } @@ -25336,6 +25340,15 @@ "universalify": "^0.1.0" } }, + "node_modules/testplane/node_modules/gemini-configparser": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.1.tgz", + "integrity": "sha512-vfaj/nMgyHrcruEyk9BApLLqWusuQbdH0+awSTCrpTMam1XoM1NDYJEz/iEW7NZjUEl+Bh/tYM9lLPlvwCi1kA==", + "dev": true, + "dependencies": { + "lodash": "^4.17.4" + } + }, "node_modules/testplane/node_modules/is-fullwidth-code-point": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz", @@ -29210,6 +29223,9 @@ "safe-buffer": "~5.2.0" } }, + "test/func/fixtures/analytics": { + "version": "0.0.0" + }, "test/func/fixtures/hermione": { "name": "hermione-fixture-report", "version": "0.0.0", @@ -33473,6 +33489,9 @@ "fast-deep-equal": "^3.1.3" } }, + "analytics": { + "version": "file:test/func/fixtures/analytics" + }, "ansi-colors": { "version": "3.2.4", "resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-3.2.4.tgz", @@ -39483,9 +39502,9 @@ } }, "gemini-configparser": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.1.tgz", - "integrity": "sha512-vfaj/nMgyHrcruEyk9BApLLqWusuQbdH0+awSTCrpTMam1XoM1NDYJEz/iEW7NZjUEl+Bh/tYM9lLPlvwCi1kA==", + "version": "1.4.2", + "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.2.tgz", + "integrity": "sha512-UgCSMUdWpordb1UgbUuJCMCv8fqdkOhu23k8QyaTdUzp4urREwjMCzCRBJtFR7B/sPtBxsJSef/b3CZJNUuBvw==", "requires": { "lodash": "^4.17.4" } @@ -48315,6 +48334,15 @@ "universalify": "^0.1.0" } }, + "gemini-configparser": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/gemini-configparser/-/gemini-configparser-1.4.1.tgz", + "integrity": "sha512-vfaj/nMgyHrcruEyk9BApLLqWusuQbdH0+awSTCrpTMam1XoM1NDYJEz/iEW7NZjUEl+Bh/tYM9lLPlvwCi1kA==", + "dev": true, + "requires": { + "lodash": "^4.17.4" + } + }, "is-fullwidth-code-point": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz", diff --git a/package.json b/package.json index 6bd9263ea..b0e97c2c3 100644 --- a/package.json +++ b/package.json @@ -98,7 +98,7 @@ "fast-glob": "^3.2.12", "filesize": "^8.0.6", "fs-extra": "^7.0.1", - "gemini-configparser": "^1.0.0", + "gemini-configparser": "^1.4.2", "http-codes": "1.0.0", "image-size": "^1.0.2", "inquirer": "^8.2.0", diff --git a/test/func/fixtures/analytics/disabled.testplane.conf.js b/test/func/fixtures/analytics/disabled.testplane.conf.js new file mode 100644 index 000000000..61af1e1dc --- /dev/null +++ b/test/func/fixtures/analytics/disabled.testplane.conf.js @@ -0,0 +1,16 @@ +'use strict'; + +const _ = require('lodash'); + +const {getFixturesConfig} = require('../fixtures.testplane.conf'); + +module.exports = _.merge(getFixturesConfig(__dirname), { + plugins: { + 'html-reporter-tester': { + baseHost: 'https://example.com:123', + yandexMetrika: { + enabled: false + } + } + } +}); diff --git a/test/func/fixtures/analytics/enabled.testplane.conf.js b/test/func/fixtures/analytics/enabled.testplane.conf.js new file mode 100644 index 000000000..06c818694 --- /dev/null +++ b/test/func/fixtures/analytics/enabled.testplane.conf.js @@ -0,0 +1,13 @@ +'use strict'; + +const _ = require('lodash'); + +const {getFixturesConfig} = require('../fixtures.testplane.conf'); + +module.exports = _.merge(getFixturesConfig(__dirname), { + plugins: { + 'html-reporter-tester': { + baseHost: 'https://example.com:123' + } + } +}); diff --git a/test/func/fixtures/analytics/package.json b/test/func/fixtures/analytics/package.json new file mode 100644 index 000000000..4af20d0cc --- /dev/null +++ b/test/func/fixtures/analytics/package.json @@ -0,0 +1,11 @@ +{ + "name": "analytics", + "description": "Test project that generates html-report for testing analytics", + "version": "0.0.0", + "private": true, + "scripts": { + "clean": "rm -rf report", + "generate": "true # This report should be generated with different env vars during tests", + "gui": "npx testplane gui --hostname 0.0.0.0 --port $(../../utils/get-port.js analytics gui)" + } +} diff --git a/test/func/fixtures/analytics/test.testplane.js b/test/func/fixtures/analytics/test.testplane.js new file mode 100644 index 000000000..731eeba6e --- /dev/null +++ b/test/func/fixtures/analytics/test.testplane.js @@ -0,0 +1,3 @@ +it('some test', async () => { + throw new Error('Test should fail'); +}); diff --git a/test/func/tests/.testplane.conf.js b/test/func/tests/.testplane.conf.js index 8077cea91..7961749b2 100644 --- a/test/func/tests/.testplane.conf.js +++ b/test/func/tests/.testplane.conf.js @@ -15,6 +15,7 @@ const serverPort = process.env.SERVER_PORT ?? 8083; const projectUnderTest = process.env.PROJECT_UNDER_TEST; const isRunningGuiTests = projectUnderTest.includes('gui'); +const isRunningAnalyticsTests = projectUnderTest.includes('analytics'); if (!projectUnderTest) { throw 'Project under test was not specified'; } @@ -40,12 +41,15 @@ const config = _.merge(commonConfig, { }, plugins: { files: 'plugins/**/*.testplane.js' + }, + analytics: { + files: 'analytics/**/*.testplane.js' } }, plugins: { 'html-reporter-test-server': { - enabled: !isRunningGuiTests, + enabled: !isRunningGuiTests && !isRunningAnalyticsTests, port: serverPort }, 'html-reporter-tester': { @@ -56,7 +60,7 @@ const config = _.merge(commonConfig, { } }); -if (!isRunningGuiTests) { +if (!isRunningGuiTests && !isRunningAnalyticsTests) { _.set(config.plugins, ['hermione-global-hook', 'beforeEach'], async function({browser}) { await browser.url(this.browser.options.baseUrl); diff --git a/test/func/tests/analytics/index.testplane.js b/test/func/tests/analytics/index.testplane.js new file mode 100644 index 000000000..a13ed9d91 --- /dev/null +++ b/test/func/tests/analytics/index.testplane.js @@ -0,0 +1,120 @@ +const childProcess = require('child_process'); +const path = require('path'); +const {PORTS} = require('../../utils/constants'); + +const projectDir = path.resolve(__dirname, '../../fixtures/analytics'); + +const generateFixtureReport = async (args, env) => { + await new Promise(resolve => { + const proc = childProcess.spawn('npx', ['testplane', ...args], {cwd: projectDir, env: {...process.env, ...env}}); + + proc.on('exit', () => { + resolve(); + }); + }); +}; + +const launchStaticServer = () => { + return new Promise((resolve) => { + const proc = childProcess.spawn('node', [path.resolve(__dirname, '../static-server.js')], { + stdio: ['pipe', 'pipe', 'pipe', 'ipc'], + env: { + ...process.env, + STATIC_DIR: path.resolve(__dirname, '../..'), + PORT: PORTS.analytics.server + } + }); + + proc.on('message', () => { + resolve(proc); + }); + }); +}; + +describe('Analytics', () => { + let server; + + afterEach(() => { + server?.kill(); + }); + + it('should include metrika script by default', async ({browser}) => { + await generateFixtureReport(['-c', 'enabled.testplane.conf.js']); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + const scriptElement = await browser.$('div[data-qa="metrika-script"] script'); + + await expect(scriptElement).toBeExisting(); + }); + + it('should track feature usage when opening info panel', async ({browser}) => { + await generateFixtureReport(['-c', 'enabled.testplane.conf.js']); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + await browser.execute(() => { + window.ym = (...args) => { + if (!window.ym.calls) { + window.ym.calls = []; + } + window.ym.calls.push(args); + }; + }); + + await browser.$('[data-qa="footer-item-info"]').click(); + + const metrikaCalls = await browser.execute(() => { + return window.ym.calls; + }); + + expect(metrikaCalls).toContainEqual([ + 99267510, + 'reachGoal', + 'FEATURE_USAGE', + {featureName: 'Open info panel'} + ]); + }); + + it('should not fail when opening info panel with analytics not available', async ({browser}) => { + await generateFixtureReport(['-c', 'disabled.testplane.conf.js']); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + await browser.$('[data-qa="footer-item-info"]').click(); + + const infoPanelElement = await browser.$('div*=Data sources'); + + await expect(infoPanelElement).toBeExisting(); + }); + + it('should not include metrika script if analytics are disabled in config', async ({browser}) => { + await generateFixtureReport(['-c', 'disabled.testplane.conf.js']); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + const scriptElement = await browser.$('div[data-qa="metrika-script"] script'); + + await expect(scriptElement).not.toBeExisting(); + }); + + it('should not include metrika script if analytics are disabled via gemini-configparser env var', async ({browser}) => { + await generateFixtureReport(['-c', 'enabled.testplane.conf.js'], {'html_reporter_yandex_metrika_enabled': false}); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + const scriptElement = await browser.$('div[data-qa="metrika-script"] script'); + + await expect(scriptElement).not.toBeExisting(); + }); + + it('should not include metrika script if analytics are disabled via NO_ANALYTICS env var', async ({browser}) => { + await generateFixtureReport(['-c', 'enabled.testplane.conf.js'], {'NO_ANALYTICS': true}); + server = await launchStaticServer(); + + await browser.url(browser.options.baseUrl.replace('index.html', 'new-ui.html')); + const scriptElement = await browser.$('div[data-qa="metrika-script"] script'); + + await expect(scriptElement).not.toBeExisting(); + }); +}); diff --git a/test/func/tests/package.json b/test/func/tests/package.json index a8d6af648..5dfbc578e 100644 --- a/test/func/tests/package.json +++ b/test/func/tests/package.json @@ -9,12 +9,14 @@ "gui:playwright": "TOOL=playwright PROJECT_UNDER_TEST=playwright npx testplane --set common gui", "gui:plugins": "TOOL=testplane PROJECT_UNDER_TEST=plugins SERVER_PORT=8084 npx testplane --set plugins gui", "gui:testplane-tinder": "TOOL=testplane PROJECT_UNDER_TEST=testplane-gui SERVER_PORT=8084 npx testplane --set common-tinder gui", + "gui:analytics": "TOOL=testplane PROJECT_UNDER_TEST=analytics SERVER_PORT=8085 npx testplane --set analytics gui", "testplane:testplane-common": "TOOL=testplane PROJECT_UNDER_TEST=testplane SERVER_PORT=8061 npx testplane --set common", "testplane:testplane-eye": "TOOL=testplane PROJECT_UNDER_TEST=testplane-eye SERVER_PORT=8062 npx testplane --set eye", "testplane:testplane-gui": "TOOL=testplane PROJECT_UNDER_TEST=testplane-gui SERVER_PORT=8063 npx testplane --no --set common-gui", "testplane:playwright": "TOOL=playwright PROJECT_UNDER_TEST=playwright SERVER_PORT=8065 npx testplane --set common", "testplane:plugins": "TOOL=testplane PROJECT_UNDER_TEST=plugins SERVER_PORT=8064 npx testplane --set plugins", "testplane:testplane-tinder": "TOOL=testplane PROJECT_UNDER_TEST=testplane-gui SERVER_PORT=8084 npx testplane --set common-tinder", + "testplane:analytics": "TOOL=testplane PROJECT_UNDER_TEST=analytics SERVER_PORT=8085 npx testplane --set analytics", "test": "run-s testplane:*" }, "devDependencies": { diff --git a/test/func/tests/static-server.js b/test/func/tests/static-server.js new file mode 100644 index 000000000..9d3cd1dfb --- /dev/null +++ b/test/func/tests/static-server.js @@ -0,0 +1,17 @@ +const express = require('express'); + +const dir = process.env.STATIC_DIR; +const port = process.env.PORT; +const host = process.env.HOST ?? 'localhost'; + +const app = express(); +app.use(express.static(dir)); +app.listen(port, (err) => { + if (err) { + console.error('Failed to start test server:'); + throw new Error(err); + } + + process.send('Ready'); + console.info(`Server is listening on ${host}:${port}`); +}); diff --git a/test/func/utils/constants.js b/test/func/utils/constants.js index cbeb6b47e..f1c612a50 100644 --- a/test/func/utils/constants.js +++ b/test/func/utils/constants.js @@ -17,6 +17,10 @@ module.exports = { plugins: { server: 8084, gui: 8074 + }, + analytics: { + server: 8085, + gui: 8075 } } }; diff --git a/test/unit/lib/config/index.js b/test/unit/lib/config/index.js index 17370f3b9..df7a8400c 100644 --- a/test/unit/lib/config/index.js +++ b/test/unit/lib/config/index.js @@ -417,31 +417,57 @@ describe('config', () => { }); describe('yandexMetrika', () => { - it('should have default value', () => { - assert.deepEqual(parseConfig({}).yandexMetrika, configDefaults.yandexMetrika); + afterEach(() => { + sinon.sandbox.restore(); }); - describe('counterNumber', () => { - it('should throw error if option is not a null or number', () => { - assert.throws(() => { - parseConfig({ - yandexMetrika: { - counterNumber: 'string' - } - }), - Error, - /option must be number, but got string/; - }); + describe('"enabled" option', () => { + it('should be enabled by default', () => { + assert.deepEqual(parseConfig({}).yandexMetrika.enabled, true); }); it('should set value from config file', () => { const config = parseConfig({ + yandexMetrika: { + enabled: false + } + }); + + assert.equal(config.yandexMetrika.enabled, false); + }); + + it('should respect NO_ANALYTICS env', () => { + process.env['NO_ANALYTICS'] = 'true'; + const config = parseConfig({}); + + assert.equal(config.yandexMetrika.enabled, false); + }); + }); + + describe('counterNumber', () => { + it('should have default value', () => { + assert.deepEqual(parseConfig({}).yandexMetrika.counterNumber, configDefaults.yandexMetrika.counterNumber); + }); + + it('should ignore any value passed by user', () => { + const config = parseConfig({ + yandexMetrika: { + counterNumber: 100500 + } + }); + + assert.equal(config.yandexMetrika.counterNumber, configDefaults.yandexMetrika.counterNumber); + }); + + it('should write deprecation warning', () => { + sinon.sandbox.stub(console, 'warn'); + parseConfig({ yandexMetrika: { counterNumber: 100500 } }); - assert.equal(config.yandexMetrika.counterNumber, 100500); + assert.calledOnceWith(console.warn, sinon.match('"yandexMetrika.counterNumber" option is deprecated')); }); }); });