Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

polkadot-types-from-chain removes imports which are used later in the same file #5765

Open
4 of 10 tasks
atodorov opened this issue Nov 28, 2023 · 4 comments · May be fixed by #5769
Open
4 of 10 tasks

polkadot-types-from-chain removes imports which are used later in the same file #5765

atodorov opened this issue Nov 28, 2023 · 4 comments · May be fixed by #5769
Labels
P4 - Needs Investigation Requires analysis to determine cause or feasibility. Not fully understood, needs research first.

Comments

@atodorov
Copy link

  • I'm submitting a ...
  • Bug report
  • Feature request
  • Support request
  • Other
  • What is the current behavior and expected behavior?

Rebuilding type definitions removes imports which are actually being used, which in turn results to build issues with my package later on.

I am expecting that these imports will not be removed.

  • Please tell us about your environment:

  • Version: @polkadot/typegen": "^10.10.1",, "@polkadot/api": "^10.10.1",

  • Environment: Linux

    • Node.js - v18.18.2
    • Browser
    • Other (limited support for other environments)

  • Language:

    • JavaScript
    • TypeScript (include tsc --version: 5.3.2)
    • Other

** Steps to reproduce **

$ git clone https://github.com/gluwa/creditcoin3
$ cd creditcoin3/
$ cargo build --release
   Compiling creditcoin3-node v3.6.3 (/home/gluwa/creditcoin-3.0/creditcoin3/node)
    Finished release [optimized] target(s) in 53.77s

$ ./target/release/creditcoin3-node --dev --validator --pruning archive  --node-key d182d503b7dd97e7c055f33438c7717145840fd66b2a055284ee8d768241a463             --base-path ./alice-data


$ cd cli/
$ yarn install
$ ./get-metadata.sh

$ git status
On branch testing/csub-862-only-update-typedefs
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   creditcoin.json


$ yarn build:types
yarn run v1.22.19
$ yarn generate:defs && yarn generate:meta
$ ts-node --skip-project node_modules/.bin/polkadot-types-from-defs --package creditcoin3/lib/interfaces --input ./src/lib/interfaces --endpoint ./creditcoin.json
ERROR: Unable to load user definitions: /home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/definitions.ts is not a file
/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/types.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-types.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/lookup.ts
	Generating
	Writing

$ ts-node --skip-project node_modules/.bin/polkadot-types-from-chain --package creditcoin3/lib/interfaces  --endpoint ./creditcoin.json --output ./src/lib/interfaces
Generating from metadata, 106,995 bytes
ERROR: No custom definitions found: /home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/definitions.ts is not a file
ERROR: No lookup definitions found: Unknown file extension ".ts" for /home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/lookup.ts
/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-consts.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-errors.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-events.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-query.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-rpc.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-runtime.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api-tx.ts
	Generating
	Writing

/home/gluwa/creditcoin-3.0/creditcoin3/cli/src/lib/interfaces/augment-api.ts
	Generating
	Writing

Done in 1.99s.


$ yarn format  ### because all files in this repo are explicitly formatted

$$ git diff src/
diff --git a/cli/src/lib/interfaces/augment-api-consts.ts b/cli/src/lib/interfaces/augment-api-consts.ts
index c2418f9..32df384 100644
--- a/cli/src/lib/interfaces/augment-api-consts.ts
+++ b/cli/src/lib/interfaces/augment-api-consts.ts
@@ -8,13 +8,6 @@ import '@polkadot/api-base/types/consts';
 import type { ApiTypes, AugmentedConst } from '@polkadot/api-base/types';
 import type { Vec, u128, u16, u32, u64, u8 } from '@polkadot/types-codec';
 import type { Codec } from '@polkadot/types-codec/types';
-import {
-    FrameSupportPalletId,
-    FrameSystemLimitsBlockLength,
-    FrameSystemLimitsBlockWeights,
-    SpWeightsRuntimeDbWeight,
-    SpVersionRuntimeVersion,
-} from '@polkadot/types/lookup';
 
 export type __AugmentedConst<ApiType extends ApiTypes> = AugmentedConst<ApiType>;
 
diff --git a/cli/src/lib/interfaces/augment-api-events.ts b/cli/src/lib/interfaces/augment-api-events.ts
index 5dac02c..8b4b8c0 100644
--- a/cli/src/lib/interfaces/augment-api-events.ts
+++ b/cli/src/lib/interfaces/augment-api-events.ts
@@ -22,21 +22,6 @@ import type {
 } from '@polkadot/types-codec';
 import type { ITuple } from '@polkadot/types-codec/types';
 import type { AccountId32, H160, H256, Perbill, Permill } from '@polkadot/types/interfaces/runtime';
-import {
-    FrameSupportTokensMiscBalanceStatus,
-    EvmCoreErrorExitReason,
-    EthereumLog,
-    SpRuntimeDispatchError,
-    SpConsensusGrandpaAppPublic,
-    PalletImOnlineSr25519AppSr25519Public,
-    PalletStakingExposure,
-    PalletNominationPoolsCommissionChangeRate,
-    PalletNominationPoolsPoolState,
-    Creditcoin3RuntimeProxyFilter,
-    PalletStakingForcing,
-    PalletStakingValidatorPrefs,
-    FrameSupportDispatchDispatchInfo,
-} from '@polkadot/types/lookup';
 
 export type __AugmentedEvent<ApiType extends ApiTypes> = AugmentedEvent<ApiType>;
 
diff --git a/cli/src/lib/interfaces/augment-api-query.ts b/cli/src/lib/interfaces/augment-api-query.ts
index f7f2097..67bcbeb 100644
--- a/cli/src/lib/interfaces/augment-api-query.ts
+++ b/cli/src/lib/interfaces/augment-api-query.ts
@@ -7,61 +7,10 @@ import '@polkadot/api-base/types/storage';
 
 import type { ApiTypes, AugmentedQuery, QueryableStorageEntry } from '@polkadot/api-base/types';
 import type { Data } from '@polkadot/types';
-import type { Bytes, Option, U256, U8aFixed, Vec, bool, u128, u32, u64 } from '@polkadot/types-codec';
+import type { Bytes, Null, Option, U256, U8aFixed, Vec, bool, u128, u32, u64 } from '@polkadot/types-codec';
 import type { AnyNumber, ITuple } from '@polkadot/types-codec/types';
 import type { AccountId32, H160, H256, Perbill, Percent, Permill } from '@polkadot/types/interfaces/runtime';
 import type { Observable } from '@polkadot/types/types';
-import {
-    SpConsensusBabeAppPublic,
-    SpConsensusBabeBabeEpochConfiguration,
-    SpConsensusBabeDigestsPreDigest,
-    SpConsensusBabeDigestsNextConfigDescriptor,
-    PalletBalancesAccountData,
-    PalletBalancesIdAmount,
-    PalletBalancesBalanceLock,
-    PalletBalancesReserveData,
-    EthereumBlock,
-    EthereumReceiptReceiptV3,
-    FpRpcTransactionStatus,
-    EthereumTransactionTransactionV2,
-    PalletEvmCodeMetadata,
-    PalletFastUnstakeUnstakeRequest,
-    PalletGrandpaStoredPendingChange,
-    PalletGrandpaStoredState,
-    PalletIdentityRegistration,
-    PalletIdentityRegistrarInfo,
-    PalletImOnlineSr25519AppSr25519Public,
-    PalletNominationPoolsBondedPoolInner,
-    PalletNominationPoolsClaimPermission,
-    PalletNominationPoolsPoolMember,
-    PalletNominationPoolsRewardPool,
-    PalletNominationPoolsSubPools,
-    SpStakingOffenceOffenceDetails,
-    PalletProxyAnnouncement,
-    PalletProxyProxyDefinition,
-    SpCoreCryptoKeyTypeId,
-    Creditcoin3RuntimeOpaqueSessionKeys,
-    PalletStakingActiveEraInfo,
-    PalletStakingEraRewardPoints,
-    PalletStakingExposure,
-    PalletStakingValidatorPrefs,
-    PalletStakingForcing,
-    PalletStakingStakingLedger,
-    PalletStakingNominations,
-    PalletStakingRewardDestination,
-    PalletStakingSlashingSlashingSpans,
-    PalletStakingSlashingSpanRecord,
-    PalletStakingUnappliedSlash,
-    FrameSystemAccountInfo,
-    FrameSupportDispatchPerDispatchClassWeight,
-    SpRuntimeDigest,
-    FrameSystemEventRecord,
-    FrameSystemPhase,
-    FrameSystemLastRuntimeUpgradeInfo,
-    PalletTransactionPaymentReleases,
-    PalletBagsListListBag,
-    PalletBagsListListNode,
-} from '@polkadot/types/lookup';
 
 export type __AugmentedQuery<ApiType extends ApiTypes> = AugmentedQuery<ApiType, () => unknown>;
 export type __QueryableStorageEntry<ApiType extends ApiTypes> = QueryableStorageEntry<ApiType>;
@@ -376,6 +325,8 @@ declare module '@polkadot/api-base/types/storage' {
                 [H160, H256]
             > &
                 QueryableStorageEntry<ApiType, [H160, H256]>;
+            suicided: AugmentedQuery<ApiType, (arg: H160 | string | Uint8Array) => Observable<Option<Null>>, [H160]> &
+                QueryableStorageEntry<ApiType, [H160]>;
             /**
              * Generic query
              **/
diff --git a/cli/src/lib/interfaces/augment-api-tx.ts b/cli/src/lib/interfaces/augment-api-tx.ts
index b1480d7..ce6304a 100644
--- a/cli/src/lib/interfaces/augment-api-tx.ts
+++ b/cli/src/lib/interfaces/augment-api-tx.ts
@@ -15,36 +15,6 @@ import type { Data } from '@polkadot/types';
 import type { Bytes, Compact, Option, U256, Vec, bool, u128, u16, u32, u64 } from '@polkadot/types-codec';
 import type { AnyNumber, IMethod, ITuple } from '@polkadot/types-codec/types';
 import type { AccountId32, Call, H160, H256, Perbill, Percent, Permill } from '@polkadot/types/interfaces/runtime';
-import {
-    SpConsensusBabeDigestsNextConfigDescriptor,
-    SpConsensusSlotsEquivocationProof,
-    SpSessionMembershipProof,
-    EthereumTransactionTransactionV2,
-    SpConsensusGrandpaEquivocationProof,
-    PalletIdentityJudgement,
-    PalletIdentityBitFlags,
-    PalletIdentityIdentityInfo,
-    PalletImOnlineHeartbeat,
-    PalletImOnlineSr25519AppSr25519Signature,
-    PalletNominationPoolsBondExtra,
-    PalletNominationPoolsClaimPermission,
-    PalletNominationPoolsCommissionChangeRate,
-    PalletNominationPoolsConfigOpU128,
-    PalletNominationPoolsConfigOpU32,
-    PalletNominationPoolsConfigOpPerbill,
-    PalletNominationPoolsPoolState,
-    PalletNominationPoolsConfigOpAccountId32,
-    Creditcoin3RuntimeProxyFilter,
-    Creditcoin3RuntimeOpaqueSessionKeys,
-    PalletStakingRewardDestination,
-    PalletStakingPalletConfigOpU128,
-    PalletStakingPalletConfigOpU32,
-    PalletStakingPalletConfigOpPercent,
-    PalletStakingPalletConfigOpPerbill,
-    PalletStakingValidatorPrefs,
-    SpWeightsWeightV2Weight,
-    Creditcoin3RuntimeOriginCaller,
-} from '@polkadot/types/lookup';
 
 export type __AugmentedSubmittable = AugmentedSubmittable<() => unknown>;
 export type __SubmittableExtrinsic<ApiType extends ApiTypes> = SubmittableExtrinsic<ApiType>;
diff --git a/cli/src/lib/interfaces/index.ts b/cli/src/lib/interfaces/index.ts
index 58052a7..7f1de78 100644
--- a/cli/src/lib/interfaces/index.ts
+++ b/cli/src/lib/interfaces/index.ts
@@ -1,4 +1,4 @@
 // Auto-generated via `yarn polkadot-types-from-defs`, do not edit
 /* eslint-disable */
 
-// export * from './types.js';
+export * from './types.js';

build:types is defined as

        "build:types": "yarn generate:defs && yarn generate:meta",
        "generate:defs": "ts-node --skip-project node_modules/.bin/polkadot-types-from-defs --package creditcoin3/lib/interfaces --input ./src/lib/interfaces --endpoint ./creditcoin.json",
        "generate:meta": "ts-node --skip-project node_modules/.bin/polkadot-types-from-chain --package creditcoin3/lib/interfaces  --endpoint ./creditcoin.json --output ./src/lib/interfaces",

and I've identified that the removal of these imports is coming from polkadot-types-from-chain.

gluwa/creditcoin3#95 - bumps versions of Polkadot.js packages and just rebuilds the type definitions and fails because of the error above - see the failing CI job javascript-typecheck / cli for example

gluwa/creditcoin3#99 - adds automatic regeneration of type definitions (will push an automatic commit to GitHub if successfull) and fails because of this very same issue - see the failing CI job integration-test-cli for example (actual diff in CI logs).
@smohan-dw
Copy link

The error occurs only with Node versions 20 and above.

@atodorov
Copy link
Author

The error occurs only with Node versions 20 and above.

This statement doesn't appear to be true!

Locally I am seeing the same problem with Node v18.18.2. Here we see the same problem with v18.18.2. Actual diff & node --version output is in the logs (search for DEBUG).

@smohan-dw do you have a Node version which you know to be working so that I can give it a try, instead of checking every possible version ?

@smohan-dw
Copy link

@smohan-dw do you have a Node version which you know to be working so that I can give it a try, instead of checking every possible version ?

Node v18.18.2 is working fine for me. You can take a look at https://github.com/smohan-dw/cord.js/tree/sm-updates-0x3

@Muhammad-Altabba Muhammad-Altabba linked a pull request Dec 10, 2023 that will close this issue
Open
@Muhammad-Altabba
Copy link

The lookupTypes are missed from the list used to extract imports for the generated types. And the PR #5769 should fix this.
Thanks,

@Muhammad-Altabba Muhammad-Altabba mentioned this issue Dec 18, 2023
Closed
10 tasks
@TarikGul TarikGul added the P4 - Needs Investigation Requires analysis to determine cause or feasibility. Not fully understood, needs research first. label Oct 30, 2024
@TarikGul TarikGul moved this from Issues (Bugs) to P4 - Needs Investigation in Polkadot-js general project board Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P4 - Needs Investigation Requires analysis to determine cause or feasibility. Not fully understood, needs research first.
Projects
Polkadot-js general project board
Status: P4 - Needs Investigation
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: add lookupTypes to typegen imports Muhammad-Altabba/api
4 participants
@atodorov @Muhammad-Altabba @TarikGul @smohan-dw