Co-authored-by: KAI <[email protected]>
Co-authored-by: Felix Lange <[email protected]>

rm error when marshal block to rpc type allen

This PR adds server-side limits for JSON-RPC batch requests. Before this change, batches
were limited only by processing time. The server would pick calls from the batch and
answer them until the response timeout occurred, then stop processing the remaining batch
items.

Here, we are adding two additional limits which can be configured:

- the 'item limit': batches can have at most N items
- the 'response size limit': batches can contain at most X response bytes

These limits are optional in package rpc. In Geth, we set a default limit of 1000 items
and 25MB response size.

When a batch goes over the limit, an error response is returned to the client. However,
doing this correctly isn't always possible. In JSON-RPC, only method calls with a valid
`id` can be responded to. Since batches may also contain non-call messages or
notifications, the best effort thing we can do to report an error with the batch itself is
reporting the limit violation as an error for the first method call in the batch. If a batch is
too large, but contains only notifications and responses, the error will be reported with
a null `id`.

The RPC client was also changed so it can deal with errors resulting from too large
batches. An older client connected to the server code in this PR could get stuck
until the request timeout occurred when the batch is too large. **Upgrading to a version
of the RPC client containing this change is strongly recommended to avoid timeout issues.**

For some weird reason, when writing the original client implementation, @fjl worked off of
the assumption that responses could be distributed across batches arbitrarily. So for a
batch request containing requests `[A B C]`, the server could respond with `[A B C]` but
also with `[A B] [C]` or even `[A] [B] [C]` and it wouldn't make a difference to the
client.

So in the implementation of BatchCallContext, the client waited for all requests in the
batch individually. If the server didn't respond to some of the requests in the batch, the
client would eventually just time out (if a context was used).

With the addition of batch limits into the server, we anticipate that people will hit this
kind of error way more often. To handle this properly, the client now waits for a single
response batch and expects it to contain all responses to the requests.

---------

Co-authored-by: Felix Lange <[email protected]>
Co-authored-by: Martin Holst Swende <[email protected]>

Co-authored-by: lightclient <[email protected]>

This adds two ways to check for subscription support. First, one can now check
whether the transport method (HTTP/WS/etc.) is capable of subscriptions using
the new Client.SupportsSubscriptions method.

Second, the error returned by Subscribe can now reliably be tested using this
pattern:
    
    sub, err := client.Subscribe(...)
    if errors.Is(err, rpc.ErrNotificationsUnsupported) {
        // no subscription support
    }

---------

Co-authored-by: Felix Lange <[email protected]>

node: Delete the unused error from return parameters of Node.Attach() func

We had to do this workaround because it wasn't possible to export typed arrays from
JS to []byte. This was added in dop251/goja@2352993, so we can use the better way now.

The logs in this function are pulled straight from disk in rawdb.ReadRawReceipts and 
also modified in receipts.DeriveFields, so removing the copy should be fine.

Co-authored-by: Felix Lange <[email protected]>

fix some typos

Signed-off-by: cui fliter <[email protected]>

fix typos

Co-authored-by: john <[email protected]>

* all: move main transaction pool into a subpool

* go.mod: remove superfluous updates

* core/txpool: review fixes, handle txs rejected by all subpools

* core/txpool: typos

… (#27481)

This change ensures Reheap will be called even before the London fork activates.
Since Reheap would otherwise only be called through `SetBaseFee` after London,
the list would just keep growing if the fork was not enabled or not reached yet.

Co-authored-by: Felix Lange <[email protected]>

… (#27477)

floatingRatio is a constant and always non-zero. So there is no need to
check for == 0.

Fixes #27301, a crash that could occur during txpool reorg handling.

Package rpc uses cgo to find the maximum UNIX domain socket path 
length. If exceeded, a warning is printed. This is the only use of cgo in this
package. It seems excessive to depend on cgo just for this warning, so
we now hard-code the usual limit for Linux instead.

---------

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

Also adds Address.Less for sorting use in other packages.

---------

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

* go.mod: update kzg libraries to use big-endian

* go.sum: ran go mod tidy

* core/testdata/precompiles: fix blob verification test

* core/testdata/precompiles: fix blob verification test

This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth 
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.

Variables discarded, included can be defined closer to their usage.

Co-authored-by: Martin Holst Swende <[email protected]>

Also adds Hash.Less method for sorting purposes.

---------

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

his function was added in Go 1.20, but our compatibility target
is Go 1.19.

…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.

The state availability is checked during the creation of a state reader.

-    In hash-based database, if the specified root node does not exist on disk disk, then
    the state reader won't be created and an error will be returned.

-    In path-based database, if the specified state layer is not available, then the
    state reader won't be created and an error will be returned.

This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.

This is likely the culprit behind several data corruption issues, e.g. where data has been
written to the freezer, but the deletion from pebble does not go through due to process
crash.

…e (#27309)

This changes the eth_getProof method implementation to re-encode the requested
storage keys, canonicalizing them in the response. For backwards-compatibility reasons,
go-ethereum accepts non-canonical hex keys. Accepting them is fine, but we should
not mirror invalid inputs into the output.

Closes #27306

---------

Co-authored-by: Martin Holst Swende <[email protected]>
Co-authored-by: Felix Lange <[email protected]>

In all other UDPv4 methods, the deadline is checked first. It seems weird to me that ping is an exception. Deadline comparison is also less resource intensive.

Co-authored-by: Exca-DK <[email protected]>

… (#27476)

Verkle trees store the code inside the trie. This PR changes the interface to pass the code, as well as the dirty flag to tell the trie package if the code is dirty and needs to be updated. This is a no-op for the MPT and the odr trie.

Signed-off-by: jsvisa <[email protected]>

error message should not be capitalized / consistency

* core/vm: move TSTORE,TLOAD to correct opcode nums

* core/vm: cleanup

This error will never happen, but if it ever does happen, we will at least see it.

* trie, core/state: revert error removal in (*state.Trie).Commit

* Gary's nitpick :)

Co-Authored-By:  rjl493456442 <[email protected]>

---------

Co-authored-by: rjl493456442 <[email protected]>

* eth: send big transactions by announce/retrieve only

* Update eth/handler.go

Co-authored-by: Martin Holst Swende <[email protected]>

* eth: remove superfluous bracket

* eth: add whitespace

---------

Co-authored-by: Martin Holst Swende <[email protected]>

* params, trie: add verkle fork management + upgrade go-verkle

* remove the two verkle files

* core, eth, params: add missing function

* Gary's feedback

* remove trie/utils/verkle.go

* add verkle block override

---------

Co-authored-by: Gary Rong <[email protected]>

…nt (#27209)

* core/state, light, les: make signature of ContractCode hash-independent

* push current state for feedback

* les: fix unit test

* core, les, light: fix les unittests

* core/state, trie, les, light: fix state iterator

* core, les: address comments

* les: fix lint

---------

Co-authored-by: Gary Rong <[email protected]>

…7525)

The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.

This removes text parsing in leveldb metrics collection code. All metrics
can now be accessed through the stats API provided by leveldb.

We also add new gauge-typed metrics that count the number of tables at each level.

---------

Co-authored-by: Exca-DK <[email protected]>
Co-authored-by: Gary Rong <[email protected]>
Co-authored-by: Felix Lange <[email protected]>

core (blockchain_test): add chain.Stop() to tests

Co-authored-by: Martin Holst Swende <[email protected]>

This change adds back the 'geth --dev' mode of operation, using a cl-mocker. 

---------

Co-authored-by: Martin Holst Swende <[email protected]>
Co-authored-by: rjl493456442 <[email protected]>
Co-authored-by: lightclient <[email protected]>

This PR adds metrics for p2p dialing, which gives us visibility into the quality of the dial 
candidates  returned by our discovery methods.

Back before #27178 , we spun up a number of ethash verifiers to verify headers. So we also had tests to ensure that we were indeed able to abort verification even if we had multiple workers running.

With PR #27178, we removed the parallelism in verification, and these tests are now failing, since we now just sequentially fire away the results as fast as possible on one routine.

This change removes the (sometimes failing) tests

Implements [EIP 5656](https://eips.ethereum.org/EIPS/eip-5656), MCOPY instruction, and enables it for Cancun. 

---------

Co-authored-by: Martin Holst Swende <[email protected]>

Fixes an issue where waitgroups were used erroneously, which could lead to waitgroup being added to while wait was already invoked.

This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.

This simplifies the code that initializes the discovery a bit, and
adds new flags for enabling/disabling discv4 and discv5 separately.

---------

Co-authored-by: Felix Lange <[email protected]>

This change adds the ability to perform reads from freezer without size limitation. This can be useful in cases where callers are certain that out-of-memory will not happen (e.g. reading only a few elements). 

The previous API was designed to behave both optimally and secure while servicing a request from a peer, whereas this change should _not_ be used when an untrusted peer can influence the query size.

This prevents an issue where the node would attempt to contact the bootstrap
nodes even if they weren't contained in the netrestrict list.

fix function name in comment

Signed-off-by: cui fliter <[email protected]>

* eth: rm redundant type from array, slice, or map

* miner: rm redundant type from array, slice, or map

graphql: fix race in test

internal/ethapi: fast exit if tx notfound

* internal/ethapi: testBackend reuse the same db

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: implment GetTransaction

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: implement GetReceipts

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: insert receipts and setup txlookup

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: add simple success tx receipt test

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: add case create contract

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: add contract call receipt

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: add tx notfound

Signed-off-by: jsvisa <[email protected]>

* internal/ethapi: add dynamic fee testcase

* internal/ethapi: add accessList receipt

* internal/ethapi: no need to insert receipt chain, no error

* internal/ethapi: use HeaderByHash instead

* internal/ethapi: add one more case

---------

Signed-off-by: jsvisa <[email protected]>

The struct logger (or opcode tracer) was missing the return data field even
if this was explicitly enabled by user via `"enableReturnData": true` in the config.
This PR fixes this issue.

Co-authored-by: Sina Mahmoodi <[email protected]>

Co-authored-by: Sina Mahmoodi <[email protected]>

This changes the port mapping procedure such that, when the requested port is unavailable
an alternative port suggested by the router is used instead.

We now also repeatedly request the external IP from the router in order to catch any IP changes.

Co-authored-by: Felix Lange <[email protected]>

It is usually best to set GOMAXPROCS to the number of available CPU cores. However, setting
it like that does not work well when the process is quota-limited to a certain number of CPUs.
The automaxprocs library configures GOMAXPROCS, taking such limits into account.

…7724)

…ode consistency. (#27716)


---------

Co-authored-by: Martin Holst Swende <[email protected]>
Co-authored-by: lightclient <[email protected]>

This updates the reference tests to the latest version and also adds logic
to process EIP-4844 blob transactions into the state transition. We are now
passing most Cancun fork tests.

Co-authored-by: Marius van der Wijden <[email protected]>
Co-authored-by: Felix Lange <[email protected]>

EIP-6780: SELFDESTRUCT only in same transaction

>     SELFDESTRUCT will recover all funds to the caller but not delete the account, except when called in the same transaction as creation

---------

Co-authored-by: Martin Holst Swende <[email protected]>

This change removes PoW header syncing related code from LES and also deletes 
duplicated packages les/catalyst, les/downloader and les/fetcher. These package copies
were created because people wanted to make changes in their eth/ counterparts, but weren't
able to adapt LES code to the API changes.

…736)

This is a spin-out from the EIP-4844 devnet branch, containing just the Engine API modifications
and nothing else. The newPayloadV3 endpoint won't really work in this version, but we need the
data structures for testing so I'd like to get this in early.

Co-authored-by: Marius van der Wijden <[email protected]>