| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
* dex: remove node table
Node table is not so useful, go back to rely on kademlia
* p2p: fix direct dial still have resolve delay
|
| |
|
|
|
| |
* p2p/discover: bump failure counter only if no nodes were provided
* p2p: more aggressive dial strategy for direct dial
|
| |
|
|
| |
This reverts commit 3ca44e556bd9561b0c6c64c7d3a4f95726c78be8.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
The refactor in 4cd90e02e23ecf2bb11bcb4bba4fea2ae164ef74 introduced
livness checks for nodes. However, the change in
86ec742f975d825f42dd69ebf17b0adaa66542c0 did not properly set the
initial liveness check value for verified node. For verified node we
should set livenessCheck to 1 initially.
Without this change, the node entry will not be properly send to other
nodes and the network would be broken.
|
| | |
|
| |
|
|
|
| |
* p2p: try resolving direct dial when dial fail
* dex: avoid concurrent map read and map write
|
| |
|
|
|
|
| |
* p2p, dex: rework connection management
* dex: refresh our node record periodically
* dex: don't send new record event if no new record
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
Remove custom Dexon validator by adding a new `ValidateWitnessData`
method into the validator interface. This allow us to properly detect
know blocks. This also allow other gdex "light" client to sync
compaction chain. Also, setup a standalone RPC node for handling RPC
reqeusts.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
- Let p2p server support direct connection and group connection.
- Introduce node meta table to maintain IP of all nodes in node set,
in memory and let nodes in the network can sync this table.
- Let peerSet able to manage direct connections to notary set and dkg set.
The mechanism to refresh the network topology when configuration round
change is not done yet.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
AddNotaryPeer adds node to static node set so that server will maintain
the connection with the notary node.
AddNotaryPeer also sets the notaryConn flag to allow the node to always
connect, even if the slot are full.
RemoveNotaryPeer removes node from static, then disconnect and unsets
the notaryConn flag.
|
| |
|
|
|
|
|
|
| |
This resolves a minor issue where neighbors responses containing less
than 16 nodes would bump the failure counter, removing the node. One
situation where this can happen is a private deployment where the total
number of extant nodes is less than 16.
Issue found by @jsying.
|
| |
|
|
| |
(cherry picked from commit f28da4f602fcd17624cf6d40d070253dd6663121)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm/network: DRY out repeated giga comment
I not necessarily agree with the way we wait for event propagation.
But I truly disagree with having duplicated giga comments.
* p2p/simulations: encapsulate Node.Up field so we avoid data races
The Node.Up field was accessed concurrently without "proper" locking.
There was a lock on Network and that was used sometimes to access
the field. Other times the locking was missed and we had
a data race.
For example: https://github.com/ethereum/go-ethereum/pull/18464
The case above was solved, but there were still intermittent/hard to
reproduce races. So let's solve the issue permanently.
resolves: ethersphere/go-ethereum#1146
* p2p/simulations: fix unmarshal of simulations.Node
Making Node.Up field private in 13292ee897e345045fbfab3bda23a77589a271c1
broke TestHTTPNetwork and TestHTTPSnapshot. Because the default
UnmarshalJSON does not handle unexported fields.
Important: The fix is partial and not proper to my taste. But I cut
scope as I think the fix may require a change to the current
serialization format. New ticket:
https://github.com/ethersphere/go-ethereum/issues/1177
* p2p/simulations: Add a sanity test case for Node.Config UnmarshalJSON
* p2p/simulations: revert back to defer Unlock() pattern for Network
It's a good patten to call `defer Unlock()` right after `Lock()` so
(new) error cases won't miss to unlock. Let's get back to that pattern.
The patten was abandoned in 85a79b3ad3c5863f8612d25c246bcfad339f36b7,
while fixing a data race. That data race does not exist anymore,
since the Node.Up field got hidden behind its own lock.
* p2p/simulations: consistent naming for test providers Node.UnmarshalJSON
* p2p/simulations: remove JSON annotation from private fields of Node
As unexported fields are not serialized.
* p2p/simulations: fix deadlock in Network.GetRandomDownNode()
Problem: GetRandomDownNode() locks -> getDownNodeIDs() ->
GetNodes() tries to lock -> deadlock
On Network type, unexported functions must assume that `net.lock`
is already acquired and should not call exported functions which
might try to lock again.
* p2p/simulations: ensure method conformity for Network
Connect* methods were moved to p2p/simulations.Network from
swarm/network/simulation. However these new methods did not follow
the pattern of Network methods, i.e., all exported method locks
the whole Network either for read or write.
* p2p/simulations: fix deadlock during network shutdown
`TestDiscoveryPersistenceSimulationSimAdapter` often got into deadlock.
The execution was stuck on two locks, i.e, `Kademlia.lock` and
`p2p/simulations.Network.lock`. Usually the test got stuck once in each
20 executions with high confidence.
`Kademlia` was stuck in `Kademlia.EachAddr()` and `Network` in
`Network.Stop()`.
Solution: in `Network.Stop()` `net.lock` must be released before
calling `node.Stop()` as stopping a node (somehow - I did not find
the exact code path) causes `Network.InitConn()` to be called from
`Kademlia.SuggestPeer()` and that blocks on `net.lock`.
Related ticket: https://github.com/ethersphere/go-ethereum/issues/1223
* swarm/state: simplify if statement in DBStore.Put()
* p2p/simulations: remove faulty godoc from private function
The comment started with the wrong method name.
The method is simple and self explanatory. Also, it's private.
=> Let's just remove the comment.
(cherry picked from commit 50b872bf05b8644f14b9bea340092ced6968dd59)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm/network/stream: newStreamerTester cleanup only if err is nil
* swarm/network/stream: raise newStreamerTester waitForPeers timeout
* swarm/network/stream: fix data races in GetPeerSubscriptions
* swarm/storage: prevent data race on LDBStore.batchesC
https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461775049
* swarm/network/stream: fix TestGetSubscriptionsRPC data race
https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461768477
* swarm/network/stream: correctly use Simulation.Run callback
https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461783804
* swarm/network: protect addrCountC in Kademlia.AddrCountC function
https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-462273444
* p2p/simulations: fix a deadlock calling getRandomNode with lock
https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-462317407
* swarm/network/stream: terminate disconnect goruotines in tests
* swarm/network/stream: reduce memory consumption when testing data races
* swarm/network/stream: add watchDisconnections helper function
* swarm/network/stream: add concurrent counter for tests
* swarm/network/stream: rename race/norace test files and use const
* swarm/network/stream: remove watchSim and its panic
* swarm/network/stream: pass context in watchDisconnections
* swarm/network/stream: add concurrent safe bool for watchDisconnections
* swarm/storage: fix LDBStore.batchesC data race by not closing it
(cherry picked from commit 3fd6db2bf63ce90232de445c7f33943406a5e634)
|
| |
|
|
| |
(cherry picked from commit 2abeb35d5425d72c2f7fdfe4209f7a94fac52a8e)
|
| |
|
|
| |
(cherry picked from commit bbd120354a8d226b446591eeda9f9462cb9b690a)
|
| |
|
|
| |
(cherry picked from commit 85a79b3ad3c5863f8612d25c246bcfad339f36b7)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This change clears up confusion around the two ways in which nodes
can be added to the table.
When a neighbors packet is received as a reply to findnode, the nodes
contained in the reply are added as 'seen' entries if sufficient space
is available.
When a ping is received and the endpoint verification has taken place,
the remote node is added as a 'verified' entry or moved to the front of
the bucket if present. This also updates the node's IP address and port
if they have changed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change resolves multiple issues around handling of endpoint proofs.
The proof is now done separately for each IP and completing the proof
requires a matching ping hash.
Also remove waitping because it's equivalent to sleep. waitping was
slightly more efficient, but that may cause issues with findnode if
packets are reordered and the remote end sees findnode before pong.
Logging of received packets was hitherto done after handling the packet,
which meant that sent replies were logged before the packet that
generated them. This change splits up packet handling into 'preverify'
and 'handle'. The error from 'preverify' is logged, but 'handle' happens
after the message is logged. This fixes the order. Packet logs now
contain the node ID.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm: completed 1st phase of swap accounting
* swarm, p2p/protocols: added stream pricing
* swarm/network/stream: gofmt simplify stream.go
* swarm: fixed review comments
* swarm: used snapshots for swap tests
* swarm: custom retrieve for swap (less cascaded requests at any one time)
* swarm: addressed PR comments
* swarm: log output formatting
* swarm: removed parallelism in swap tests
* swarm: swap tests simplification
* swarm: removed swap_test.go
* swarm/network/stream: added prefix space for comments
* swarm/network/stream: unit test for prices
* swarm/network/stream: don't hardcode price
* swarm/network/stream: fixed invalid price check
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm/network: fix code inspection problems
- typos
- redundant import alias
* p2p/simulations: fix code inspection problems
- typos
- unused function parameters
- redundant import alias
- code style issue: snake case
* swarm/network: fix unused method parameters inspections
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* p2p/protocols: accounting metrics rpc added (#847)
* p2p/protocols: accounting api documentation added (#847)
* p2p/protocols: accounting api doc updated (#847)
* p2p/protocols: accounting api doc update (#847)
* p2p/protocols: accounting api doc update (#847)
* p2p/protocols: fix file is not gofmted
* fix lint error
* updated comments after review
* add account balance to rpc
* naming changed after review
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* p2p/simulation: WIP minimal snapshot test
* p2p/simulation: Add snapshot create, load and verify to snapshot test
* build: add test tag for tests
* p2p/simulations, build: Revert travis change, build test sym always
* p2p/simulations: Add comments, timeout check on additional events
* p2p/simulation: Add benchmark template for minimal peer protocol init
* p2p/simulations: Remove unused code
* p2p/simulation: Correct timer reset
* p2p/simulations: Put snapshot check events in buffer and call blocking
* p2p/simulations: TestSnapshot fail if Load function returns early
* p2p/simulations: TestSnapshot wait for all connections before returning
* p2p/simulation: Revert to before wait for snap load (5e75594)
* p2p/simulations: add "conns after load" subtest to TestSnapshot
and nudge
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm/network: Hive - do not notify peer if discovery is disabled
* p2p/simulations: validate all connections on loading a snapshot
* p2p/simulations: track all connections in on snapshot loading
* p2p/simulations: add snapshotLoadTimeout variable
* p2p/simulations: ignore control events in snapshot load
* p2p/simulations: simplify event loop synchronization
* p2p/simulations: return already connected error from Load function
* p2p/simulations: log warning on snapshot loading disconnection
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
* Update net.go
more simple
* Update net.go
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* p2p/simulations: Add custom services to simnodes + remove sim down conn objs
* p2p/simulation, swarm/network: Add selective services to discovery sim
* p2p/simulations, swarm/network: Remove useless comments
* p2p/simulations, swarm/network: Clean up mess from rebase
* p2p/simulation: Add sleep to prevent connect flakiness in http test
* p2p/simulations: added concurrent goroutines to prevent sleeps on simulation connect/disconnect
* p2p/simulations, swarm/network/simulations: address pr comments
* reinstated dummy service
* fixed http snapshot test
|
| |
|
|
|
|
| |
* metrics, p2p: add ephemeral registry
* metrics: fix linter issue
|
| | |
|
| | |
|
| |
|
| |
Change the type of the metered connection's id field from string to enode.ID.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* p2p/protocols: introduced protocol accounting
* p2p/protocols: added TestExchange simulation
* p2p/protocols: add accounting simulation
* p2p/protocols: remove unnecessary tests
* p2p/protocols: comments for accounting simulation
* p2p/protocols: addressed PR comments
* p2p/protocols: finalized accounting implementation
* p2p/protocols: removed unused code
* p2p/protocols: addressed @nonsense PR comments
|
| |
|
|
|
|
|
|
|
| |
This change extends the peer metrics collection:
- traces the life-cycle of the peers
- meters the peer traffic separately for every peer
- creates event feed for the peer events
- emits the peer events
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR adds enode.LocalNode and integrates it into the p2p
subsystem. This new object is the keeper of the local node
record. For now, a new version of the record is produced every
time the client restarts. We'll make it smarter to avoid that in
the future.
There are a couple of other changes in this commit: discovery now
waits for all of its goroutines at shutdown and the p2p server
now closes the node database after discovery has shut down. This
fixes a leveldb crash in tests. p2p server startup is faster
because it doesn't need to wait for the external IP query
anymore.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a rare deadlock with the inproc adapter:
- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
which waits for Network.lock.
The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.
Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.
In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.
A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* swarm/network/stream: introduced visualized snapshot sync test
* swarm/network/stream: non-existing hash visualization sim
* swarm/network/stream: fixed retrieval tests; new backend for visualization
* swarm/network/stream: cleanup of visualized_snapshot_sync_sim_test.go
* swarm/network/stream: rebased PR on master
* swarm/network/stream: fixed loop logic in retrieval tests
* swarm/network/stream: fixed iterations for snapshot tests
* swarm/network/stream: address PR comments
* swarm/network/stream: addressed PR comments
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.
Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.
The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.
* p2p/discover: port to p2p/enode
This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:
- Table.Lookup is not available anymore. It used to take a public key
as argument because v4 protocol requires one. Its replacement is
LookupRandom.
- Table.Resolve takes *enode.Node instead of NodeID. This is also for
v4 protocol compatibility because nodes cannot be looked up by ID
alone.
- Types Node and NodeID are gone. Further commits in the series will be
fixes all over the the codebase to deal with those removals.
* p2p: port to p2p/enode and discovery changes
This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.
New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.
* p2p/simulations, p2p/testing: port to p2p/enode
No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:
- testing.ProtocolSession tracks complete nodes, not just their IDs.
- adapters.NodeConfig has a new method to create a complete node.
These changes were needed to make swarm tests work.
Note that the NodeID change makes the code incompatible with old
simulation snapshots.
* whisper/whisperv5, whisper/whisperv6: port to p2p/enode
This port was easy because whisper uses []byte for node IDs and
URL strings in the API.
* eth: port to p2p/enode
Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.
* les: port to p2p/enode
Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.
* node: port to p2p/enode
This change simply replaces discover.Node and discover.NodeID with their
new equivalents.
* swarm/network: port to p2p/enode
Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).
There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.
Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
rpc: Add admin_addTrustedPeer and admin_removeTrustedPeer.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
These RPC calls are analogous to Parity's parity_addReservedPeer and
parity_removeReservedPeer.
They are useful for adjusting the trusted peer set during runtime,
without requiring restarting the server.
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
* all: simplify switches
* silly mistake
|
| |\ \
| | |
| | | |
swarm: client-side MRU signatures ; BMT fixes ; network simulation tests
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* cmd/swarm: minor cli flag text adjustments
* cmd/swarm, swarm/storage, swarm: fix mingw on windows test issues
* cmd/swarm: support for smoke tests on the production swarm cluster
* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion
* changed colour of landing page
* landing page reacts to enter keypress
* swarm/api/http: sticky footer for swarm landing page using flex
* swarm/api/http: sticky footer for error pages and fix for multiple choices
* swarm: propagate ctx to internal apis (#754)
* swarm/simnet: add basic node/service functions
* swarm/netsim: add buckets for global state and kademlia health check
* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...
* swarm, swarm/netsim: adjust SwarmNetworkTest
* swarm/netsim: fix tests
* swarm: added visualization option to sim net redesign
* swarm/netsim: support multiple services per node
* swarm/netsim: remove redundant return statement
* swarm/netsim: add comments
* swarm: shutdown HTTP in Simulation.Close
* swarm: sim HTTP server timeout
* swarm/netsim: add more simulation methods and peer events examples
* swarm/netsim: add WaitKademlia example
* swarm/netsim: fix comments
* swarm/netsim: terminate peer events goroutines on simulation done
* swarm, swarm/netsim: naming updates
* swarm/netsim: return not healthy kademlias on WaitTillHealthy
* swarm: fix WaitTillHealthy call in testSwarmNetwork
* swarm/netsim: allow bucket to have any type for a key
* swarm: Added snapshots to new netsim
* swarm/netsim: add more tests for bucket
* swarm/netsim: move http related things into separate files
* swarm/netsim: add AddNodeWithService option
* swarm/netsim: add more tests and Start* methods
* swarm/netsim: add peer events and kademlia tests
* swarm/netsim: fix some tests flakiness
* swarm/netsim: improve random nodes selection, fix TestStartStop* tests
* swarm/netsim: remove time measurement from TestClose to avoid flakiness
* swarm/netsim: builder pattern for netsim HTTP server (#773)
* swarm/netsim: add connect related tests
* swarm/netsim: add comment for TestPeerEvents
* swarm: rename netsim package to network/simulation
|
| |/ / |
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
* swarm: propagate ctx, enable opentracing
* swarm/tracing: log error when tracing is misconfigured
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* p2p/discover: move bond logic from table to transport
This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.
It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.
The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.
The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.
* p2p/discover: reduce findnode failure counter on success
* p2p/discover: remove unused parameter of loadSeedNodes
* p2p/discover: improve ping-back check and comments
* p2p/discover: add neighbors reply nodes always, not just during init
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:
- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
|
| |
|
|
|
|
| |
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This applies spec changes from ethereum/EIPs#1049 and adds support for
pluggable identity schemes.
Some care has been taken to make the "v4" scheme standalone. It uses
public APIs only and could be moved out of package enr at any time.
A couple of minor changes were needed to make identity schemes work:
- The sequence number is now updated in Set instead of when signing.
- Record is now copy-safe, i.e. calling Set on a shallow copy doesn't
modify the record it was copied from.
|
| |
|
|
| |
Peer.run was discarding the reason for disconnection sent to the disc
channel by Disconnect.
|
| |
|
|
| |
(#16667)
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
The parameter `rand` is unused in `PrivateKey.Decrypt`. Decryption in
the ECIES encryption scheme is deterministic, so randomness isn't
needed.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.
* vendor: change nonsense/go-metrics to ethersphere/go-metrics
* go-metrics: add tests. move ResettingTimer logic from reporter to type.
* all, metrics: pull in metrics package in go-ethereum
* metrics/test: make sure metrics are enabled for tests
* metrics: apply gosimple rules
* metrics/exp, internal/debug: init expvar endpoint when starting pprof server
* internal/debug: tiny comment formatting fix
|
| |
|
|
|
|
| |
* p2p: remove unused code
* p2p: remove unused imports
|
| |\
| |
| | |
p2p/protocols, p2p/testing: protocol abstraction and testing
|
| | | |
|
| | |
| |
| |
| |
| | |
Handle all expectations in ProtocolSession.TestExchanges in any
order that are received.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This change removes a peer information from dialing history
when peer is removed from static list. It allows to force a
server to re-dial concrete peer if it is needed.
In our case we are running geth node on mobile devices, and
it is common for a network connection to flap on mobile.
Almost every time it flaps or network connection is changed
from cellular to wifi peers are disconnected with read
timeout. And usually it takes 30 seconds (default expiration
timeout) to recover connection with static peers after
connectivity is restored.
This change allows us to reconnect with peers almost
immediately and it seems harmless enough.
|
| | |
| |
| |
| |
| |
| |
| | |
I forgot to change the check in udp.go when I changed Table.bond to be
based on lastPong instead of node presence in db. Rename lastPong to
bondTime and add hasBond so it's clearer what this DB key is used for
now.
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks
* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d
* rpc: add wildcard support for rpcallowedhosts + go fmt
* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check
* http, rpc, utils: make vhosts into map, address review concerns
* node: change log messages to use geth standard (not sprintf)
* rpc: fix spelling
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* p2p: add DialRatio for configuration of inbound vs. dialed connections
* p2p: add connection flags to PeerInfo
* p2p/netutil: add SameNet, DistinctNetSet
* p2p/discover: improve revalidation and seeding
This changes node revalidation to be periodic instead of on-demand. This
should prevent issues where dead nodes get stuck in closer buckets
because no other node will ever come along to replace them.
Every 5 seconds (on average), the last node in a random bucket is
checked and moved to the front of the bucket if it is still responding.
If revalidation fails, the last node is replaced by an entry of the
'replacement list' containing recently-seen nodes.
Most close buckets are removed because it's very unlikely we'll ever
encounter a node that would fall into any of those buckets.
Table seeding is also improved: we now require a few minutes of table
membership before considering a node as a potential seed node. This
should make it less likely to store short-lived nodes as potential
seeds.
* p2p/discover: fix nits in UDP transport
We would skip sending neighbors replies if there were fewer than
maxNeighbors results and CheckRelayIP returned an error for the last
one. While here, also resolve a TODO about pong reply tokens.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* p2p/discv5: add query delay, fix node address update logic, retry refresh if empty
* p2p/discv5: remove unnecessary ping before topic query
* p2p/discv5: do not filter local address from topicNodes
* p2p/discv5: remove canQuery()
* p2p/discv5: gofmt
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.
DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* core/types, core/vm, eth, tests: regenerate gencodec files
* Makefile: update devtools target
Install protoc-gen-go and print reminders about npm, solc and protoc.
Also switch to github.com/kevinburke/go-bindata because it's more
maintained.
* contracts/ens: update contracts and regenerate with solidity v0.4.19
The newer upstream version of the FIFSRegistrar contract doesn't set the
resolver anymore. The resolver is now deployed separately.
* contracts/release: regenerate with solidity v0.4.19
* contracts/chequebook: fix fallback and regenerate with solidity v0.4.19
The contract didn't have a fallback function, payments would be rejected
when compiled with newer solidity. References to 'mortal' and 'owned'
use the local file system so we can compile without network access.
* p2p/discv5: regenerate with recent stringer
* cmd/faucet: regenerate
* dashboard: regenerate
* eth/tracers: regenerate
* internal/jsre/deps: regenerate
* dashboard: avoid sed -i because it's not portable
* accounts/usbwallet/internal/trezor: fix go generate warnings
|
| | |
|
| |
|
|
| |
Initial implementation of ENR according to ethereum/EIPs#778
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This commit adds mocker functionality to p2p/simulations. A
mocker allows to starting/stopping of nodes via the HTTP API.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
p2p/simulations: introduce dialBan
- Refactor simulations/network connection getters to support
avoiding simultaneous dials between two peers If two peers dial
simultaneously, the connection will be dropped to help avoid
that, we essentially lock the connection object with a
timestamp which serves as a ban on dialing for a period of time
(dialBanTimeout).
- The connection getter InitConn can be wrapped and passed to the
nodes via adapters.NodeConfig#Reachable field and then used by
the respective services when they initiate connections. This
massively stablise the emerging connectivity when running with
hundreds of nodes bootstrapping a network.
p2p: add Inbound public method to p2p.Peer
p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.
p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
|
| |
|
|
|
|
|
|
|
| |
* build: enable unconvert linter
- fixes #15453
- update code base for failing cases
* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
|
| |
|
| |
Fixes #1024
|
| |
|
|
|
|
| |
* p2p: snappy encoding for devp2p (version bump to 5)
* p2p: remove lazy decompression, enforce 16MB limit
|
| |
|
|
|
|
| |
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
|
| |
|
|
|
|
| |
Using a Timer over Ticker seems to be a lot better, though I cannot fully
account for why that it behaves so (since Ticker should be more bursty, but not
necessarily more active over time, but that may depend on how long window it
uses to decide on when to tick next)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* p2p/discover, p2p/discv5: add marshaling methods to Node
* p2p/netutil: make Netlist decodable from TOML
* common/math: encode nil HexOrDecimal256 as 0x0
* cmd/geth: add --config file flag
* cmd/geth: add missing license header
* eth: prettify Config again, fix tests
* eth: use gasprice.Config instead of duplicating its fields
* eth/gasprice: hide nil default from dumpconfig output
* cmd/geth: hide genesis block in dumpconfig output
* node: make tests compile
* console: fix tests
* cmd/geth: make TOML keys look exactly like Go struct fields
* p2p: use discovery by default
This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.
* cmd/geth: make ethstats URL settable through config file
* cmd/faucet: fix configuration
* cmd/geth: dedup attach tests
* eth: add comment for DefaultConfig
* eth: pass downloader.SyncMode in Config
This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.
* cmd/utils: remove jitvm flags
* cmd/utils: make mutually exclusive flag error prettier
It now reads:
Fatal: flags --dev, --testnet can't be used at the same time
* p2p: fix typo
* node: add DefaultConfig, use it for geth
* mobile: add missing NoDiscovery option
* cmd/utils: drop MakeNode
This exposed a couple of places that needed to be updated to use
node.DefaultConfig.
* node: fix typo
* eth: make fast sync the default mode
* cmd/utils: remove IPCApiFlag (unused)
* node: remove default IPC path
Set it in the frontends instead.
* cmd/geth: add --syncmode
* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive
* cmd/utils: don't enable WS, HTTP when setting addr
* cmd/utils: fix --identity
|
| | |
|
| | |
|
| |
|
|
| |
Also tweak behaviour so colors are only enabled when stderr is a terminal.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
* vendor: add github.com/btcsuite/btcd/btcec
* crypto: add btcec fallback for sign/recover without cgo
This commit adds a non-cgo fallback implementation of secp256k1
operations.
* crypto, core/vm: remove wrappers for sha256, ripemd160
|
| |
|
|
|
| |
go get github.com/gordonklaus/ineffassign
ineffassign .
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
The p2p packages can now be configured to restrict all communication to
a certain subset of IP networks. This feature is meant to be used for
private networks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The discovery DHT contains a number of hosts with LAN and loopback IPs.
These get relayed because some implementations do not perform any checks
on the IP.
go-ethereum already prevented relay in most cases because it verifies
that the host actually exists before adding it to the local table. But
this verification causes other issues. We have received several reports
where people's VPSs got shut down by hosting providers because sending
packets to random LAN hosts is indistinguishable from a slow port scan.
The new check prevents sending random packets to LAN by discarding LAN
IPs sent by Internet hosts (and loopback IPs from LAN and Internet
hosts). The new check also blacklists almost all currently registered
special-purpose networks assigned by IANA to avoid inciting random
responses from services in the LAN.
As another precaution against abuse of the DHT, ports below 1024 are now
considered invalid.
|
| | |
|
| |
|
|
|
|
|
|
| |
The new package contains three things for now:
- IP network list parsing and matching
- The WSAEMSGSIZE workaround, which is duplicated in p2p/discover and
p2p/discv5.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
cmd/utils, node: make datadir reusable for bzzd
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Port mapper auto discovery used to run immediately after parsing the
--nat flag, giving it a slight performance boost. But this is becoming
inconvenient because we create node.Node for all geth operations
including account management and bare chain interaction. Delay
autodiscovery until the first use instead, which avoids any network
interaction until the node is actually started.
|
| |/ |
|
| |\
| |
| | |
node, p2p, internal: Add ability to remove peers via admin interface
|
| | | |
|
| |/ |
|
| |
|
|
| |
This silences a go vet message about copying p2p.Server in package node.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
With the Ethereum Homestead fork is now behind us, we can
assume that everyone runs an EIP-8 capable client.
|
| | |
|
| |\
| |
| | |
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
|
| | |
| |
| |
| | |
As we aren't really using the standarized SHA-3
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
On Windows, UDPConn.ReadFrom returns an error for packets larger
than the receive buffer. The error is not marked temporary, causing
our loop to exit when the first oversized packet arrived. The fix
is to treat this particular error as temporary.
Fixes: #1579, #2087
Updates: #2082
|
| |
|
|
|
| |
The test expected the timeout to fire after a matcher for the response
was added, but the timeout is random and fired sooner sometimes.
|
| |
|
|
|
|
| |
This change makes it possible to add peers without providing their IP
address. The endpoint of the target node is resolved using the discovery
protocol.
|
| |
|
|
|
|
| |
This change simplifies the dial scheduling logic because it
no longer needs to track whether the discovery table has been
bootstrapped.
|
| | |
|
| | |
|
| |
|
|
| |
thanks to Felix Lange (fjl) for help with design & impl
|
| | |
|
| |\
| |
| | |
eth, p2p, rpc/api: polish protocol info gathering
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
| |
The strict matching can get in the way of protocol upgrades.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nodeDB.querySeeds was not safe for concurrent use but could be called
concurrenty on multiple goroutines in the following case:
- the table was empty
- a timed refresh started
- a lookup was started and initiated refresh
These conditions are unlikely to coincide during normal use, but are
much more likely to occur all at once when the user's machine just woke
from sleep. The root cause of the issue is that querySeeds reused the
same leveldb iterator until it was exhausted.
This commit moves the refresh scheduling logic into its own goroutine
(so only one refresh is ever active) and changes querySeeds to not use
a persistent iterator. The seed node selection is now more random and
ignores nodes that have not been contacted in the last 5 days.
|
| | |
|
| |\
| |
| | |
fdtrack: hide message
|
| | |
| |
| |
| | |
This reverts commit 5c949d3b3ba81ea0563575b19a7b148aeac4bf61.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
PR #1621 changed Table locking so the mutex is not held while a
contested node is being pinged. If multiple nodes ping the local node
during this time window, multiple ping packets will be sent to the
contested node. The changes in this commit prevent multiple packets by
tracking whether the node is being replaced.
|
| | | |
|
| |/
|
|
| |
Might solve #1579
|
| |\
| |
| | |
p2p: validate recovered ephemeral pubkey
|
| | | |
|
| | |
| |
| |
| | |
We had the wrong value (12) since forever.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If the timeout fired (even just nanoseconds) before the deadline of the
next pending reply, the timer was not rescheduled. The timer would've
been rescheduled anyway once the next packet was sent, but there were
cases where no next packet could ever be sent due to the locking issue
fixed in the previous commit.
As timing-related bugs go, this issue had been present for a long time
and I could never reproduce it. The test added in this commit did
reproduce the issue on about one out of 15 runs.
|
| | |
| |
| |
| |
| |
| | |
Table.mutex was being held while waiting for a reply packet, which
effectively made many parts of the whole stack block on that packet,
including the net_peerCount RPC call.
|
| | | |
|
| | |
| |
| |
| | |
Not closing the table used to be fine, but now the table has a database.
|
| | |
| |
| |
| |
| | |
Package fdtrack logs statistics about open file descriptors.
This should help identify the source of #1549.
|
| | |
| |
| |
| | |
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
|
| |/
|
|
|
| |
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Lookup calls would spin out of control when network connectivity was
lost. The throttling that was in place only took effect when the table
returned zero results, which doesn't happen very often.
The new throttling should not have a negative impact when the host is
online. Lookups against the network take some time and dials for all
results must complete or hit the cache before a new one is started. This
usually takes longer than four seconds, leaving online lookups
unaffected.
Fixes #1296
|
| | |
|
| |
|
|
|
|
|
| |
As of this commit, we no longer rely on the protocol handler to report
write errors in a timely fashion. When a write fails, shutdown is
initiated immediately and no new writes can start. This will also
prevent new writes from starting after Server.Stop has been called.
|
| |
|
|
| |
rand.Source isn't safe for concurrent use.
|
| | |
|
| | |
|
| |
|
|
|
| |
The previous value of 5 seconds causes timeouts for legitimate messages
if large messages are sent.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
This detects hanging connections sooner. We send a ping every 15s and
other implementation have similar limits.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The most visible change is event-based dialing, which should be an
improvement over the timer-based system that we have at the moment.
The dialer gets a chance to compute new tasks whenever peers change or
dials complete. This is better than checking peers on a timer because
dials happen faster. The dialer can now make more precise decisions
about whom to dial based on the peer set and we can test those
decisions without actually opening any sockets.
Peer management is easier to test because the tests can inject
connections at checkpoints (after enc handshake, after protocol
handshake).
Most of the handshake stuff is now part of the RLPx code. It could be
exported or move to its own package because it is no longer entangled
with Server logic.
|
| | |
|
| |
|
|
|
|
| |
The previous limit was 10MB which is unacceptable for all kinds
of reasons, the most important one being that we don't want to
allow the remote side to make us allocate 10MB at handshake time.
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
p2p: tweak connection limits
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
This should increase the speed a bit because all findnode
results (up to 16) can be verified at the same time.
|
| | |
| |
| |
| |
| |
| | |
The returned reason is currently not used except for the log
message. This change makes the log messages a bit more useful.
The handshake code also returns the remote reason.
|
| | |
| |
| |
| |
| | |
On the test network, we've seen that it becomes harder to connect
if the queues are so short.
|
| | |
| |
| |
| |
| |
| | |
People stil get confused about the messages. This commit changes
the levels so that the only thing printed at the default level (info)
is a successful mapping.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
The test listens for multicast UDP packets on the default interface
because I couldn't get it to work reliably on loopback without massive
changes to goupnp. This means that the test might fail when there is a
UPnP-enabled router attached on that interface. I checked that locally
by looping the test and it passes reliably because the local SSDP server
always responds faster.
|
| |/
|
|
|
|
|
|
|
|
|
|
| |
Concurrent calls to Interface methods on autodisc could return a "not
discovered" error if the discovery did not finish before the call.
autodisc.wait expected the done channel to carry the found Interface
but it was closed instead.
The fix is to use sync.Once for now, which is easier to get right.
And there is a test. Finally.
This will have to change again when we introduce re-discovery.
|
| |
|
|
|
| |
The code assumed that Table.closest always returns at least 13 nodes.
This is not true for small tables (e.g. during bootstrap).
|
| | |
|
