close tikv#5753, ref tikv#5754

Fix unsafe recovery auto-detect mode by passing detected failed stores to TiKV

Signed-off-by: Connor1996 <[email protected]>

Co-authored-by: Connor1996 <[email protected]>

ref tikv#5755, close tikv#5757, ref tikv#5758

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>

…kv#5787) (tikv#5818)

ref tikv#5786, ref tikv#5787

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

Signed-off-by: bufferflies <[email protected]>

Signed-off-by: bufferflies <[email protected]>
Co-authored-by: bufferflies <[email protected]>

… leave-joint-state (tikv#5789) (tikv#5795)

ref tikv#5568, close tikv#5788, ref tikv#5789

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

…any peers (tikv#5920) (tikv#5923)

ref tikv#4570, close tikv#5909, ref tikv#5920

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

…called (tikv#5966) (tikv#5979)

close tikv#5965, ref tikv#5966

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

close tikv#5973, ref tikv#5974

dashboard: default close telemetry

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>

tikv#6016)

close tikv#5898, ref tikv#5899

The old version of idalloc is a globally unique path, and keyspace needs to use id allocator to allocate keyspaceID, so in this pr the id allocator is changed to a general purpose, but not sync metrics.
tikv#5284

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>

…atter region (tikv#6054) (tikv#6061)

close tikv#6017, ref tikv#6054

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

close tikv#6081

*: upgrade the dependencies

Signed-off-by: nolouch <[email protected]>

ref tikv#5798, close tikv#6069, ref tikv#6080

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: lhy1024 <[email protected]>

ref tikv#5310, ref tikv#6069, ref tikv#6070

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>

…gion is unhealthy (tikv#6128) (tikv#6129)

close tikv#6099, ref tikv#6128

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>

…ikv#6202)

close tikv#6195, ref tikv#6200

1. role can be transfer when their's rule role is same.

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: bufferflies <[email protected]>

Co-authored-by: buffer <[email protected]>
Co-authored-by: bufferflies <[email protected]>

…ikv#6112)

close tikv#6107, ref tikv#6108

fix online recovery timeout mechanism

Signed-off-by: Connor1996 <[email protected]>

Co-authored-by: Connor1996 <[email protected]>

* server: add accelerate-schedule/batch api

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>

* address comments

Signed-off-by: Lloyd-Pottiger <[email protected]>

---------

Signed-off-by: Lloyd-Pottiger <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>

ref tikv#6099, ref tikv#6123

add defer logs the panic reason and stack

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>
Co-authored-by: Hu# <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

… (tikv#6469)

ref tikv#5510, close tikv#6467, ref tikv#6468

tikv#5510 introduced a bug that would cause the store labels to be overwritten wrongly after the store reboot.
This PR fixed this issue.

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: JmPotato <[email protected]>

Co-authored-by: JmPotato <[email protected]>

…d leader intact (tikv#6447) (tikv#6460)

close tikv#6403, ref tikv#6447

server: fix the leader cannot election after pd leader lost while etcd leader intact

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

ref tikv#6403, ref tikv#6409

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

… learner node (tikv#6452) (tikv#6483)

ref tikv#4399, ref tikv#6452, close tikv/tikv#14704

replication_mode: fix the state cannot switch to async while existing learner node
 - skip the learner node when check the stores state

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#6426, ref tikv#6427

allow the `schedule=deny` label can do rule constraints check

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>
Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>
Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

fix pingcap/tidb-dashboard#1515, close tikv#6520

Update tidb-dashboard to the lastest version (v2023.05.08.1)

Signed-off-by: baurine <[email protected]>

…) (tikv#6522)

close tikv#6386, ref tikv#6413

Provide an API to fetch the store resolved_ts

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>
Co-authored-by: Hu# <[email protected]>

) (tikv#6558)

ref tikv#6493, ref tikv#6498

Add an option to halt the cluster scheduling.

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>

close tikv#6564

Update tidb-dashboard to the lastest version (v2023.06.07.1)

Signed-off-by: baurine <[email protected]>

…ikv#6575)

close tikv#6573, ref tikv#6574

rule-checker: fix the too many orphan peers that cannot be removed
- let the health peer can be removed once there exist redundant

Signed-off-by: nolouch <[email protected]>

Co-authored-by: nolouch <[email protected]>

close tikv#6635, ref tikv#6636

Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: lhy1024 <[email protected]>

ref tikv#6690, ref tikv#6709

Fix learner store/ replica being ignored error in auto detect mode.

Signed-off-by: Yang Zhang <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#6649, ref tikv#6650

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: lhy1024 <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…v#6789)

close tikv#6786

/swagger/* return 404

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: bufferflies <[email protected]>

Co-authored-by: buffer <[email protected]>
Co-authored-by: bufferflies <[email protected]>

fix tikv#6672

Signed-off-by: disksing <[email protected]>

…ikv#6843)

close tikv#6559

add logic try to replace unhealthy peer with orphan peer

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>

…ikv#6705)

fix tikv#6883

Signed-off-by: disksing <[email protected]>

…kv#6864)

close tikv#6645

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: lhy1024 <[email protected]>

close tikv#6913

utils: let HTTP client can reuse connections

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: ShuNing <[email protected]>
Co-authored-by: nolouch <[email protected]>

close tikv#6860

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>

close tikv#6926

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#6918

add timeout context for observer tikv config to avoid wait too long

Signed-off-by: bufferflies <[email protected]>

Co-authored-by: bufferflies <[email protected]>
Co-authored-by: buffer <[email protected]>

…6855)

close tikv#6835

Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: lhy1024 <[email protected]>
Co-authored-by: Hu# <[email protected]>

close tikv#6966

1. This PR fixed the statements list export failed issue
2. This PR support config the ngm component timeout param

Signed-off-by: baurine <[email protected]>

…ikv#6948)

ref tikv#6912, close tikv/tikv#15258

TiKV Region merge checks region in operator and in local strictly,
and drops commit merge if there is a mismatch between them.

This change is necessary to prevent flashback from blocking
Region merge. For more details, see tikv/tikv#15258.

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: Neil Shen <[email protected]>

ref tikv#6920

Signed-off-by: husharp <[email protected]>

ref tikv#6556

Signed-off-by: husharp <[email protected]>

ref pingcap/tidb#46540

Signed-off-by: lance6716 <[email protected]>

close tikv#7053

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>

…kv#7027)

close tikv#6962

In past, PD conside peer distribution in the different group influenece by using `TotalCountByStore` , but not include the leader distribution. The max used situation is partition table. 
After this pr, TIDB call scatter api will use same group not different. 

ref:
tikv#3422
pingcap/tidb#46156

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: bufferflies <[email protected]>

Co-authored-by: buffer <[email protected]>
Co-authored-by: bufferflies <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#7008

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Cabinfever_B <[email protected]>

Co-authored-by: Yongbo Jiang <[email protected]>
Co-authored-by: Cabinfever_B <[email protected]>

close tikv#7066

1. security: improve tidb-dashboard login security, encrypt the login password to avoid transport plain text (required for 6.5 version)
2. debug-api: support pagination for ddl history for debug-api
3. keyvisual: use scanRegions instead of fetch all regions (required for 6.5 version)
4. refine execution plan, now it can be showed as table style

Signed-off-by: baurine <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#7137

Signed-off-by: baurine <[email protected]>

…tikv#7087) (tikv#7141)

close tikv#7094

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>
Co-authored-by: Hu# <[email protected]>

…ikv#70… (tikv#7150)

ref tikv#7148

Signed-off-by: Cabinfever_B <[email protected]>

close tikv#6975

Signed-off-by: disksing <[email protected]>

Co-authored-by: disksing <[email protected]>

close tikv#7109

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Cabinfever_B <[email protected]>

Co-authored-by: Yongbo Jiang <[email protected]>
Co-authored-by: Cabinfever_B <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#6897

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…ikv#7202) (tikv#7209)

close tikv#7201

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: disksing <[email protected]>

Co-authored-by: disksing <[email protected]>

close tikv#7221

Signed-off-by: disksing <[email protected]>

Co-authored-by: disksing <[email protected]>

close tikv#7248

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: nolouch <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: nolouch <[email protected]>

…kv#7126)

close tikv#7121

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…ore becomes unavailable (tikv#7249) (tikv#7337)

close tikv#7249

Signed-off-by: lhy1024 <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…) (tikv#7363)

close tikv#6988, close tikv#7016

Signed-off-by: husharp <[email protected]>

Co-authored-by: husharp <[email protected]>
Co-authored-by: Hu# <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…7423)

close tikv#7414

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: Hu# <[email protected]>

)

close tikv#7438

Upgrade Gin version to v1.9.1 to solve some security issues

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: niubell <[email protected]>
Co-authored-by: Ryan Leung <[email protected]>

close tikv#7457

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Yexiang Zhang <[email protected]>

…e checker (tikv#6660) (tikv#6956)

close tikv#6637

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

…e count (tikv#7581) (tikv#7591)

close tikv#7584

The healthy orphan peer should be the last one to be removed only if there are extra peers to keep the high availablility.

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: bufferflies <[email protected]>

Co-authored-by: tongjian <[email protected]>
Co-authored-by: bufferflies <[email protected]>

…r evict-leader (tikv#7674) (tikv#7686)

close tikv#7672

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Cabinfever_B <[email protected]>

Co-authored-by: Yongbo Jiang <[email protected]>
Co-authored-by: Cabinfever_B <[email protected]>

close tikv#7630

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: Ryan Leung <[email protected]>