-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OracleService #326
Merged
Merged
OracleService #326
Changes from 85 commits
Commits
Show all changes
176 commits
Select commit
Hold shift + click to select a range
05f9e74
init
Tommo-L 78944c6
move
Tommo-L c033672
Merge branch 'master' into async-oracle
shargon 6760302
Clean code
shargon 76bee8c
Update src/Oracle/Oracle.cs
shargon 68a2f4d
fix
Tommo-L 9101194
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 1cd3843
Update src/Oracle/Oracle.cs
shargon c31654a
fix oracle
Tommo-L a7b3d5f
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 2d53f13
format
Tommo-L 22edae9
fix
Tommo-L 79630a6
Merge branch 'master' into async-oracle
Tommo-L 2e64c1e
optimize
Tommo-L de131c4
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 19053d2
optimize
Tommo-L 49abda8
fix
Tommo-L 66a5497
format
Tommo-L 379e732
fix
Tommo-L a2b3f5f
add CheckOracleAccount
Tommo-L ffba962
fix
Tommo-L 7aaf41b
resolve
Tommo-L 1af31e5
Update src/Oracle/Protocol/OracleHttpProtocol.cs
shargon d8ec4c8
Update src/Oracle/Oracle.cs
shargon cc230bd
fix
Tommo-L 0aec6db
format
Tommo-L 60b7744
fix
Tommo-L ff2c4ca
Update Oracle.cs
shargon f51d9c0
fix timeout
Tommo-L 3d9efc7
fix timeout2
Tommo-L d91f5ae
optimze
Tommo-L 0d78083
format
Tommo-L 6e41bc4
Apply OracleHttpProtocol.cs
Tommo-L 91f693c
optimize
Tommo-L 6d9e5ab
fix
Tommo-L 4439d8c
remove
Tommo-L bf70455
fix
Tommo-L 2463e84
optimize
Tommo-L b97ff6c
Merge branch 'master' into async-oracle
Tommo-L 270a2c7
Apply recommmendations
Tommo-L 41d7532
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 11f4d51
fix
Tommo-L 359ca79
Merge branch 'master' into async-oracle
superboyiii db4f0ce
Merge branch 'master' into async-oracle
superboyiii b190152
Merge remote-tracking branch 'upstream/master' into async-oracle
Tommo-L 237ef4b
Merge branch 'master' into async-oracle
superboyiii 6758e36
add backupTx
Tommo-L 1446f3f
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 129bd1d
sort public key
Tommo-L 2b50e71
avoid two txs
Tommo-L 035b894
send backup sign after timeout
Tommo-L db2d56a
exchange arguments of AddResponseTxSign
Tommo-L 9406e88
format
Tommo-L 560d7a1
format
Tommo-L 7b04c4c
add cache
Tommo-L e8185ac
fix
Tommo-L fdc173e
optimize password
Tommo-L 7d8ca71
up error code
Tommo-L 6f39602
format
Tommo-L e25af79
Merge branch 'master' into async-oracle
superboyiii 756161a
fix and add lock
Tommo-L a221d1c
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L bc427a3
fix
Tommo-L 5321fd8
optimize
Tommo-L 4e4c3fc
Merge branch 'master' into async-oracle
superboyiii ba7f5ca
up oracle response code
Tommo-L 93e4f98
Merge branch 'master' into async-oracle
superboyiii 685e1e0
use base64
Tommo-L 81e892c
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 55dc212
Merge branch 'master' into async-oracle
shargon b842450
fix
Tommo-L 9927512
Merge branch 'master' into async-oracle
shargon a67ffac
remove useless library
Tommo-L 5c76239
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L df3b851
Merge branch 'master' into async-oracle
superboyiii d49eb3a
fix
Tommo-L fcd680d
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L d4cf93a
optimize
Tommo-L 4c3ca6a
optimize
Tommo-L 9e1e424
Merge branch 'master' into async-oracle
erikzhang 77dc6a7
rename to OracleService
Tommo-L 62e1e14
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 868f195
Merge branch 'master' into async-oracle
shargon 14d9423
avoid duplicate request
Tommo-L 4a24565
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 4750aee
fix conflicts
Tommo-L 023784b
Replace HashsetCache by ConcurrentDictionary
Tommo-L b16904b
fix
Tommo-L 0e15809
fix conflicts
Tommo-L 27077d5
fix conflicts
Tommo-L f6da41e
fix
Tommo-L 98c931a
fix
Tommo-L bc3d7b7
Merge branch 'master' into async-oracle
Tommo-L b5d0999
clean code
Tommo-L 3f74781
rename Designate to Designation
Tommo-L befb3a3
Update src/Oracle/OracleService.cs
Tommo-L 395a312
fix
Tommo-L 36f44c9
Update projects
erikzhang 047482e
Rename
erikzhang 66df414
optimize code
Tommo-L fff4f88
static AllowPrivateHost/Timeout
Tommo-L 9929856
Update src/OracleService/Protocols/OracleHttpProtocol.cs
Tommo-L 3c1d6ad
apply erik's suggestion
Tommo-L a19940c
fix
Tommo-L 55c770d
fix
Tommo-L 84634ab
fix 2/2
Tommo-L f18a4da
Merge branch 'master' into async-oracle
Tommo-L 348b184
Update OracleService.cs
erikzhang 9a01b54
CI01118
erikzhang 2c630c0
Merge branch 'master' into async-oracle
erikzhang b6e6bd4
add Process
Tommo-L 074620e
Optimize
erikzhang 01fbc3f
Remove unused parameter
erikzhang f7dda23
readonly
erikzhang ff63668
Use System.Threading.Timer
erikzhang 8a8cb80
Rename
erikzhang fb0dbed
Update OracleService.cs
erikzhang fb1cff7
Update OracleService.cs
erikzhang 5d85f8e
Rename
erikzhang 92a7fa0
Add Settings.cs
erikzhang 919df58
Move IsInternal to Helper.cs
erikzhang 42105bf
Update OracleService.cs
erikzhang dd2798d
Remove Utility alias
erikzhang f88a750
Use a single HttpClient
erikzhang 7b5817f
Use HttpCompletionOption.ResponseContentRead
erikzhang 478ce59
private class
erikzhang a9a8c49
Simplify OracleService.cs
erikzhang f33c969
Use Interlocked
erikzhang 9b5cd31
fix
Tommo-L 05524e4
Using lock in OnTimer method
Tommo-L 4422506
Replace lock by concurrentdictionary
Tommo-L cd058c5
Merge branch 'master' into async-oracle
erikzhang 6bc6b46
Optimize
erikzhang 253bc72
Simplify
erikzhang 30aa901
async
erikzhang 7883937
Prevent reentry
erikzhang ce78551
Optimize
erikzhang 6572a85
private
erikzhang 2c82d96
Prevent multiple starts
erikzhang a6bf504
fix
erikzhang f63a81a
fix
erikzhang dfbd954
Optimize CheckOracleAccount
erikzhang 878b3ba
Fix OnStop
erikzhang 1eba58d
Reorder methods
erikzhang 3d8a9da
Optimize CreateResponseTx
erikzhang f7e1505
fix
Tommo-L 470254f
Remove WithTimeout
erikzhang a552748
Merge branch 'master' into async-oracle
shargon 294e620
Add check
erikzhang eaed2ee
add oracle ut
Tommo-L fab735c
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 16185cc
Delete Neo.Plugins.OracleService.Tests.csproj
Tommo-L cd98a45
Delete OracleServiceTest.cs
Tommo-L 4330353
Merge branch 'master' into async-oracle
shargon ca1368c
add try-catch when call Filter
Tommo-L ed20cac
Merge branch 'async-oracle' of https://github.com/Tommo-L/neo-modules…
Tommo-L 4ba236d
fix and add UT
Tommo-L ea272c7
Remove empty line
erikzhang b173638
Merge branch 'master' into async-oracle
superboyiii 973ca66
Merge branch 'master' into async-oracle
erikzhang f13d895
Merge branch 'master' into async-oracle
shargon b9ffecc
Fix OnPersist
shargon df70225
Optimize iteration
shargon 10e7c2b
more optimize
erikzhang 89a16ba
fix ut
Tommo-L 883f779
fix
Tommo-L 6803733
Merge branch 'master' into async-oracle
erikzhang e88be60
fix conflicts
Tommo-L 3139d80
Optimize
erikzhang 12ef99c
use random
Tommo-L e51a1a3
Json un-indented
shargon 31c0d98
Replace nonce for deterministic value
shargon f1d5e67
Replace nonce
shargon 15e4c49
Add UT for filters
shargon c4ec4fe
Merge branch 'master' into async-oracle
Tommo-L 0acf93b
Merge branch 'master' into async-oracle
Tommo-L File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<Version>3.0.0-preview2</Version> | ||
<TargetFramework>netstandard2.1</TargetFramework> | ||
<RootNamespace>Neo.Plugins</RootNamespace> | ||
</PropertyGroup> | ||
|
||
<ItemGroup> | ||
<PackageReference Include="Neo.ConsoleService" Version="1.0.0" /> | ||
</ItemGroup> | ||
|
||
<ItemGroup> | ||
<None Update="OracleService\config.json"> | ||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> | ||
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory> | ||
</None> | ||
</ItemGroup> | ||
|
||
<ItemGroup> | ||
<ProjectReference Include="..\RpcServer\RpcServer.csproj" /> | ||
</ItemGroup> | ||
|
||
</Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{ | ||
"PluginConfiguration": { | ||
"Nodes": [], | ||
"Wallet": "", | ||
"MaxTaskTimeout": 432000000, | ||
"HttpTimeout": 5000, | ||
"AllowPrivateHost": false | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
namespace Neo.Plugins | ||
{ | ||
interface IOracleProtocol | ||
{ | ||
string Request(string url); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
using System; | ||
using System.Diagnostics; | ||
using System.IO; | ||
using System.Linq; | ||
using System.Net; | ||
using System.Net.Http; | ||
using System.Threading.Tasks; | ||
|
||
namespace Neo.Plugins | ||
{ | ||
public class OracleHttpProtocol : IOracleProtocol | ||
{ | ||
public int Timeout { get; set; } = 5000; | ||
public bool AllowPrivateHost { get; set; } = false; | ||
public readonly string[] AllowedFormats = new string[] { "application/json" }; | ||
|
||
public string Request(string url) | ||
{ | ||
Utility.Log(nameof(OracleHttpProtocol), LogLevel.Debug, $"Request: {url}"); | ||
|
||
if (!Uri.TryCreate(url, UriKind.Absolute, out var uri)) throw new InvalidOperationException("UrlError"); | ||
if (!AllowPrivateHost && IsInternal(Dns.GetHostEntry(uri.Host))) throw new InvalidOperationException("Access to private host is not allowed"); | ||
|
||
using var handler = new HttpClientHandler(); | ||
using var client = new HttpClient(handler); | ||
client.DefaultRequestHeaders.Add("Accept", string.Join(",", AllowedFormats)); | ||
|
||
Task<HttpResponseMessage> result = client.GetAsync(uri); | ||
Stopwatch sw = new Stopwatch(); | ||
sw.Start(); | ||
if (!result.Wait(Timeout)) throw new TimeoutException("Timeout"); | ||
if (result.Result.StatusCode == HttpStatusCode.NotFound) throw new FileNotFoundException($"{url} is not found"); | ||
if (!result.Result.IsSuccessStatusCode) throw new InvalidOperationException("Response error"); | ||
if (!AllowedFormats.Contains(result.Result.Content.Headers.ContentType.MediaType)) throw new InvalidOperationException("ContentType it's not allowed"); | ||
sw.Stop(); | ||
var taskRet = result.Result.Content.ReadAsStringAsync(); | ||
if (Timeout <= sw.ElapsedMilliseconds || !taskRet.Wait(Timeout - (int)sw.ElapsedMilliseconds)) throw new TimeoutException("Timeout"); | ||
return taskRet.Result; | ||
} | ||
|
||
internal static bool IsInternal(IPHostEntry entry) | ||
{ | ||
foreach (var ip in entry.AddressList) | ||
if (IsInternal(ip)) | ||
return true; | ||
|
||
return false; | ||
} | ||
|
||
/// <summary> | ||
/// ::1 - IPv6 loopback | ||
/// 10.0.0.0 - 10.255.255.255 (10/8 prefix) | ||
/// 127.0.0.0 - 127.255.255.255 (127/8 prefix) | ||
/// 172.16.0.0 - 172.31.255.255 (172.16/12 prefix) | ||
/// 192.168.0.0 - 192.168.255.255 (192.168/16 prefix) | ||
/// </summary> | ||
/// <param name="ipAddress">Address</param> | ||
/// <returns>True if it was an internal address</returns> | ||
internal static bool IsInternal(IPAddress ipAddress) | ||
{ | ||
if (IPAddress.IsLoopback(ipAddress)) return true; | ||
if (IPAddress.Broadcast.Equals(ipAddress)) return true; | ||
if (IPAddress.Any.Equals(ipAddress)) return true; | ||
if (IPAddress.IPv6Any.Equals(ipAddress)) return true; | ||
if (IPAddress.IPv6Loopback.Equals(ipAddress)) return true; | ||
|
||
var ip = ipAddress.GetAddressBytes(); | ||
switch (ip[0]) | ||
{ | ||
case 10: | ||
case 127: return true; | ||
case 172: return ip[1] >= 16 && ip[1] < 32; | ||
case 192: return ip[1] == 168; | ||
default: return false; | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can improve this code in a future PR, but there are better ways to handle HttpClient timeouts: https://thomaslevesque.com/2018/02/25/better-timeout-handling-with-httpclient/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we also need to consider the parsing time, then we need to record the time consumed by the request.