Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HH-199730 try use orjson for log dumps #663

Merged
merged 2 commits into from
Nov 27, 2023
Merged

HH-199730 try use orjson for log dumps #663

merged 2 commits into from
Nov 27, 2023

Conversation

bokshitsky
Copy link
Contributor

@bokshitsky bokshitsky commented Nov 25, 2023

Результат по скорости такой:

async def test_speed_test():
    MDC.init('worker')
    count = 10**5
    fmt = JSONFormatter()
    record = LogRecord(
        "ssss",
        50,
        "sssss",
        14,
        "GET /hh-session/full?host=hh.ru&lang=RU&userAgent=ApplicantHHWidget+(iPhone+SE+2nd+Gen%3B+iOS+16.6%3B+Version/7.41.83594%3Bru.hh.iphone)&originalRequestUri=https://api.hh.ru/me?with_user_statuses%3Dtrue%26host%3Dhh.ru%26locale%3DRU&originalHost=api.hh.ru&httpMethod=GET&geoIpArea=1",
        {},
        None,
    )

    for i in range(2):
        options.log_use_orjson = True
        time_start = perf_counter()
        for i in range(count):
            fmt.format(record)
        print("orjson", perf_counter() - time_start)

        options.log_use_orjson = False
        time_start = perf_counter()
        for i in range(count):
            fmt.format(record)
        print("std", perf_counter() - time_start)

    for i in range(2):
        options.log_use_orjson = False
        time_start = perf_counter()
        for i in range(count):
            fmt.format(record)
        print("std", perf_counter() - time_start)

        options.log_use_orjson = True
        time_start = perf_counter()
        for i in range(count):
            fmt.format(record)
        print("orjson", perf_counter() - time_start)


orjson 0.39401591708883643
std 0.7382588749751449
orjson 0.39261041628196836
std 0.7373545421287417
std 0.7282978328876197
orjson 0.3886946253478527
std 0.724845124874264
orjson 0.3871235833503306

@bokshitsky bokshitsky requested a review from a team as a code owner November 25, 2023 14:06
@@ -30,6 +30,7 @@ class Options:

log_dir: str | None = None
log_level: str = 'info'
log_use_orjson: bool = True
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

типа чтобы не повторить ошибку и если че легко выключить

Copy link

hh-sonar bot commented Nov 25, 2023

SonarQube Quality Gate

Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@hhrelease hhrelease merged commit 0eb8666 into master Nov 27, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants