High CPU Usage & Crashes During ASP.NET API Calls

[PapouMansot]

Deployment Type

Self-hosted

Triage priority

N/A

NetBox Version

v4.2.1

Python Version

3.11

Steps to Reproduce

Set up a NetBox instance with virtual machines data
Create an ASP.NET application with the following service code:

public class NetBoxService : INetBoxService
{
private readonly HttpClient _httpClient;
private readonly ILogger _logger;

public NetBoxService(HttpClient httpClient, ILogger<NetBoxService> logger)
{
    _httpClient = httpClient;
    _logger = logger;
}

public async Task<List<VirtualMachine>> GetAllVirtualMachinesAsync()
{
    var allVMs = new List<VirtualMachine>();
    try
    {
        string url = "virtualization/virtual-machines/?limit=20";
        int maxPages = 10;
        int currentPage = 0;
        while (!string.IsNullOrEmpty(url) && currentPage < maxPages)
        {
            currentPage++;
            _logger.LogInformation($"Fetching page {currentPage}: {url}");
            var response = await _httpClient.GetAsync(url);
            if (!response.IsSuccessStatusCode)
            {
                _logger.LogError($"API Error: {response.StatusCode}");
                throw new Exception($"API Error: {response.StatusCode}");
            }
            var content = await response.Content.ReadAsStringAsync();
            var netBoxResponse = JsonConvert.DeserializeObject<NetBoxResponse<VirtualMachine>>(content);
            if (netBoxResponse?.Results != null)
            {
                allVMs.AddRange(netBoxResponse.Results);
            }
            url = netBoxResponse?.Next;
        }
    }
    catch (Exception ex)
    {
        _logger.LogError($"An error occurred: {ex.Message}");
        throw;
    }
    return allVMs;
}

}

Execute API calls to retrieve virtual machines data

Expected Behavior

API calls should complete successfully
CPU usage should remain stable
Service should continue running normally

Observed Behavior

CPU usage spikes to 100%
NetBox container crashes with signal 11 (core dumped)
Container enters restart loop
Error logs show:
2025/01/16 13:18:08 [alert] 7#7 process 174 exited on signal 11 (core dumped)
2025/01/16 13:18:08 [alert] 7#7 sendmsg(10, -1, -1, 1) failed (32: Broken pipe)
Workaround
Downgrading to NetBox 4.1.11 resolves the issue:
services:
netbox:
image: netboxcommunity/netbox:4.1.11
Additional Context

Issue occurs specifically with ASP.NET API calls
Problem is reproducible across multiple deployments
Affects virtual machines endpoint but may impact other endpoints as well

[DanSheps]

Can you identify what process that is part of NetBox itself that is causing this CPU spike?

[PapouMansot]

Even with all logging enabled, I couldn't identify the root cause of the CPU spike within NetBox itself. After the API request, the container crashes without providing any significant error messages or logs. This suggests that the issue might be related to an underlying process in NetBox or an external dependency. Further investigation, such as profiling the application or monitoring the container's resource usage in real-time, might be necessary to pinpoint the problem.

[jeremystretch]

I'm converting this to a discussion as no specific unexpected behavior in NetBox has been identified. Please also note that we cannot accept arbitrary scripts as a means for reproducing issues; you would need to provide specific reproduction steps and explicit API calls.