Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add blobKeepAliveTimeout, queueKeepAliveTimeout, tableKeepAliveTimeout #2454

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
9 changes: 9 additions & 0 deletions ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,15 @@ Blob:
- Fixed issue of download a blob range without header x-ms-range-get-content-md5, should not return content-md5. (issue #2409)
- Fixed issue of list container without include=metadata should not clear container metadata on server. (issue #2416)
- Supported x-ms-copy-source-tag-option in copy blob from Uri. (issue #2398)
- Added blobKeepAliveTimeout option (issue #2053)

Table:

- Added tableKeepAliveTimeout option (issue #2053)

Queue:

- Added queueKeepAliveTimeout option (issue #2053)

## 2024.06 Version 3.31.0

Expand Down
7 changes: 7 additions & 0 deletions README.mcr.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,13 @@ Above command will try to start Azurite image with configurations:

Please refer to this [document](https://github.com/Azure/Azurite/blob/master/README.md) for **More supported parameters** like HTTPS or OAuth.

**Customize HTTP Keep-Alive behavior**
To avoid TCP RST related errors in tests with SDK which not properly handle KeepAlive-Timeout response header, you could increase keep alive timeout, increase values in seconds.

```bash
docker run azurite --blobHost 0.0.0.0 --blobKeepAliveTimeout 300 --queueHost 0.0.0.0 --queueKeepAliveTimeout 300 --tableHost 0.0.0.0 --tableKeepAliveTimeout 300
Slach marked this conversation as resolved.
Show resolved Hide resolved
```

## Documentation

Please refer to this [document](https://github.com/Azure/Azurite/blob/master/README.md).
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,10 +186,13 @@ Following extension configurations are supported:

- `azurite.blobHost` Blob service listening endpoint, by default 127.0.0.1
- `azurite.blobPort` Blob service listening port, by default 10000
- `azurite.blobKeepAliveTimeout` Blob service keep alive timeout in seconds, by default 5
- `azurite.queueHost` Queue service listening endpoint, by default 127.0.0.1
- `azurite.queuePort` Queue service listening port, by default 10001
- `azurite.queueKeepAliveTimeout` Queue service keep alive timeout in seconds, by default 5
- `azurite.tableHost` Table service listening endpoint, by default 127.0.0.1
- `azurite.tablePort` Table service listening port, by default 10002
- `azurite.tableKeepAliveTimeout` Queue service keep alive timeout in seconds, by default 5
- `azurite.location` Workspace location folder path (can be relative or absolute). By default, in the VS Code extension, the currently opened folder is used. If launched from the command line, the current process working directory is the default. Relative paths are resolved relative to the default folder.
- `azurite.silent` Silent mode to disable access log in Visual Studio channel, by default false
- `azurite.debug` Output debug log into Azurite channel, by default false
Expand Down Expand Up @@ -247,14 +250,20 @@ Above command will try to start Azurite image with configurations:

`--blobHost 0.0.0.0` defines blob service listening endpoint to accept requests from host machine.

`--blobKeepAliveTimeout 5` blob service keep alive timeout in seconds

`--queuePort 8888` makes Azurite queue service listen to port 8888, while `-p 8888:8888` redirects requests from host machine's port 8888 to docker instance.

`--queueHost 0.0.0.0` defines queue service listening endpoint to accept requests from host machine.

`--queueKeepAliveTimeout 5` queue service keep alive timeout in seconds

`--tablePort 9999` makes Azurite table service listen to port 9999, while `-p 9999:9999` redirects requests from host machine's port 9999 to docker instance.

`--tableHost 0.0.0.0` defines table service listening endpoint to accept requests from host machine.

`--tableKeepAliveTimeout 5` table service keep alive timeout in seconds

`--loose` enables loose mode which ignore unsupported headers and parameters.

`--skipApiVersionCheck` skip the request API version check.
Expand Down
15 changes: 15 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,11 @@
"default": 10000,
"description": "Blob service listening port, by default 10000"
},
"azurite.blobKeepAliveTimeout": {
"type": "number",
"default": 5,
"description": "Blob service keep alive timeout in seconds, by default 5"
},
"azurite.queueHost": {
"type": "string",
"default": "127.0.0.1",
Expand All @@ -217,6 +222,11 @@
"default": 10001,
"description": "Queue service listening port, by default 10001"
},
"azurite.queueKeepAliveTimeout": {
"type": "number",
"default": 5,
"description": "Queue service keep alive timeout in seconds, by default 5"
},
"azurite.tableHost": {
"type": "string",
"default": "127.0.0.1",
Expand All @@ -227,6 +237,11 @@
"default": 10002,
"description": "Table service listening port, by default 10002"
},
"azurite.tableKeepAliveTimeout": {
"type": "number",
"default": 5,
"description": "Table service keep alive timeout in seconds, by default 5"
},
"azurite.skipApiVersionCheck": {
"type": "boolean",
"default": false,
Expand Down
2 changes: 2 additions & 0 deletions src/azurite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ async function main() {
const queueConfig = new QueueConfiguration(
env.queueHost(),
env.queuePort(),
env.queueKeepAliveTimeout(),
join(location, DEFAULT_QUEUE_LOKI_DB_PATH),
join(location, DEFAULT_QUEUE_EXTENT_LOKI_DB_PATH),
DEFAULT_QUEUE_PERSISTENCE_ARRAY,
Expand All @@ -106,6 +107,7 @@ async function main() {
const tableConfig = new TableConfiguration(
env.tableHost(),
env.tablePort(),
env.tableKeepAliveTimeout(),
join(location, DEFAULT_TABLE_LOKI_DB_PATH),
env.debug() !== undefined,
!env.silent(),
Expand Down
5 changes: 4 additions & 1 deletion src/blob/BlobConfiguration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ import {
DEFAULT_BLOB_PERSISTENCE_ARRAY,
DEFAULT_BLOB_SERVER_HOST_NAME,
DEFAULT_ENABLE_ACCESS_LOG,
DEFAULT_ENABLE_DEBUG_LOG
DEFAULT_ENABLE_DEBUG_LOG,
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT
} from "./utils/constants";

/**
Expand All @@ -27,6 +28,7 @@ export default class BlobConfiguration extends ConfigurationBase {
public constructor(
host: string = DEFAULT_BLOB_SERVER_HOST_NAME,
port: number = DEFAULT_BLOB_LISTENING_PORT,
keepAliveTimeout: number = DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT,
public readonly metadataDBPath: string = DEFAULT_BLOB_LOKI_DB_PATH,
public readonly extentDBPath: string = DEFAULT_BLOB_EXTENT_LOKI_DB_PATH,
public readonly persistencePathArray: StoreDestinationArray = DEFAULT_BLOB_PERSISTENCE_ARRAY,
Expand All @@ -47,6 +49,7 @@ export default class BlobConfiguration extends ConfigurationBase {
super(
host,
port,
keepAliveTimeout,
enableAccessLog,
accessLogWriteStream,
enableDebugLog,
Expand Down
12 changes: 11 additions & 1 deletion src/blob/BlobEnvironment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ import { dirname } from "path";
import IBlobEnvironment from "./IBlobEnvironment";
import {
DEFAULT_BLOB_LISTENING_PORT,
DEFAULT_BLOB_SERVER_HOST_NAME
DEFAULT_BLOB_SERVER_HOST_NAME,
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT
} from "./utils/constants";

if (!(args as any).config.name) {
Expand All @@ -20,6 +21,11 @@ if (!(args as any).config.name) {
"Optional. Customize listening port for blob",
DEFAULT_BLOB_LISTENING_PORT
)
.option(
["", "blobKeepAliveTimeout"],
"Optional. Customize http keep alive timeout for blob",
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT,
)
.option(
["l", "location"],
"Optional. Use an existing folder as workspace path, default is current working directory",
Expand Down Expand Up @@ -75,6 +81,10 @@ export default class BlobEnvironment implements IBlobEnvironment {
return this.flags.blobPort;
}

public blobKeepAliveTimeout(): number | undefined {
return this.flags.keepAliveTimeout;
}

public async location(): Promise<string> {
const location = this.flags.location || process.cwd();
await ensureDir(location);
Expand Down
2 changes: 2 additions & 0 deletions src/blob/BlobServerFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ export class BlobServerFactory {
const config = new SqlBlobConfiguration(
env.blobHost(),
env.blobPort(),
env.blobKeepAliveTimeout(),
databaseConnectionString!,
DEFAULT_SQL_OPTIONS,
DEFAULT_BLOB_PERSISTENCE_ARRAY,
Expand All @@ -73,6 +74,7 @@ export class BlobServerFactory {
const config = new BlobConfiguration(
env.blobHost(),
env.blobPort(),
env.blobKeepAliveTimeout(),
join(location, DEFAULT_BLOB_LOKI_DB_PATH),
join(location, DEFAULT_BLOB_EXTENT_LOKI_DB_PATH),
DEFAULT_BLOB_PERSISTENCE_ARRAY,
Expand Down
1 change: 1 addition & 0 deletions src/blob/IBlobEnvironment.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export default interface IBlobEnvironment {
blobHost(): string | undefined;
blobPort(): number | undefined;
blobKeepAliveTimeout(): number | undefined;
location(): Promise<string>;
silent(): boolean;
loose(): boolean;
Expand Down
5 changes: 4 additions & 1 deletion src/blob/SqlBlobConfiguration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ import {
DEFAULT_BLOB_PERSISTENCE_ARRAY,
DEFAULT_BLOB_SERVER_HOST_NAME,
DEFAULT_ENABLE_ACCESS_LOG,
DEFAULT_ENABLE_DEBUG_LOG
DEFAULT_ENABLE_DEBUG_LOG,
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT
} from "./utils/constants";

/**
Expand All @@ -22,6 +23,7 @@ export default class SqlBlobConfiguration extends ConfigurationBase {
public constructor(
host: string = DEFAULT_BLOB_SERVER_HOST_NAME,
port: number = DEFAULT_BLOB_LISTENING_PORT,
keepAliveTimeout: number = DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT,
public readonly sqlURL: string,
public readonly sequelizeOptions: SequelizeOptions = DEFAULT_SQL_OPTIONS,
public readonly persistenceArray: StoreDestinationArray = DEFAULT_BLOB_PERSISTENCE_ARRAY,
Expand All @@ -40,6 +42,7 @@ export default class SqlBlobConfiguration extends ConfigurationBase {
super(
host,
port,
keepAliveTimeout,
enableAccessLog,
accessLogWriteStream,
enableDebugLog,
Expand Down
2 changes: 2 additions & 0 deletions src/blob/utils/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ export const EMULATOR_ACCOUNT_KEY = Buffer.from(
export const EMULATOR_ACCOUNT_SKUNAME = Models.SkuName.StandardRAGRS;
export const EMULATOR_ACCOUNT_KIND = Models.AccountKind.StorageV2;
export const EMULATOR_ACCOUNT_ISHIERARCHICALNAMESPACEENABLED = false;
export const DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT = 5;


export const HeaderConstants = {
AUTHORIZATION: "authorization",
Expand Down
1 change: 1 addition & 0 deletions src/common/ConfigurationBase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ export default abstract class ConfigurationBase {
public constructor(
public readonly host: string,
public readonly port: number,
public readonly keepAliveTimeout: number,
public readonly enableAccessLog: boolean = false,
public readonly accessLogWriteStream?: NodeJS.WritableStream,
public readonly enableDebugLog: boolean = false,
Expand Down
30 changes: 30 additions & 0 deletions src/common/Environment.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
import args from "args";

import {
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT,
DEFAULT_BLOB_LISTENING_PORT,
DEFAULT_BLOB_SERVER_HOST_NAME
} from "../blob/utils/constants";

import {
DEFAULT_QUEUE_KEEP_ALIVE_TIMEOUT,
DEFAULT_QUEUE_LISTENING_PORT,
DEFAULT_QUEUE_SERVER_HOST_NAME
} from "../queue/utils/constants";

import {
DEFAULT_TABLE_KEEP_ALIVE_TIMEOUT,
DEFAULT_TABLE_LISTENING_PORT,
DEFAULT_TABLE_SERVER_HOST_NAME
} from "../table/utils/constants";
Expand All @@ -28,6 +31,11 @@ args
"Optional. Customize listening port for blob",
DEFAULT_BLOB_LISTENING_PORT
)
.option(
["", "blobKeepAliveTimeout"],
"Optional. Customize http keep alive timeout for blob",
DEFAULT_BLOB_KEEP_ALIVE_TIMEOUT,
)
.option(
["", "queueHost"],
"Optional. Customize listening address for queue",
Expand All @@ -38,6 +46,11 @@ args
"Optional. Customize listening port for queue",
DEFAULT_QUEUE_LISTENING_PORT
)
.option(
["", "queueKeepAliveTimeout"],
"Optional. Customize http keep alive timeout for queue",
DEFAULT_QUEUE_KEEP_ALIVE_TIMEOUT,
)
.option(
["", "tableHost"],
"Optional. Customize listening address for table",
Expand All @@ -48,6 +61,11 @@ args
"Optional. Customize listening port for table",
DEFAULT_TABLE_LISTENING_PORT
)
.option(
["", "tableKeepAliveTimeout"],
"Optional. Customize http keep alive timeout for table",
DEFAULT_TABLE_KEEP_ALIVE_TIMEOUT,
)
.option(
["l", "location"],
"Optional. Use an existing folder as workspace path, default is current working directory",
Expand Down Expand Up @@ -99,6 +117,10 @@ export default class Environment implements IEnvironment {
return this.flags.blobPort;
}

public blobKeepAliveTimeout(): number | undefined {
return this.flags.blobKeepAliveTimeout;
}

public queueHost(): string | undefined {
return this.flags.queueHost;
}
Expand All @@ -107,6 +129,10 @@ export default class Environment implements IEnvironment {
return this.flags.queuePort;
}

public queueKeepAliveTimeout(): number | undefined {
return this.flags.queueKeepAliveTimeout;
}

public tableHost(): string | undefined {
return this.flags.tableHost;
}
Expand All @@ -115,6 +141,10 @@ export default class Environment implements IEnvironment {
return this.flags.tablePort;
}

public tableKeepAliveTimeout(): number | undefined {
return this.flags.tableKeepAliveTimeout;
}

public async location(): Promise<string> {
return this.flags.location || process.cwd();
}
Expand Down
4 changes: 4 additions & 0 deletions src/common/ServerBase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export default abstract class ServerBase implements ICleaner {
* @param {number} port Server port, for example, 10000
* @param {http.Server | https.Server} httpServer A HTTP or HTTPS server instance without request listener bound
* @param {IRequestListenerFactory} requestListenerFactory A request listener factory
* @param config ConfigurationBase configuration
* @memberof ServerBase
*/
public constructor(
Expand All @@ -45,6 +46,9 @@ export default abstract class ServerBase implements ICleaner {
requestListenerFactory: IRequestListenerFactory,
public readonly config: ConfigurationBase
) {
if (this.config.keepAliveTimeout > 0) {
httpServer.keepAliveTimeout = this.config.keepAliveTimeout * 1000;
}
// Remove predefined request listeners to avoid double request handling
this.httpServer = stoppable(httpServer);
this.httpServer.removeAllListeners("request");
Expand Down
12 changes: 12 additions & 0 deletions src/common/VSCEnvironment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ export default class VSCEnvironment implements IEnvironment {
return this.workspaceConfiguration.get<number>("blobPort");
}

public blobKeepAliveTimeout(): number | undefined {
return this.workspaceConfiguration.get<number>("blobKeepAliveTimeout");
}

public queueHost(): string | undefined {
return this.workspaceConfiguration.get<string>("queueHost");
}
Expand All @@ -23,6 +27,10 @@ export default class VSCEnvironment implements IEnvironment {
return this.workspaceConfiguration.get<number>("queuePort");
}

public queueKeepAliveTimeout(): number | undefined {
return this.workspaceConfiguration.get<number>("queueKeepAliveTimeout");
}

public tableHost(): string | undefined {
return this.workspaceConfiguration.get<string>("tableHost");
}
Expand All @@ -31,6 +39,10 @@ export default class VSCEnvironment implements IEnvironment {
return this.workspaceConfiguration.get<number>("tablePort");
}

public tableKeepAliveTimeout(): number | undefined {
return this.workspaceConfiguration.get<number>("tableKeepAliveTimeout");
}

public async location(): Promise<string> {
let location = this.workspaceConfiguration.get<string>("location");

Expand Down
1 change: 1 addition & 0 deletions src/common/VSCServerManagerBlob.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ export default class VSCServerManagerBlob extends VSCServerManagerBase {
const config = new BlobConfiguration(
env.blobHost(),
env.blobPort(),
env.blobKeepAliveTimeout(),
join(location, DEFAULT_BLOB_LOKI_DB_PATH),
join(location, DEFAULT_BLOB_EXTENT_LOKI_DB_PATH),
DEFAULT_BLOB_PERSISTENCE_ARRAY,
Expand Down
1 change: 1 addition & 0 deletions src/common/VSCServerManagerQueue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export default class VSCServerManagerBlob extends VSCServerManagerBase {
const config = new QueueConfiguration(
env.queueHost(),
env.queuePort(),
env.blobKeepAliveTimeout(),
Copy link
Member

@blueww blueww Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be env.queueKeepAliveTimeout()?

join(location, DEFAULT_QUEUE_LOKI_DB_PATH),
join(location, DEFAULT_QUEUE_EXTENT_LOKI_DB_PATH),
DEFAULT_QUEUE_PERSISTENCE_ARRAY,
Expand Down
Loading
Loading