Fluid Framework v2.0.0-rc.4.0.7 (patch)

What's Changed

• [Port rc 4.0] Stop using short data store IDs in detached container (#21613) #21630
• RC.4.0 Port - Put Memory Blob Storage Under a Flag (#21612) #21619
• [main>rc4] ci: fix build-tools installation in GH actions workflow #21594
• Update Arrow To V5 In RC4 #21561
• [bump] client: 2.0.0-rc.4.0.6 => 2.0.0-rc.4.0.7 (patch) #21441

Full Changelog: client_v2.0.0-rc.4.0.6...client_v2.0.0-rc.4.0.7

Fluid Framework v2.0.0-rc.3.0.9 (patch)

What's Changed

• [Port rc 3.0] Stop using short data store IDs in detached container (#21613) #21628
• RC.3.0 Port - Put Memory Blob Storage Under a Flag (#21612) #21620
• Update Arrow to v5 in RC3 #21559
• [bump] client: 2.0.0-rc.3.0.8 => 2.0.0-rc.3.0.9 (patch) #21440

Full Changelog: client_v2.0.0-rc.3.0.8...client_v2.0.0-rc.3.0.9

Fluid Framework v2.0.0-rc.2.0.9 (patch)

What's Changed

• [Port rc.2.0] Stop using short data store IDs in detached container (#21613) #21631
• Update Arrow to v5 in RC2 #21557
• [bump] client: 2.0.0-rc.2.0.8 => 2.0.0-rc.2.0.9 (patch) #21334

Full Changelog: client_v2.0.0-rc.2.0.8...client_v2.0.0-rc.2.0.9

Fluid Framework v2.0.0

Fluid Framework 2 is now generally available and ready for production deployments! 🎉

New Features

🌳 SharedTree DDS

In Fluid Framework 2, we're introducing a new DDS called SharedTree, a powerful and flexible DDS designed to keep hierarchical data synchronized between clients. We encourage you to move all new development and containers to SharedTree. We believe SharedTree is the best option for building with Fluid Framework 2. To learn more, visit the SharedTree docs

Note: As a result of the new DDS, we have marked most pre-existing DDSes as Legacy (not deprecated yet), so you will have to update your import paths. Learn more about API support levels and our goal for a graceful DDS deprecation
in future.

📁 SharePoint Embedded Support

We are also introducing support for a new relay option in Fluid Framework 2, called SharePoint Embedded. This is a Microsoft 365 hosted service that keeps your collaborative Fluid data saved in a partition on SharePoint, allowing you to harness the power of Microsoft 365 file and document storage platform for you apps. You can integrate with SharePoint Embedded using the @fluidframework/odsp-client package. To learn more, visit the SharePoint Embedded docs.

🩺 Fluid DevTools

Fluid Developer Tools is a browser extension that improves the developer experience when writing and debugging Fluid apps. It allows you to peek inside your application's Fluid container and diagnose issues. You can learn more about it in the DevTools docs.

🛠️ Start Building Today!

And many more updates! Please continue to engage with us on GitHub Discussion and Issue pages as you adopt Fluid Framework 2!

Fluid Framework v2.0.0-rc.5.0.1

What's Changed

Other changes

• [bump] client: 2.0.0-rc.5.0.0 => 2.0.0-rc.5.0.1 (patch) by @tylerbutler in #21546
• Update Arrow To V5 In RC5 by @RishhiB in #21564
• fix(test-version-utils): Add temporary workaround for calculating N-1 for 2.0 by @tylerbutler in #21562
• tree(fix): Validate source indices correctly on moves (#21552) by @daesunp in #21558
• Patch (#21568): Remove O(n) slowdown from event listeners by @noencke in #21573
• Patch (#21570) tree: Reuse flex tree fields in most cases by @noencke in #21577

Full Changelog: client_v2.0.0-rc.5.0.0...client_v2.0.0-rc.5.0.1

Fluid Framework v2.0.0-rc.5.0.0 (major)

New Features

tree: Added support for optional schema validation on newly inserted content in SharedTree

When defining how to view a SharedTree, an application can now specify that new content inserted into the tree should be subject to schema validation at the time it is inserted, so if it's not valid according to the stored schema in the tree an error is thrown immediately.

This can be accomplished by passing an ITreeConfigurationOptions argument with enableSchemaValidation set to true when creating a TreeConfiguration to use with the SharedTree.

Since this feature requires additional compute when inserting new content into the tree, it is not enabled by default.

fluid-framework: Type Erase ISharedObjectKind

A new type, SharedObjectKind is added as a type erased version of ISharedObjectKind and DataObjectClass.

This type fills the role of both ISharedObjectKind and DataObjectClass in the @public "declarative API" exposed in the fluid-framework package.

This allows several types referenced by ISharedObjectKind to be made @alpha as they should only need to be used by legacy code and users of the unstable/alpha/legacy "encapsulated API".

Access to these now less public types should not be required for users of the @public "declarative API" exposed in the fluid-framework package, but can still be accessed for those who need them under the /legacy import paths. The full list of such types is:

• SharedTree as exported from @fluidframwork/tree: It is still exported as @public from fluid-framework as SharedObjectKind.
• ISharedObjectKind: See new SharedObjectKind type for use in @public APIs. ISharedObject
• IChannel
• IChannelAttributes
• IChannelFactory
• IExperimentalIncrementalSummaryContext
• IGarbageCollectionData
• ISummaryStats
• ISummaryTreeWithStats
• ITelemetryContext
• IDeltaManagerErased
• IFluidDataStoreRuntimeEvents
• IFluidHandleContext
• IProvideFluidHandleContext

Removed APIs:

• DataObjectClass: Usages replaced with SharedObjectKind.
• LoadableObjectClass: Replaced with SharedObjectKind.
• LoadableObjectClassRecord: Replaced with Record<string, SharedObjectKind>.

tree: A new tree status has been added for SharedTree nodes.

TreeStatus.New indicates that a SharedTree node has been constructed but not yet inserted into the tree. Constraints passed to the runTransaction API are now marked as readonly.

Breaking Changes

odsp-client: Move odsp-client out of experimental

The scope of the odsp-client package is changed from @fluid-experimental/odsp-client to @fluidframework/odsp-client.

fluid-framework: Remove some types from @public that are not needed

Mark the following APIs @alpha instead of @public:

• IBranchOrigin
• ISequencedDocumentMessage
• ISignalMessage
• ISignalMessageBase
• ITrace

fluid-framework: Remove several types from @public scope

The following types have been moved from @public to @alpha:

• IFluidSerializer
• ISharedObjectEvents
• IChannelServices
• IChannelStorageService
• IDeltaConnection
• IDeltaHandler

These should not be needed by users of the declarative API, which is what @public is targeting.

tree: Event types have been renamed

• ISubscribable is renamed to Listenable.
• IsEvent type helper is renamed to IsListener.
• Events is renamed to Listeners.

tree: Breaking change: Removed the "afterBatch" event from Treeview

This event is no longer necessary. In the past, it provided a means for waiting for a batch of changes to finish applying to the tree before taking some action. However, the tree change events exposed via Tree.on wait for a batch to complete before firing, so the "afterBatch" event provides no additional guarantees. Listeners of this event who wish to respond to changes to the tree view can use "rootChanged" instead.

tree: Move several types into InternalTypes

The stable public API surface for Tree has been reduced.
Several types have been moved into InternalTypes, indicating that they are not fully stable nor intended to be referenced by users of Tree.

• NodeBuilderData
• FieldHasDefault
• TreeNodeSchemaNonClass
• TreeArrayNodeBase
• ScopedSchemaName
• DefaultProvider
• typeNameSymbol
• InsertableObjectFromSchemaRecord
• ObjectFromSchemaRecord
• FieldHasDefaultUnsafe
• ObjectFromSchemaRecordUnsafe
• TreeObjectNodeUnsafe
• TreeFieldFromImplicitFieldUnsafe
• TreeNodeFromImplicitAllowedTypesUnsafe
• InsertableTreeNodeFromImplicitAllowedTypesUnsafe
• TreeArrayNodeUnsafe
• TreeMapNodeUnsafe
• InsertableObjectFromSchemaRecordUnsafe
• InsertableTreeFieldFromImplicitFieldUnsafe
• InsertableTypedNodeUnsafe
• NodeBuilderDataUnsafe
• NodeFromSchemaUnsafe
• FlexList
• TreeApi

Additionally a few more types which could not be moved due to technically limitations have been documented that they should be treated similarly.

• TreeNodeApi
• TreeNodeSchemaCore
• All *Unsafe type (use for construction of recursive schema).
• WithType
• AllowedTypes
• FieldSchemaUnsafe

Also to reduce confusion type was renamed to typeNameSymbol, and is now only type exported. Tree.is should be used to get type information from TreeNodes instead.

ITree.schematize removal

ITree.schematize (and its argument TreeConfiguration) has been removed. Instead, call ITree.viewWith and provide it a TreeViewConfiguration. Unlike schematize, viewWith does not implicitly initialize the document. As such, it doesn't take an initialTree property. Instead, applications should initialize their trees in document creation codepaths using the added TreeView.initialize API.

Old

As an example, something like the following code may have been used before for both the document create and document load codepaths:

// -- fluid-framework API for statically defined objects in container schema --
const tree = container.initialObjects.myTree;
const view = tree.schematize(
  new TreeConfiguration(Point, () => new Point({ x: 0, y: 0 })),
);

// -- fluid-framework API for dynamically created objects --
const tree = await container.create(SharedTree);
const view = tree.schematize(
  new TreeConfiguration(Point, () => new Point({ x: 0, y: 0 })),
);

When using the encapsulated API, creating a tree looks a bit different but the call to schematize is the same:

// -- encapsulated API --
const tree = SharedTree.create(runtime, "foo");
const view = tree.schematize(
  new TreeConfiguration(Point, () => new Point({ x: 0, y: 0 })),
);

New

After migrating this code away from schematize and onto viewWith, it would look like this on the create codepath:

const treeConfig = new TreeViewConfiguration({ schema: Point });

// The following line reflects the first-party API (e.g. @fluidframework/aqueduct). If using the third-party API, obtaining
// a SharedTree is unaffected by this changeset.
const tree = SharedTree.create(runtime, "foo");
const view = tree.viewWith(treeConfig);
view.initialize(new Point({ x: 0, y: 0 }));

and this on the load codepath:

// 'tree' would typically be obtained by retrieving it from a well-known location, e.g. within a `DataObject`'s
// root directory or in `IFluidContainer.initialObjects`
const view = tree.viewWith(treeConfig);

Besides only making the initial tree required to specify in places that actually perform document initialization, this is beneficial for mutation semantics: tree.viewWith never modifies the state of the underlying tree. This means applications are free to attempt to view a document using multiple schemas (e.g. legacy versions of their document format) without worrying about altering the document state.

If existing code used schematize in a context where it wasn't known whether the document needed to be initialized, you can leverage TreeView.compatibility like so:

const view = tree.viewWith(config);
if (view.compatibility.canInitialize) {
  view.initialize(initialTree);
}

core-interfaces, tree: Unify IDisposable interfaces

Public APIs in @fluidframework/tree now use IDisposable from @fluidframework/core-interfaces replacing disposeSymbol with "dispose".

IDisposable in @fluidframework/core-interfaces is now @sealed indicating that third parties should not implement it to reserve the ability for Fluid Framework to extend it to include Symbol.dispose as a future non-breaking change.

telemetry-utils: Breaking Change: Update MockLogger's events property is no longer externally mutable

If you depended on this mutability to implement some behavior, you should create your own mock logger implementation.

If you depended on this mutability to work around the logger's self-clearing behavior after running a match check, you can now override this behavior via the clearEventsAfterCheck parameter.

telemetry-utils: Deprecate MockLogger for external use.

No replacement API is given. This type was never intended for use outside of the fluid-framework repository.
If you were depending on this class for testing purposes, we recommend creating your own mock logger implementation,
or copy and adapt the code from fluid-framework as needed.

Notable Updates

Update to TypeScript 5.4

Update package implementations to use TypeScript 5.4.5.

Update to ES 2022

Update tsconfig to target ES 2022.

azure-client, tinylicious-client: compatibilityMode parameter added to createContainer and getContainer on AzureClient and TinyliciousClient

To support migration from 1.x to 2.0, a compatibility mode parameter has been added to these ...

Routerlicious v5.0.0

server-services-core: New configuration setting for ephemeral container soft delete

IDeliServerConfiguration defines a new optional property, ephemeralContainerSoftDeleteTimeInMs, that controls when ephemeral containers are soft-deleted.

You can find more details in pull request #20731.

server-services-core: New optional dispose method

Adds optional dispose method to IWebSocket for disposing event listeners on disconnect in Nexus lambda.

You can find more details in pull request #21211.

server-services-core: Reduce session grace period for ephemeral containers to 2 minutes (was 10 minutes)

For ephermeral container, the session grace period is reduced from 10 minutes to 2 minutes when cluster is draining. This ensures the ephemeral container gets cleaned after disconnection sooner. Clients will not find old EH containers and will need to create new containers. This logic only takes effect when forcing draining.

You can find more details in pull request #21010.

server-lambdas: Nexus client connections can now disconnect in batches

Added the option to make Nexus client connections disconnect in batches. The new options are within socketIo element of the Nexus config:

• gracefulShutdownEnabled (true or false)
• gracefulShutdownDrainTimeMs (overall time for disconnection)
• gracefulShutdownDrainIntervalMs (how long each batch has to disconnect)

Additionally, the DrainTimeMs setting should be set to a value greater than the setting shared:runnerServerCloseTimeoutMs which governs how long Alfred and Nexus have to shutdown.

You can find more details in pull request #19938.

server-lambdas: Performance: Keep pending checkpoint message for future summaries

During a session there may be multiple client/service summary calls, and independently, multiple checkpoints. Checkpoint will clear messages storage in pendingCheckpointMessages, which is also used for writing summaries. Because of this cleanup, when we write new summaries, it often needs to request the ops from Alfred again, which is not quite efficient.

Now the pending messages are cached for improved performance.

You can find more details in pull request #20029.

server-services-core: Fix: Limit max length of validParentSummaries in checkpoints

Limits maximum number of tracked valid parent (service) summaries to 10 by default. Configurable via IScribeServerConfiguration in scribe property of IServiceConfiguration.

You can find more details in pull request #20850.

server-lambdas: Fix: send correct connection scopes for client

When a client joins in "write" mode with only "read" scopes in their token, the connection message from server will reflect a "read" client mode.

You can find more details in pull request #20312.

protocol-base: Fix: ensure immutability of quorum snapshot

Creates a deeper clone of the quorum members when snapshotting to make sure the snapshot is immutable.

You can find more details in pull request #20329.

server-lambdas: Fix: cover edge cases for scrubbed checkpoint users

Overhauled how the Scribe lambda handles invalid, missing, or outdated checkpoint data via fallbacks.

Before:

if (no global checkpoint)
  use Default checkpoint
elsif (global checkpoint was cleared or  global checkpoint quorum was scrubbed)
  use Summary checkpoint
else
  use latest DB checkpoint (local or global)

After:

if (no global and no local checkpoint and no summary checkpoint)
  use Default checkpoint
elsif (
	global checkpoint was cleared and summary checkpoint ahead of local db checkpoint
	or latest DB checkpoint quorum was scrubbed
	or summary checkpoint ahead of latest DB checkpoint
)
  use Summary checkpoint
else
  use latest DB checkpoint (local or  global)

Also: Updated CheckpointService with additional fallback logic for loading a checkpoint from local or global DB depending on whether the quorum information in the checkpoint is valid (i.e. does not contain scrubbed users).

You can find more details in pull request #20259.

server-routerlicious-base: Add support for custom tenant key generators

Added support to add a custom tenant key generator instead of using just the default 128-bit sha256 key.

You can find more details in pull request #20844.

server-routerlicious-base: Remove Riddler HTTP request for performance

The getOrderer workflow no longer calls getTenant when globalDb is enabled. This saves two HTTP calls to Riddler and will improve performance.

You can find more details in pull request #20773.

protocol-base: Fix: configure user data scrubbing in checkpoints and summaries

Note: This change is primarily internal to routerlicious.

• When scribe boots from a checkpoint, it fails over to the latest summary checkpoint if the quorum is corrupted (i.e. user data is scrubbed).

• When scribe writes a checkpoint to DB or a summary, it respects new IScribeServerConfiguration options (scrubUserDataInSummaries, scrubUserDataInLocalCheckpoints, and scrubUserDataInGlobalCheckpoints) when determining whether to scrub user data in the quorum.

• Added optional param, scrubUserData, to ProtocolOpHandler.getProtocolState(). When true, user data in the quorum is replaced with { id: "" }. Defaults to false. Previously was always scrubbed.

• Added the following configuration options for IScribeServerConfiguration:

  • scrubUserDataInSummaries
  • scrubUserDataInLocalCheckpoints
  • scrubUserDataInGlobalCheckpoints

  All default to false.

You can find more details in pull request #20150.

server-services-utils: Add support for custom authentication with Redis

Added support for custom authentication with Redis instead of only password based authentication. This includes support for Microsoft Entra-ID based authentication for Redis.

You can find more details in pull request #20214.

server-services-client: Add optional internalErrorCode property to NetworkError and INetworkErrorDetails

NetworkErrors now include an optional property, internalErrorCode, which can contain additional information about the internal error.

You can find more details in pull request #21429.

server-services-shared: Fixed the ordering in Nexus shutdown

Before, the Redis Pub/Sub would be disposed before the socket connections were closed. Now we first close socket connections then do Redis disposal.

You can find more details in pull request #20429.

server-lambdas, server-services-core: SessionStartMetric removed from Scribe and Deli microservices

This change removes the SessionStartMetric from Scribe and Deli. The metric is a source of bugs and has been superseded by the restoreFromCheckpoint and RunService metrics.

You can find more details about the reasons for this change in pull request #21125.

Fluid Framework v2.0.0-rc.4.0.6 (patch)

What's Changed

• Remove deprecation from SharedDirectory (#21423) #21432
• [bump] client: 2.0.0-rc.4.0.5 => 2.0.0-rc.4.0.6 (patch) #21332

Full Changelog: client_v2.0.0-rc.4.0.5...client_v2.0.0-rc.4.0.6

Fluid Framework v2.0.0-rc.3.0.8 (patch)

What's Changed

• Remove deprecation from SharedDirectory (#21423) #21431
• [bump] client: 2.0.0-rc.3.0.7 => 2.0.0-rc.3.0.8 (patch) #21333

Full Changelog: client_v2.0.0-rc.3.0.7...client_v2.0.0-rc.3.0.8

Fluid Framework v2.0.0-rc.4.0.5 (patch)

What's Changed

• fix(client-utils): fix trace.js import of performance (#21257) #21323
• ci: Publish patch GitHub releases automatically #21309
• Use post-creation resolved URL to handle discovery pattern (#21080) #21306
• [bump] client: 2.0.0-rc.4.0.4 => 2.0.0-rc.4.0.5 (patch) #21253

Full Changelog: client_v2.0.0-rc.4.0.4...client_v2.0.0-rc.4.0.5