Release 3.2.3

Features

• Enable RabbitMQ username/password to be configurable via CLUEDIN_RABBITMQ_USERNAME + CLUEDIN_RABBITMQ_PASSWORD

Fixes

• Fixed an issue where file paths contain spaces causes docker to fail
• Now uses nip.io instead of xip.io for dns resolution.

Release 3.2.2

Features

• Added connectionstring configuration for AuditLog db.

Added new environment variables to control features

• Roles - controls exposure fo the new Roles feature.

Name	Default Value	Possible Values
CLUEDIN_UI_FEATURES_ROLES	roles	roles

Fixes

• Added GQL => SQLSERVER environment variables to GQL compose file.
• Correctly use CLUEDIN_AGENT_ENABLED value in server configuration
• Ensure null/empty values from environment are treated as null

Release 3.2.2-beta.6

Features

• Added connectionstring configuration for AuditLog db.

Added new environment variables to control features

• Roles - controls exposure fo the new Roles feature.

Name	Default Value	Possible Values
CLUEDIN_UI_FEATURES_ROLES	roles	roles

Fixes

• Added GQL => SQLSERVER environment variables to GQL compose file.
• Correctly use CLUEDIN_AGENT_ENABLED value in server configuration

Release 3.2.0

Features

New Command open

Run the open command to open a CluedIn instance in your browser. The instance must have already been started with up or start:

Examples

pwsh ./cluedin.ps1 open #=> Opens the browser for the default environment
pwsh ./cluedin.ps1 open local #=> Opens the browser for the 'local' environment
pwsh ./cluedin.ps1 open -org custom #=> Opens the browser for the default environment and the 'custom' org

Jobs Dashboard Visibility

A new environment variable is available to configure the visibility of the jobs server dashboard CLUEDIN_JOBS_DASHBOARD. By default, the dashboard is configured to be visible.

Examples

pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=true #=> Enables access to the dashboard on localhost:9003/hangfire
pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=false #=> Disables access to the dashboard
pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=localOnly #=> Enables access to the dashboard on localhost:9003/hangfire only for requests from localhost

The local port that is used to access the dashboard is controlled by the CLUEDIN_SERVER_JOB_LOCALPORT environment variable

Added new environment variables to control features

Features for the UI instance can now be controlled through environment variables.
Set the environment variable to a blank value to disable.

Name	Default Value	Possible Values
CLUEDIN_UI_ALT_UPLOAD	http://gql:8888	Any uri
CLUEDIN_UI_FEATURES_ANNOTATION	annotation	annotation
CLUEDIN_UI_FEATURES_CLEAN	(blank)	clean
CLUEDIN_UI_FEATURES_CONNECTOR	connector	connector
CLUEDIN_UI_FEATURES_DATAGOVERNANCE	data-governance	data-governance
CLUEDIN_UI_FEATURES_DATASOURCE	datasource	datasource
CLUEDIN_UI_FEATURES_DEVELOPER	developer	developer
CLUEDIN_UI_FEATURES_ENDPOINT	endpoint	endpoint
CLUEDIN_UI_FEATURES_ENRICHER	enricher	enricher
CLUEDIN_UI_FEATURES_EXTENDEDSAR	extended-sar	extended-sar
CLUEDIN_UI_FEATURES_FOLLOW	(blank)	follow
CLUEDIN_UI_FEATURES_GLOSSARY	glossary	glossary
CLUEDIN_UI_FEATURES_INTEGRATIONS	integrations	integrations
CLUEDIN_UI_FEATURES_METRICS	metrics	metrics
CLUEDIN_UI_FEATURES_OPENREFINE	cluedinClean	cluedinClean
CLUEDIN_UI_FEATURES_PREPARATION	preparation	preparation
CLUEDIN_UI_FEATURES_SAVEDSEARCHES	(blank)	savedSearches
CLUEDIN_UI_FEATURES_SETTINGS	settings	settings
CLUEDIN_UI_FEATURES_STREAMS	stream;streams	stream;streams
CLUEDIN_UI_FEATURES_USERS	users	users

Note: OpenRefine is enabled by default so CLEAN is blank by default

Fixes

Disk drop off for Server emails

In previous releases, the CluedIn Server instance was not configured to drop emails to a local directory.
This would cause errors when sending emails after crawl jobs have completed.
The server image is configured using the same folder as the webapi with the environment variable CLUEDIN_EMAIL_DIR

Map NODE_ENV to CLUEDIN_ENVIRONMENT

For NodeJS apps (clean / gql / annotation / datasource) set the NODE_ENV correctly based on CLUEDIN_ENVIRONMENT

Default UI Port changed to 9080

To simplify support for developers and normalize the ports across all services under 90** the default port for the UI has changed to port 9080.

To change the port to an alternative (e.g. port 80):

pwsh ./cluedin.ps1 env -set CLUEDIN_UI_LOCALPORT=80 #=> Set to port 80 for the default environment
pwsh ./cluedin.ps1 env local -set CLUEDIN_UI_LOCALPORT=80 #=> Set to port 80 for the custom 'local' environment

check uses configured ports

The check command now correctly uses the ports configured for an environment.

Release 3.2.0-beta.5

Features

Jobs Dashboard Visibility

A new environment variable is available to configure the visibility of the jobs server dashboard CLUEDIN_JOBS_DASHBOARD. By default, the dashboard is configured to be visible.

Examples

pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=true #=> Enables access to the dashboard on localhost:9003/hangfire
pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=false #=> Disables access to the dashboard
pwsh ./cluedin.ps1 env -set CLUEDIN_JOBS_DASHBOARD=localOnly #=> Enables access to the dashboard on localhost:9003/hangfire only for requests from localhost

The local port that is used to access the dashboard is controlled by the CLUEDIN_SERVER_JOB_LOCALPORT environment variable

Added new environment variables to control features

Features for the UI instance can now be controlled through environment variables.

Name	Default Value	Possible Values
CLUEDIN_UI_FEATURES_OPENREFINE	cluedinClean	cluedinClean
CLUEDIN_UI_FEATURES_CLEAN	(blank)	clean
CLUEDIN_UI_FEATURES_GLOSSARY	(blank)	glossary
CLUEDIN_UI_FEATURES_STREAMS	(blank)	stream;streams
CLUEDIN_UI_FEATURES_FOLLOW	(blank)	follow
CLUEDIN_UI_FEATURES_SAVEDSEARCHES	(blank)	savedSearches
CLUEDIN_UI_FEATURES_USERS	users	users
CLUEDIN_UI_FEATURES_DATAGOVERNANCE	data-governance	data-governance
CLUEDIN_UI_FEATURES_DEVELOPER	developer	developer
CLUEDIN_UI_FEATURES_SETTINGS	settings	settings
CLUEDIN_UI_FEATURES_INTEGRATIONS	integrations	integrations
CLUEDIN_UI_FEATURES_EXTENDEDSAR	extended-sar	extended-sar
CLUEDIN_UI_FEATURES_ANNOTATION	annotation	annotation
CLUEDIN_UI_FEATURES_DATASOURCE	datasource	datasource
CLUEDIN_UI_FEATURES_METRICS	metrics	metrics
CLUEDIN_UI_FEATURES_ENRICHER	enricher	enricher

Note: OpenRefine is enabled by default so CLEAN is blank by default

Fixes

Disk drop off for Server emails

In previous releases, the CluedIn Server instance was not configured to drop emails to a local directory.
This would cause errors when sending emails after crawl jobs have completed.
The server image is configured using the same folder as the webapi with the environment variable CLUEDIN_EMAIL_DIR

Map NODE_ENV to CLUEDIN_ENVIRONMENT

For NodeJS apps (clean / gql / annotation / datasource) set the NODE_ENV correctly based on CLUEDIN_ENVIRONMENT

Hotfix - 3.1.1

Fixes

1. Resolved an issue with createorg where the incorrect port was being when registering an organization

Feature Release - 3.1.0

Features

Service Specific Tags

Service specific tags allow each individual in service in the CluedIn environment to be deployed under a specific version.

1. [BREAKING] - The CLUEDIN_DOCKER_TAG environment variable no longer exists, it has been replaced by....
2. Each service has a unique CLUEDIN_<servicename>_TAG environment variable (set to latest by default). The service name is the name of the service specified in the docker-compose files.
3. All docker commands (up,down,pull,start,stop) now expose a -tagoverride option which can be given <key>=<value> pairs to specify service specific tags when invoked: cluedin.ps1 up -tagoverride server=develop
4. All docker commands (up,down,pull,start,stop) retain the -tag option. This will override all service tags unless the -tagoverride option is used for a service: cluedin.ps1 up -tag develop -tagoverride server=feature-rules-engine
5. The env command also supports -tag and -tagoverride persisting the changes into the relevant env file instead: cluedin.ps1 env rules -tag develop -tagoverride server=feature-rules-engine

Examples

pwsh ./cluedin.ps1 env #=> Will list the environment - note CLUEDIN_<servicename>_TAG for each service
pwsh ./cluedin.ps1 up -tag develop -tagoverride server=feature-rules-engine -pull #=> Will pull the latest images for all services, start up Cluedin with develop services, except 'server' will run feature-rules-engine
pwsh ./cluedin.ps1 down #=> Terminate all services
pwsh ./cluedin.ps1 env rules -tag develop -tagoverride server=feature-rules-engine #=> Creates a new rules environment with the specific service tags
pwsh ./cluedin.ps1 up rules #=> start up Cluedin with develop services, except 'server' will run feature-rules-engine as configured by the rules environment

Package Installation

Package installation enables selective download and installation of integration packages for the CluedIn
service through the nuget-installer docker image. Installation is managed per environment, so unique package configurations can be tested.

1. [BREAKING] A new CLUEDIN_INSTALLER_TAG with a value of latest has been added to the environment. You must add this to your own .env files if you have existing custom environments befoire using this version of the home repo.
2. The packages command is the root command for all package functionality.
3. By default the list of packages to be installed is displayed: cluedin.ps1 packages
4. A package may be added with an optional version: cluedin.ps1 packages -add <packagename> -version <version>
   • If no version is supplied the latest version will be installed.
   • Package versions may be specified using nuget floating version semantics.
5. A package may be removed: cluedin.ps1 packages -remove <packagename>
6. Feeds may be added with optional credentials: cluedin.ps1 -addfeed <feedname> -uri <uri> -user <username> -pass <password>
   • Nuget.org is pre-configured in the installation image.
   • A default folder is also configured for installation of local packages /<env>/packages/local.
7. Feeds may be removed: cluedin.ps1 -removefeed <feedname>
8. After configuration, packages may be restored ready for installation: cluedin.ps1 packages -restore
9. Previously restored packages may be cleaned from disk: cluedin.ps1 packages -clean
10. The image used for package installation and configuration can be specified with the -tag parameter when using -addfeed, -removefeed, or -restore.
    • Additionally the -pull flag can be specified to pull that image
11. ALL package commands may also specify an environment to apply the changes to that specific environment. E.g. cluedin.ps1 packages custom -add My.Custom.Package will add the package to the custom environment.
    • The default environment is used if none is specified

When building local packages you can copy the created nuget package to /<env>/packages/local and add the package name with cluedin.ps1 packages -add <name>. This removes the requirement to publish the package to a custom feed.

Examples

pwsh ./cluedin.ps1 packages -add Cluedin.Crawling.HubSpot #=> Configures the package for installation with the latest version
pwsh ./cluedin.ps1 packages -addfeed cluedin -uri <cluedinfeed> -user <username> -pass <password> #=> Configures the feed to download packages
pwsh ./cluedin.ps1 packages -restore #=> Restores packages for the environment
pwsh ./cluedin.ps1 packages up #=> Starts the environments with the packages

# The following completes the same actions as above, but for a specific environment
pwsh ./cluedin.ps1 packages custom -add Cluedin.Crawling.HubSpot
pwsh ./cluedin.ps1 packages custom -addfeed cluedin -uri <cluedinfeed> -user <username> -pass <password>
pwsh ./cluedin.ps1 packages custom -restore
pwsh ./cluedin.ps1 packages custom up

Create Organization

Once an instance of CluedIn is running the createorg command can be used to register new organizations

1. Organizations can be created using a simple name: cluedin.ps1 createorg -name <orgname>
   • A generated email address of admin@<orgname>.com would be created.
   • A generated password of P@ssword!123 will be used
2. The email can be specified with the -email parameter
3. The password can be specified with the -pass parameter
4. The createorg command accepts an optional environment name to configure the org for that environment e.g. cluedin.ps1 createorg custom -name <orgname> will create the org in the custom environment.

Examples

pwsh ./cluedin.ps1 createorg -name MyOrg #=> Creates a [email protected] user and myadmin org
pwsh ./cluedin.ps1 createorg -name MyOrg -email [email protected] #=> Creates the org with a specified admin user
pwsh ./cluedin.ps1 createorg -name MyOrg -pass LetMe!n1234 #=> Creates the org with a specified admin password
pwsh ./cluedin.ps1 createorg custom -name MyOrg #=> Creates the org for the 'custom' environment

Feature Release - 3.0.1

New Features

• Richer collection of environment variables are now avaliable
  • CLUEDIN_<servicename>_HOST - configuration of an alternative host for a service
  • CLUEDIN_<servicename>_LOCALPORT - configuration of an alternative local port for a service
  • CLUEDIN_DOMAIN - configuration of an alternative domain for the application (e.g. an external ip address)
• env configures environments in an isolated folder
• Emails are persisted to disk by default (under the environment folder)
• New version command to identify current version of script
• check and status receive an environment to check the correct service configuration

Fixes

• Improved output for terminals that do not support emoji
• When creating an environment all default environment configuration is cloned
• up now correctly disables services when specified (e.g. pwsh cluedin up -disable server)
• OpenRefine included in deployment
• Port numbers and domains listed in check

Initial Release - 3.0.0

v3.0.0

Release 3.0.0