docs: add readme and update hld

[mrickettsk]

🤖AEP PR SUMMARY🤖

• docs/mkdocs/aep-docs/docs/img/component-hld.jpg

  • Binary file was updated.

• pipeline-integration/azure-devops/readme.md

  • New file added for integrating AEP with Azure DevOps.
  • Instructions and templates provided for setting up pull request trigger within the repository.
  • Guidelines for setting up secrets securely in Azure DevOps and creating a new pipeline.

[github-actions]

General Recommendations

Documentation Improvements

1. Elaborate on the YML Template Files: The provided readme.md introduces multiple YML templates (ado_pullrequest_reviewer.yml, ado_pullrequest_summarise.yml, etc.) without detailing what their structures look like. A high-level overview or a link to a detailed document about each template's functionality and structure can enhance understanding. For example:

   For more details on the structure and parameters required for each template, see Template Documentation.

2. Include Version Compatibility: Mention which versions of Azure DevOps these configurations and templates have been tested with. This ensures users know if the documentation is compatible with their setup. For example:

   Compatibility: Tested with Azure DevOps Services as of January 2023. Ensure to check compatibility with your version.

3. Guidance on Secret Rotation: While setting up secrets is mentioned, there should be a note or link regarding best practices for rotating these secrets to maintain security. For instance:

   Remember to periodically rotate your `X_API_KEY` and `X_API_CONSUMER` keys as per [Secret Management Best Practices](https://learn.microsoft.com/en-us/azure/devops/pipelines/security/secret-management?view=azure-devops).

YML Structure & Azure DevOps Practices

1. Explicit Trigger Paths: In the GenAI-PR.yml, the triggers are disabled (trigger: none). If the objective is to run the pipeline based on pull request events, it's beneficial to specify branch paths to limit unnecessary builds. For Pull Requests, leveraging pr: triggers can be more explicit. Example modification:

   trigger: none
   pr:
     branches:
       include:
       - main
       - develop

2. Utilize Pipeline Caching: To improve build times and reduce costs, consider utilizing caching for dependencies if your pipeline includes steps for building or testing. For example, adding a cache step for Node modules:

   steps:
   - task: Cache@2
     inputs:
       key: 'npm | \"$(Agent.OS)\" | package-lock.json'
       path: $(npm_config_cache)
       cacheHitVar: CACHE_RESTORED
     displayName: Cache NPM dependencies

3. Security Linting: It's recommended to include a security linting step in your pipeline to catch common security issues early. Depending on the technology stack, tools like SonarQube, OWASP Dependency Check, or GitHub's CodeQL can be integrated. For instance:

   steps:
   - script: |
       echo \"Running security checks...\"
       # Example: Running a security linter or vulnerability scan.
     displayName: 'Security Checks'

4. Review Redundancies: The ado_pullrequest_reviewer_boards.yml is mentioned twice with different described functionalities. It's crucial to ensure there's no duplication or confusion about template purposes:

   -The `ado_pullrequest_reviewer_boards.yml` template is used to summarise the changes...
   +Ensure each template has a unique purpose and naming convention to avoid confusion.

Estimated Cost and Carbon Usage Considerations

• Utilizing pipeline caching can reduce the compute time needed for each run, slightly lowering costs and the associated carbon footprint. Although specific numbers depend on the frequency and complexity of your builds, consistent optimization can lead to noticeable savings and environmental benefits over time.

• Regularly reviewing and optimizing your CI/CD pipeline configurations can lead to more efficient use of server resources. Minimizing unnecessary builds, and ensuring resources are only used when needed, supports a cost-effective and environmentally friendly approach to DevOps.