Update robots.txt for Improved SEO and Documentation Visibility

[loadinglit]

This pull request addresses fixes #1960 of taipy by updating the robots.txt file in the taipy-doc repository to enhance search engine optimization (SEO) and user experience.

• SEO Improvement: Allowed search engines to index the latest documentation and version 4.0, ensuring users can easily access
  the most up-to-date resources.
• Disallowed indexing of older documentation versions (3.0, 3.1, 2.0, 2.1, 2.2, 2.3, 2.4) to reduce confusion and direct traffic to the current content.

These modifications are essential for maintaining clarity in documentation access and ensuring users find the most relevant information.

[FabienLelaquais]

Great!

[FabienLelaquais]

.gitignore is gone?

[loadinglit]

@jrobinAV @FabienLelaquais

• Disallowed release version 4.0 in robots.txt
• fixed the faulty irregularity of gitignore

[github-actions]

This PR has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this PR, then please add new commit or another comment, otherwise this PR will be closed in 14 days. For more information please refer to the contributing guidelines.

[jrobinAV]

Another Pull request has been proposed before this one to solve the same issue.
#1154

I propose to close this one.