see: Changelog
Please note that several packages are available which are pointed to in the installation documentation.
| Links | |
|---|---|
| Documentation | https://cloudmesh.github.io/cloudmesh-manual |
| Code | https://github.com/cloudmesh/cloudmesh-storage |
| Instalation Instructions | https://github.com/cloudmesh/cloudmesh-installer |
An dynamically extensible CMD based command shell. For en extensive documentation please see
where we also document how to use pyenv.
The user can install cloudmesh storage with
pip install cloudmesh-storageDevelopers want the source code and do the developement in the
directory. We use the cloudmesh-installer to do this. Make sure you
use a virtual environment such as ~/ENV3. We recommend using pythin
3.8.2
mkdir cm
cd cm
pip install cloudmesh-installer -U
cloudmesh-installer get storageIn case you need to reinstall the venv you can use
cloudmesh-installer new ~/ENV3 storage --python=python3.8
PLease note to use the python program for your system. THis may be different.
Some older Provider documentation is located at
TODO: you will be responsible for merging needed documentation form the old documentation and delete it if it is merged and no longer needed in the old documentation.
As we are developping multicloud environments we like to reuse and make some of the functionallity similar between clouds. for this reason we have implemented a provider concept that reads the credentials for each storage location from a cloudmesh yaml file.
Then based on the cms.kind of this storage location a Provider is
picked and we can interact with the system.
This is documented for the compute provider at
To simplify the task for storage we still define everything in one directory. However in future we will break it up and move each provider to its specific provider directory so that clouds are clearly seperated. THis move is not conducted by you but by Gregor, so we preserve the git history. He has a special git command that does this. So please for now develop in thsi directory if not already given other instructions.
Current architecture concepts: Cloudmesh Storage Concepts
-
TODO: Link to the document with the parallel architetcture is missing
-
Parallel Queue
See: Storage Commands
- TODO: If a command is missing let Gregor know and we work on integrating it.
See: Storage API
- TODO: If your docstrings are incomplete or incorrect please fix
- TODO: If your API is missing please contact Gregor
The API for storage will likely have cloudmesh.storage
HOwever if your API is in a provider specific directory, please add it to the documentation here. In that case communicate with Gregor for guidance.
In addition to the manual pages, they are also available as API so you can directly access the commands from teh API.
Please see:
TODO: Please develop an example to showcase how your code can be used form JUpyter
see: Jupyter example for compute
for an example for using compute providers.
TODO: Please also provde an example on hoe t use your provider directly. This can be most likely copied from a pytset.
The tests are included in the test directory. For the tests to work,
you need to first set the storage provider with
$ cms set storage=awss3
you can select the following storage providers:
- awss3
- gdrive
- azureblob
- box
Recently we are working on several new storage providors that improve the performance. This is based on using a parallel queue to organize the transfers.
- oracle
- parallelawss3
These new providers will eventually replace the existing providers.
Please note that if you find more or different tests that need to be added add them to the tests in the test/. directory so they work accross all storage providers. If special tests are needed for a specific provider they ban be added to the specific sub directory. Please add here a description why this specil test is needed. Please remove all tests that are already covered by the tests in the main tets directory form these special tests.