changelog.txt

=========
Changelog
=========

The **signac** package follows `semantic versioning <https://semver.org/>`_.

Version 2
=========

[2.2.0] -- 2023-xx-xx
---------------------

Changed
+++++++

 - linked views now can contain spaces and other characters except directory separators (#926).

[2.1.0] -- 2023-07-12
---------------------

Changed
+++++++

 - signac's "synced collections" backend used for reading/writing JSON files has been moved to a separate package called ``synced_collections``. See https://github.com/glotzerlab/synced_collections (#924).

[2.0.0] -- 2023-03-30
---------------------

Added
+++++

 - Official support for Python 3.11 (#875).
 - ``H5Store`` related errors are now included in the public API (#775).
 - Users can now access the project to which a job belongs with the ``Job.project`` attribute (#808).

Changed
+++++++

 - Functions raising ``AssertionError`` now raise ``RuntimeError`` (#612).
 - State points and documents require keys to be of type ``str`` (#719).
 - The keyword ``_id`` of the Job constructor has been renamed to ``id_`` (#681).
 - The signac schema to use version 2. This update involves a number of changes:
   - The project configuration is now stored at .signac/config instead of signac.rc or .signacrc (#678).
   - Schema versions are now identified by simple integers rather than semantic versioning style versions (#688).
   - Projects no longer have names and are identified solely by their root directories. This change also affects a number of public APIs where names are including, most prominently ``signac.init_project`` and ``Project.init_project``. Projects can now be constructed with just a root directory rather than a preloaded config (#677, #684, #706).
   - Project workspaces are no longer configurable, but are instead always defined as a subdirectory of the Project's root directory called ``workspace`` (#714).
 - Rather than searching upwards until the root, ``load_config`` will only load configuration files in the specified directory, which is assumed to be a project directory, as well as the user's home directory (#711).
 - Changed the ``root`` parameter to ``path`` in the ``signac.get_project`` and ``signac.init_project`` functions and corresponding ``Project`` methods (#757, #758).
 - The prefix argument to ``$ signac view`` is now optional and can be provided with ``-p/--prefix`` (#653, #774).
 - Tests are run with ``xfail_strict = True`` (#850).
 - Detection of an invalid config will raise an error rather than a debug log (#855).
 - The package namespace has been flattened so that most functionality is directly available in the ``signac`` namespace (#756, #868).
 - The ``calc_id`` function has been moved from the ``hashing`` module to the ``job`` module (#873).
 - Tests are run with all warnings treated as errors (#871).
 - The contents of the ``syncutil`` module have been moved into ``sync`` (#869).
 - The project now leverages a fully pyproject.toml-driven build (#878).

Removed
+++++++

 - The pre-SyncedCollection synchronized dictionary classes, including SyncedDict, SyncedAttrDict, and JSONDict (#577).
 - The old custom JSON encoder and dumps wrapper (#577).
 - The MPIPool and the filesystems.py module (#575).
 - The following Project methods: ``get_id``, ``build_job_search_index``, ``build_job_statepoint_index``, ``find_job_ids``, ``reset_statepoint``, ``update_statepoint``, ``create_access_module``, ``index``, ``dump_statepoints``, ``get_statepoint``, ``read_statepoints``, ``write_statepoints``, ``groupbydoc``, ``root_directory``, ``num_jobs`` (#574, #593, #599, #601, #752).
 - The following Job methods: ``get_id``, ``workspace``, ``ws``, ``reset_statepoint`` (#578, #752).
 - The ``syncutil.copytree`` method (#581).
 - All Crawlers, including ``RegexFileCrawler``, ``MainCrawler``, ``MasterCrawler``, ``SignacProjectCrawler``, and ``BaseCrawler``, in addition to all associated functionality in indexing.py (#580).
 - The cite.py module (#594).
 - The following types of exceptions: ``AuthenticationError``, ``ExportError`, ``FetchError``, ``PermissionsError`` (#595, #596).
 - All MongoDB database functionality, including the ``db`` subpackage, the ``connection``, ``crypt``, and ``host`` submodules in the ``common`` subpackage, and all associated configuration information (#576, #596).
 - The ``JobSearchIndex`` class (#587).
 - The ability to pass indexes to various ``Project`` methods (#599).
 - The following ``JobsCursor`` methods: ``groupbydoc``, ``next`` (#601, #604).
 - The ``Project.config`` property is no longer mutable. Use the command line ``$ signac config`` to modify configuration (#608, #246, #244).
 - The config module and all its functions, all of which have been made private (#674, #753, #789, #847, #877).
 - ``Project`` subclasses can no longer define a ``Job`` subclass to use (#588, #693).
 - The ``Collection`` class (#664, #667, #683).
 - The ``project`` CLI subcommand (#752).
 - The ``--workspace`` option for the command line ``job`` subcommand (#752).
 - The ability to call ``Project.workspace``, it is strictly a property now (#752).
 - ``ProjectSchema.__call__``, ``ProjectSchema.detect`` (#752).
 - The ``--doc-filter`` option for several command line subcommands (#613, #795).
 - The public API of the ``synced_collection`` subpackage (#807, #790).
 - The warnings module (#815).
 - The signac.common.validate module (#853).
 - The ``Config`` class (#860).
 - The ``DictManager`` class (#858).
 - The ``signac.testing`` module (#863).
 - Zenodo is no longer updated (#903).

Version 1
=========

[1.8.0] -- 2022-10-05
---------------------

Added
+++++

 - Official support for Python 3.10 (#631).
 - Benchmarks can be run using the ``asv`` (airspeed velocity) tool (#629).
 - Continuous integration tests run in parallel with ``pytest-xdist`` (#705).
 - The ``Project.path`` and ``Job.path`` properties (#685).

Changed
+++++++

 - Schema migration is now performed on directories rather than signac projects and supports a wider range of schemas (#654).
 - Deprecated features now use ``FutureWarning`` instead of ``DeprecationWarning``, which is hidden by default (#687, #691, #692).
 - Project names have a default in anticipation of removing names entirely. Project names will be removed in signac 2.0 (#644).
 - ``Project.workspace`` is now a property, not a method (#685).
 - Continuous integration uses GitHub Actions instead of CircleCI (#776, #788).
 - Raise errors in testing when ``DeprecatedWarning`` or ``FutureWarning`` is raised (#713).
 - Change GitHub PR to check for uncompleted tasks (i.e. unchecked checkboxes) (#686).

Deprecated
++++++++++

 - ``Project`` methods ``read_statepoints``, ``write_statepoints``, and ``dump_statepoints`` are deprecated (#579, #197).
 - ``Project.index`` method is deprecated (#591, #588).
 - ``JobSearchIndex`` class is deprecated (#600).
 - ``index`` argument is deprecated in ``Project`` methods (#602, #588).
 - ``signac.cite`` module is deprecated (#611, #592).
 - The ``config`` module and all its methods are deprecated (#675, #753, #814).
 - Accessing ``Project.workspace`` as a method, it should be accessed as a property (#685).
 - ``Project.num_jobs`` (#685).
 - ``ProjectSchema.__call__``, ``ProjectSchema.detect`` (#685).

Fixed
+++++

 - ``H5Store.mode`` returns the file mode (#607).
 - User-provided path functions now raise an error if not unique (#666).
 - ``Collection`` class no longer raises an error when searching by a primary key that does not exist (#676).
 - Relative paths on Windows are not used if the current directory has no common prefix (#777).
 - ``get_project()`` now raises an error if provided a root directory that does not exist (#779, #792).
 - Catch internally raised warnings on use of deprecated password cache (#754).
 - Catch ``KeyError`` from multithreading error (#710).
 - Tests now properly show raised warnings (#603).

Removed
+++++++

 - Removed upper bound of Python 4 on ``python_requires`` (#780, #781).
 - Dropped support for Python 3.6 and Python 3.7 (#715) following the recommended support schedules of `NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`__.
 - Dropped dependency on ``deprecation`` package (#687, #718).
 - Removed unused ``_extract`` utility function to avoid CVE-2007-4559 (#829).

[1.7.0] -- 2021-06-08
---------------------

Added
+++++

 - New ``SyncedCollection`` class and subclasses to replace ``JSONDict`` with more general support for different types of resources (such as MongoDB collections or Redis databases) and more complete support for different data types synchronized with files (#196, #234, #249, #316, #383, #397, #465, #484, #529, #530). This change introduces a minor-backwards incompatible change; for users making direct use of signac buffering, the ``force_write`` parameter is no longer respected. If the argument is passed, a warning will now be raised to indicate that it is ignored and will be removed in signac 2.0.
 - Unified querying for state point and document filters using 'sp' and 'doc' as prefixes (#332, #514). This change introduces a minor backwards-incompatible change to the ``Collection`` index schema ('statepoint'->'sp'), but this does not affect any APIs, only indexes saved to file using a previous version of signac. Indexing APIs will be removed in signac 2.0.

Changed
+++++++

 - Optimized internal path joins to speed up project iteration (#515).

Deprecated
++++++++++

 - ``doc_filter`` arguments, which are replaced by namespaced filters.
 - The modules ``signac.core.attrdict``, ``signac.core.json``, ``signac.core.jsondict``, and ``signac.core.synceddict.py`` are deprecated in favor of the new ``SyncedCollection`` classes and will be removed in signac 2.0 (#483).

Fixed
+++++

 - Corrected docstrings for ``Job.update_statepoint`` and ``Project.update_statepoint`` (#506, #563).


[1.6.0] -- 2021-01-24
---------------------

Added
+++++

 - Implemented ``JobsCursor.__contains__`` check (#449).
 - Added documentation for ``JobsCursor`` class (#475).

Changed
+++++++

 - Optimized job hash and equality checks (#442, #455).
 - Optimized ``H5Store`` initialization (#443).
 - State points are loaded lazily when ``Job`` is opened by id (#238, #239).
 - Optimized ``Job`` and ``Project`` classes to cache internal properties and initialize on access (#451).
 - Python 3.6 is only tested with oldest dependencies (#474).
 - Improved documentation for updating and resetting state points (#444).

Deprecated
++++++++++

 - Deprecate ``syncutil.copytree`` method (#439).

Fixed
+++++

 - Zero-dimensional NumPy arrays can be used in state points and documents (#449).


[1.5.1] -- 2020-12-19
---------------------

Added
+++++

 - Support for h5py version 3 (#411).
 - Added pyupgrade to pre-commit hooks (#413).
 - Code is formatted with ``black`` and ``isort`` pre-commit hooks (#415).
 - Added macOS to CircleCI testing pipeline (#281, #414).
 - Official support for Python 3.9 (#417).

Changed
+++++++

 - Optimized internal function ``_mkdir_p`` (#421).
 - Optimized performance of job initialization (#422).
 - Optimized performance of buffer storage (#428).
 - Optimized performance of creating/loading synced data structures (#429).


[1.5.0] -- 2020-09-20
---------------------

Added
+++++

 - Type annotations are validated during continuous integration (#313).
 - Added ``_repr_html_`` method in ``ProjectSchema`` class (#314, #324).
 - Allow grouping by variables that are not present in all jobs in the project in ``JobsCursor.groupby`` (#321, #323).
 - Added parameters ``usecols`` and ``flatten`` to allow selection of columns and flattening of nested data when converting signac data into a pandas DataFrame (#327, #330).
 - Added support for `pre-commit hooks <https://pre-commit.com/>`__ (#355, #358).
 - Expanded CLI documentation (#187, #359, #377).

Changed
+++++++

 - Docstrings are now written in `numpydoc style <https://numpydoc.readthedocs.io/en/latest/format.html>`__.

Fixed
+++++

 - Fix the ``signac config verify`` command (previously broken) (#301, #302).
 - Warnings now appear when raised by the ``signac`` CLI (#317, #308).
 - Fix dots in synchronization error messages (#375, #376).

Deprecated
++++++++++

 - Deprecate the ``create_access_modules`` method in ``Project``, to be removed in 2.0 (#303, #308).
 - The ``MainCrawler`` class has replaced the ``MasterCrawler`` class. Both classes are deprecated (#342).

Removed
+++++++

 - Dropped support for Python 3.5 (#340). The signac project will follow the `NEP 29 deprecation policy <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_ going forward.
 - Removed dependency on ``pytest-subtests`` (#379).


[1.4.0] -- 2020-02-28
---------------------

Added
+++++

 - Added Windows to platforms tested with continuous integration (#264, #266).
 - Add command line option ``-m/--merge`` for ``signac sync`` (#280, #230).

Changed
+++++++

 - Workspace directory is created when ``Project`` is initialized (#267, #271).
 - Changed testing framework from ``unittest`` to ``pytest`` (#212, #275).
 - Refactored internal use of deprecated ``get_statepoint`` function (#227, #282).

Fixed
+++++

 - Fixed issues on Windows with ``H5Store``, project import/export, and operations that move files (#264, #266).
 - Calling ``items`` or ``values`` on ``_SyncedDict`` objects does not mutate nested dictionaries (#234, #269).
 - Fixed issue with ``project.data`` access from separate instances of ``H5StoreManager`` (#274, #278).
 - Fixed error when launching ``signac shell`` if permissions are denied for ``.signac_shell_history`` (#279).

Removed
+++++++

 - Removed vendored ``tqdm`` module and replaced it with a requirement (#289).
 - Removed support for ``rapidjson`` as an alternative JSON library (#285, #287).
 - Removed tuple of keys implementation of nested dictionaries (#272, #296).


[1.3.0] -- 2019-12-20
---------------------

Added
+++++

 - Official support for Python 3.8 (#258).
 - Add properties ``Project.id`` and ``Job.id`` (#250).
 - Add ``signac.diff_jobs`` function and ``$ signac diff`` CLI command to compare two or more state points (#248, #247).
 - Add function to initialize a sample data space for testing purposes (#215).
 - Add schema version to ensure compatibility and enable migrations in future package versions (#165, #253).

Changed
+++++++

 - Implemented ``Project.__contains__`` check in constant time (#231).

Fixed
+++++

 - Attempting to create a linked view for a Project on Windows now raises an informative error message (#214, #236).
 - Project configuration is initialized using ConfigObj, allowing the configuration to include commas and special characters (#251, #252).

Deprecated
++++++++++

 - Deprecate the ``get_id`` method in ``Project`` and ``Job`` classes in favor of the ``id`` property, to be removed in 2.0 (#250).
 - In-memory modification of the project configuration, to be removed in 2.0 (#246).

Removed
+++++++

 - Dropped support for Python 2.7 (#232).


[1.2.0] -- 2019-07-22
---------------------

Added
+++++

 - Keep signac shell command history on a per-project basis (#134, #194).
 - Add ``read_json()`` and ``to_json()`` methods to ``Collection`` class (#104, #200).

Fixed
+++++

 - Fix issue where shallow copies of instances of ``Job`` would behave incorrectly (#153, #207).
 - Fix issue causing a failure of the automatic conversion of valid key types (#168, #205).
 - Improve the 'dots in keys' error message to make it easier to fix related issues (#170, #205).
 - Update the ``__repr__`` and ``__repr_html__`` implementations of the ``Project``, ``Job``, and ``JobsCursor`` classes (#193).
 - Reduce the logging verbosity about a missing default host key in the configuration (#201).
 - Fix issue with incorrect detection of dict-like files managed with the ``DictManager`` class (e.g. ``job.stores``) (#203).
 - Fix issue with generating views from the command line for projects with only one job (#208, #211).
 - Fix issue with heterogeneous types in state point values that are lists (#209, #210).

Deprecated
++++++++++

 - Support for Python 2.7 is deprecated with this version and will be removed in the next minor release. See the `Python 3 Statement <https://python3statement.org/>`_.


[1.1.0] -- 2019-05-19
---------------------

Added
+++++

 - Add command line options ``--sp`` and ``--doc`` for ``signac find`` that allow users to display key-value pairs of the state point and document in combination with the job id (#97, #146).
 - Improve the representation (return value of `repr()`) of instances of ``H5Group`` and ``SyncedAttrDict``.

Fixed
+++++

 - Fix: Searches for whole numbers will match all numerically matching integers regardless of whether they are stored as decimals or whole numbers (#169).
 - Fix: Passing an instance of dict to `H5Store.setdefault()` will return an instance of `H5Group` instead of a dict (#180).
 - Fix error with storing numpy arrays and scalars in a synced dictionary (e.g. `job.statepoint`, `job.document`) (#184).
 - Fix issue with ``ResourceWarning`` originating from unclosed instance of ``Collection`` (#186).
 - Fix issue with using the ``get_project()`` function with a relative path and ``search=False`` (#191).

Removed
+++++++

 - Support for Python version 3.4 (no longer tested).


[1.0.0] -- 2019-02-28
---------------------

Added
+++++

 - Official support for Python 3.7.
 - The ``H5Store`` and ``H5StoreManager`` classes, which are useful for storing (numerical) array-like data with an HDF5-backend. These classes are exposed within the root namespace.
 - The ``job.data`` and ``project.data`` properties which present an instance of ``H5Store`` to access numerical data within the job workspace and project root directory.
 - The ``job.stores`` and ``project.stores`` properties, which present an instance of ``H5StoreManager`` to manage multiple instances of ``H5Store`` to store numerical array-like data within the project workspace and project root directory.
 - The ``signac.get_job()`` and the ``signac.Project.get_job()`` functions that allow users to get a job handle by switching into or providing the job's workspace directory.
 - The ``job`` variable is automatically set when opening a ``signac shell`` from within a job's workspace directory.
 - Add the ``signac shell -c`` option which allows the direct specification of Python commands to be executed within the shell.
 - Automatic cast of ``numpy`` arrays to lists when storing them within a ``JSONDict``, e.g., a ``job.statepoint`` or ``job.document``.
 - Enable ``Collection`` class to manage collections stored in compressed files (gzip, zip, etc.).
 - Enable deleting of ``JSONDict`` keys through the attribute interface, e.g., ``del job.doc.foo``.
 - Pretty HTML representation of instances of ``Project`` and ``JobsCursor`` targeted at Jupyter Notebooks (requires pandas, automatically enabled when installed).
 - The ``to_dataframe()`` function to export the job state point and document data of a ``Project`` or a ``JobsCursor``, e.g., the result of ``Project.find_jobs()``, as a ``pandas.Dataframe`` (requires pandas).

Changed
+++++++

 - Dots (`.`) in keys are no longer allowed for ``JSONDict`` and ``Collection`` keys (previously deprecated).
 - The ``JSONDict`` module is exposed in the root namespace, which is useful for storing text-serializable data  with a JSON-backend similar to the ``job.statepoint`` or ``job.document``, etc.
 - The ``Job.init()`` method returns the job to allow one-line job creation and initialization.
 - The ``search`` argument was added to the ``signac.get_project()`` function, which when True (the default), will cause signac to search for a project within *and above* a specified root directory, not only within the root directory. The behavior without any arguments remains unchanged.

Fixed
+++++

 - Fix ``Collection.update()`` behavior such that existing documents with identical primary key are updated. Previously, a ``KeyError`` would be raised.
 - Fix issue where the ``Job.move()`` would trigger a confusing ``DestinationExists`` exception when trying to move jobs across devices / file systems.
 - Fix issue that caused failures when the ``python-rapidjson`` package is installed. The ``python-rapidjson`` package is used as the primary JSON-backend when installed.
 - Fix issue where schema with multiple keys would subset incorrectly if the list of jobs or statepoints was provided as an iterator rather than a sequence.

Removed
+++++++

 - Removes the obsolete and deprecated ``core.search_engine`` module.
 - The previously deprecated ``Project.find_statepoints()`` and ``Project.find_job_documents()`` functions have been removed.
 - The ``Project.find_jobs()`` no longer accepts the obsolete ``index`` argument.


Version 0.9
===========

[0.9.5] -- 2019-01-31
---------------------

Fixed
+++++

 - Ensure that the next() function can be called for a JobsIterator, e.g., `project.find()`.
 - Pickling issue that occurs when a _SyncedDict (job.statepoint, job.document, etc.) contains a list.
 - Issue with the readline module that would cause signac shell to fail on Windows operating systems.


[0.9.4] -- 2018-10-24
---------------------

Added
+++++

 - Adds the ``$ signac import`` command and the ``Project.import_from()`` method for the import of data spaces into a project workspace, such as a directory, a tarball, or a zip file.
 - Adds the ``$ signac export`` command and the ``Project.export_to()`` method for the export of project workspaces to an external location, such as a directory, a tarball, or a zip file.
 - Adds functionality for the rapid initialization of temporary projects with the ``signac.TemporaryProject`` context manager.
 - Adds the ``signac.Project.temporary_project()`` context manager which creates a temporary project within the root project workspace.
 - Add ``signac`` to the default namespace when invoking ``signac shell``.
 - Add option to specify a custom view path for the ``signac view``/ ``Project.create_linked_view()`` function.
 - Iterables of documents used to construct a Collection no longer require an ``_id`` field.

Changed
+++++++

 - The default path for linked views has been adjusted to match the one used for data exports.

Fixed
+++++

 - Fix issue where differently typed integer values stored within a Collection under the same key would not be indexed correctly. This issue affected the correct function of the `$type` operator for aforementioned cases and would lead to incorrect types in the Project schema detection algorithm for integer values.
 - Fix issue where jobs that are migrated (state point change), but are not initialized, were not properly updated.
 - Fix issue where changes to lists as part of synchronized dictionary, for example a state point or document would not be saved.
 - Fix non-deterministic issue occuring on network file systems when trying to open jobs where the user has no write access to the job workspace directory.

[0.9.3] -- 2018-06-14
---------------------

Added
+++++

 - Add `$near` operator to express queries for numerical values that match up to a certain precision.
 - Add the `$ signac shell` sub command to directly launch a Python interpreter within a project directory.

Fixed
+++++

 - Fix issue where a job instance would not be properly updated after more than one state point reset.

[0.9.2] -- 2017-12-18
---------------------

Added
+++++

  - Add provisional feature (persistent state point caching); calling the ``Project.update_cache()`` method will generate and store a persistent state point cache in the project root directory, which will increase the speed of many project iteration, search, and selection operations.
  - Add ``Project.check()`` method which checks for workspace corruption, but does not make any attempt to repair it.
  - The ``Project.repair()`` method will attempt to repair jobs, that have been corrupted by manually renaming the job's workspace directory.

Changed
+++++++

  - Enable the *write_concern* flag for the ``job.document``.
  - Allow to omit the specification of an authentication mechanism in the MongoDB host configuration.

Fixed
+++++

  - Fix critical issue in the ``JSONDict`` implementation that would previously overwrite the underlying file when attempting to store values that are not JSON serializable.
  - Fix issue where the ``Project.export()`` function would ignore the update argument when the index to export to would be a MongoDB collection.

[0.9.1] -- 2017-11-07
---------------------

Fixed
+++++

  - Fix critical issue in the ``SyncedAttrDict`` implementation that would previously overwrite the underlying file if the first operation was a ``__setitem__()`` operation.

[0.9.0] -- 2017-10-28
---------------------

Added
+++++

  - Introduction of ``$ signac sync``, ``Project.sync()``, and ``Job.sync()`` for the simplified and fine-grained synchronization of multiple project data spaces.
  - Introduction of ``$ signac schema`` and ``Project.detect_schema()`` for the automatic detection of the implicit and semi-structured state point schema of a project data space.
  - Simplified aggregation of jobs over projects and ``Project.find_jobs()`` results with the ``Project.groupby()`` function.
  - Support for project-centralized data with the ``Project.document`` attribute and the ``Project.fn()`` method for the wrapping of filenames within the project root directory.
  - Added the ``Job.clear()`` and the ``Job.reset()`` methods to clear or reset a job's workspace data.

Changed
+++++++

  - Both ``Job.statepoint`` and ``Job.document`` now use the same underlying data structure and provide the exact same API (copy with ``()`` and access of keys as attributes).
  - The ``Collection`` class uses an internal counter instead of UUIDs for the generation of primary keys (resulting in improved performance).
  - Major performance improvements (faster Collection, improved caching)
  - Overhaul of the reference documentation.

Version 0.8
===========

[0.8.7] -- 2017-10-05
---------------------

Fixed
+++++

 - Fix an issue where the creation of linked views was non-deterministic in some cases.
 - Fix an issue where the creation of linked views would fail when the project contains job with state points that have lists as values.

[0.8.6] -- 2017-08-25
---------------------

Fixed
+++++

 - Fix Collection append truncation issue (see issue #66).

[0.8.5] -- 2017-06-07
---------------------

Changed
+++++++

 - The signac ids in the `signac find --show` view are no longer enclosed by quotation marks.

Fixed
+++++

 - Fix compatibility issue that broke the `signac find --view` and all `--pretty` commands on Python 2.7.
 - Fix issue where view directories would be incomplete in combination with heterogeneous state point schemas.

[0.8.4] -- 2017-05-19
---------------------

Added
+++++

 - All search queries on project and collection objects support various operators including: *$and*, *$or*, *$gt*, *$gte*, *$lt*, *$lte*, *$eq*, *$ne*, *$exists*, *$regex*, *$where*, *$in*, *$nin*, and *$type*.
 - The ``$ signac find`` command supports a simple filter syntax, where key value pairs can be provided as individual arguments.
 - The ``$ signac find`` command is extended by a *--show* option, to display the state point and the document contents directly. The contents are truncated to an adjustable depth to reduce output noise.
 - The ``$ signac view`` command has an additional filter option to select a sub data space directly without needing to pipe job ids.
 - The new ``$ signac document`` command can be used to display a job's document directly.

Changed
+++++++

 - Minor performance improvements.

[0.8.3] -- 2017-05-10
---------------------

Changed
+++++++

 - Raise ``ExportError`` when updating with an empty index.

Fixed
+++++

 - Fix command line logic issue with ``$signac config host``.
 - Fix bug, where ``Collection.replace_one()`` would ignore the upsert argument under specific conditions.

[0.8.2] -- 2017-04-19
---------------------

Fixed
+++++

 - Fixes a ``TypeError`` which occurred under specific conditions when calling ``Collection.find()`` with nested filter arguments.

[0.8.1] -- 2017-04-17
---------------------

Fixed
+++++

 - Fixes wide-spread typo (*indeces* -> *indexes*).

[0.8.0] -- 2017-04-16
---------------------

Overall major simplification of the generation of indexes and the management and searching of index collections without external database.

Added
+++++

 - Introduction of the ``Collection`` class for the management of document collections, such as indexes in memory and on disk.
 - Generation of file indexes directly via the ``signac.index_files()`` function.
 - Generation of main indexes directly via the ``signac.index()`` function and the ``$ signac index`` command.
 - The API of ``signac_access.py`` files has been simplified, including the possibility to use a blank file for a minimal configuration.
 - Use the ``$ signac project --access`` command to create a minimal access module in addition to ``Project.create_access_module()``.
 - The update of existing index collections has been simplified by using the ``export()`` function with the `update=True` argument, which means that stale documents (the associated file or state point no longer exists) are automatically identified and removed.
 - Added the ``Job.ws`` attribute, as short-cut for ``Job.workspace()``.
 - The ``Job.sp`` interface has a ``get()`` function which can be used to specify a default value in case that the requested key is not part of the state point.

Changed (breaking API)
++++++++++++++++++++++

 - The ``$ signac index`` command generates a main index instead of a project index. To generate a project index from the command line use ``$ signac project --index`` instead.
 - The ``SignacProjectCrawler`` class expects the project's root directory as first argument, not the workspace directory.
 - The ``get_crawlers()`` function defined within a ``signac_access.py`` access module is expected to yield crawler instances directly, not a mapping of crawler ids and instances.
 - The simplification of the ``signac_access.py`` module API is reflected in a reduction of arguments to the ``Project.create_access_module()`` method.

Changed (non-breaking)
++++++++++++++++++++++

 - The ``RegexFileCrawler``, ``SignacProjectCrawler`` and ``MainCrawler`` classes were moved into the root namespace.
 - If a ``MainCrawler`` object is instantiated with the ``raise_on_error`` argument set to True, any errors encountered during crawling are raised instead of ignored and skipped; this simplifies the debugging of erroneous access modules.
 - Improved error message for invalid configuration files.
 - Better error messages for invalid ``$ signac find`` queries.
 - Check a host configuration on the command line via ``$ signac host --test``.
 - A MongoDB database host configuration defaults to *none* when no authentication method is explicitly specified.
 - Using the ``--debug`` option in combination with ``$ signac index`` will show the traceback of errors encountered during indexing instead of ignoring them.
 - Instances of ``Job`` are hashable, making it possible to use them as dict keys for instance.
 - The representation of ``Job`` instances via ``repr()`` can actually serves as copy constructor command.
 - The project interface implementation performs all non-trivial search operations on an internally management index collection, which improves performance and simplifies the code base.

Deprecated
++++++++++

 - The ``DocumentSearchEngine`` class has been deprecated, its functionality is now provided by the ``Collection`` class.

Fixed
+++++

 - An issue related to exporting documents to MongoDB collections via pymongo in combination with Python 2.7 has been fixed.

Version 0.7
===========

[0.7.1] -- 2017-01-09
---------------------

Added
+++++

 - When the ``python-rapidjson`` package is installed, it will be used for JSON encoding/decoding (experimental).

Changed
+++++++

 - All job move-related methods raise ``DestinationExistsError`` in case of destination conflicts.
 - Optimized ``$ signac find`` command.

Fixed
+++++

 - Fixed bug in ``$ signac statepoint``.
 - Suppress 'broken pipe error' message when using ``$ signac find`` for example in combination with ``$ head``.

[0.7.0] -- 2017-01-04
---------------------

Added
+++++

 - Add support for Python version 3.6.
 - Add support for PyPy and PyPy3.
 - Simplified iteration over project data spaces.
 - An existing linked view can be updated by executing the view command again.
 - Add attribute interface for the access and modification of job state points: ``Job.sp``.
 - Add function for moving and copying of jobs between projects.
 - All project related iterators support the ``len``-operator.
 - Enable iteration over all jobs with: ``for job in project:``.
 - Make ``len(project)`` an alias for ``project.num_jobs()``.
 - Add ``in``-operator to determine whether a job is initialized within a project.
 - Add ``Job.sp`` attribute to access and modify a job's state point.
 - The ``Project.open_job()`` method accepts abbreviated job ids.
 - Add ``Project.min_len_unique_id()`` method to determine the minimum length of job ids to be unique within the project's data space.
 - Add ``Job.move()`` method to move jobs between projects.
 - Add ``Project.clone()`` method to copy jobs between projects.
 - Add ``$ signac move`` and ``$ signac clone`` command line functions.
 - Add ``Job.reset_statepoint()`` method to reset a job's state point.
 - Add ``Job.update_statepoint()`` method to update a job's state point.
 - Add a ``Job.FN_DOCUMENT`` constant which defines the default filename of the job document file
 - The ``$ signac find`` command accepts a ``-d/--doc-filter`` option to filter by job document contents.
 - Add the ``Project.create_linked_view()`` method as replacement for the previously deprecated ``Project.create_view()`` method.

Changed
+++++++

 - Linked views use relative paths.
 - The *Guide* documentation chapter has been renamed to *Reference* and generally overhauled.
 - The *Quick Reference* documentation chapter has been extended.

Fixed
+++++

 - Fix error when using an instance of ``Job`` after calling ``Job.remove()``.
 - A project created in one the standard config directories (such as the home directory) does not take prevalence over project configurations in or above the current working directory.

Removed
+++++++

 - The *signac-gui* component has been removed.
 - The ``Project.create_linked_view()`` ``force`` argument is removed.
 - The ``Project.find_variable_parameters()`` method has been removed

Version 0.6
===========

[0.6.2] -- 2017-12-15
---------------------

Added
+++++

 - Add instructions on how to acknowledge **signac** in publications to documentation.
 - Add cite module for the auto-generation of formatted references and BibTeX entries.

Removed
+++++++

 - Remove SSL authentication support.

[0.6.1] -- 2017-11-26
---------------------

Changed
+++++++

 - The ``Project.create_view()`` method triggers a ``DeprecationWarning`` instead of a ``PendingDeprecationWarning``.
 - The ``Project.find_variable_parameters()`` method triggers a ``DeprecationWarning`` instead of a ``PendingDeprecationWarning``.

Fixed
+++++

 - Make package more robust against PySide import errors.
 - Fix ``Project.__repr__`` method.
 - Fix critical bug in ``fs.GridFS`` class, which rendered it unusuable.
 - Fix issue in ``indexing.fetch()`` previously resulting in local paths being ignored.
 - Fix error ``signac.__all__`` namespace directive.

[0.6.0] -- 2016-11-18
---------------------

Added
+++++

 - Add the ``export_to_mirror()`` function for mirroring files.
 - Introduction of the ``signac.fs`` namespace to simplify the configuration of mirror filesystems.
 - Add *errors* module to root namespace. Many exceptions raised inherit from the base exception types defined within that module, making it easier to catch signac related errors.
 - Add the ``export_one()`` function for the export of a single index document; simplifies the implementation of custom export functions.
 - Opening an instance of ``Job`` with the ``open_job()`` method multiple times and entering a job context recursively is now well-defined behavior: Opening a job now adds the current working directory onto a stack, closing it switches into the directory on top of the stack.
 - The return type of ``Project.open_job()`` can be configured to make it easier to specialize projects with custom job types.

Changed
+++++++

 - The MainCrawler logic has been simplified; their primary function is the compilation of index documents from subcrawlers, all export logic, including data mirroring is now provided by the ``signac.export()`` function.
 - Each index document is now uniquely coupled with only one file or data object, which is why ``signac.fetch()`` replaces ``signac.fetch_one()`` and the latter one has been deprecated and is currently an alias of the former one.
 - The ``signac.fetch()`` function always returns a file-like object, regardless of format definition.
 - The format argument in the crawler ``define()`` function is now optional and has now very well defined behavior for str types. It is encouraged to define a format with a str constant rather than a file-like object type.
 - The ``TextFile`` file-like object class definition in the formats module has been replaced with a constant of type ``str``.
 - The ``signac.export()`` function automatically delegates to specialized implementations such as ``export_pymongo()`` and is more robust against errors, such as broken connections.
 - The ``export_pymongo()`` function makes multiple automatic restart attempts when encountering errors.
 - Documentation: The tutorial is now based on signac-examples jupyter notebooks.
 - The ``contrib.crawler`` module has been renamed to contrib.indexing to better reflect the semantic context.
 - The ``signac.export()`` function now implements the logic for data linking and mirroring.
 - Provide default argument for '--indent' option for ``$ signac statepoint`` command.
 - Log, but do not reraise exceptions during ``MainCrawler`` execution, making the compilation of main indexes more robust against errors.
 - The object representation of ``Job`` and ``Project`` instances is simplified.
 - The warning verbosity has been reduced when importing modules with optional dependencies.

Removed
+++++++

 - All modules related to the stale *conversion framework* feature have been removed resulting in a removal of the optional networkx dependency.
 - Multiple modules related to the *conversion framework* feature have been removed, including: ``contrib.formats_network``, ``contrib.conversion``, and ``contrib.adapters``.

Fixed
+++++

 - Opening instances of ``Job`` with the ``Job.open()`` method multiple times, equivalently entering the job context recursively, does not cause an error anymore, but instead the behavior is well-defined.

Version 0.5
===========

[0.5.0] -- 2016-08-31
---------------------

Added
+++++

 - New function: ``signac.init_project()`` simplifies project initialization within Python
 - Added optional ``root`` argument to ``signac.get_project()`` to simplify getting a project handle outside of the current working directory
 - Added optional argument to ``signac.get_project()``, to allow fetching of projects outside of the current working directory.
 - Added two class factory methods to `Project`: ``get_project()`` and ``init_project()``.

Changed
+++++++

 - The performance of project indexing and crawling has been improved.

Version 0.4
===========

[0.4.0] -- 2016-08-05
---------------------

Added
+++++

 - The performance of find operations can be greatly improved by using pre-generated job indexes.
 - New top-level commands: ``$ signac find``, ``$ signac index``, ``$ signac statepoint``, and ``$ signac view``.
 - New method: ``Project.create_linked_view()``
 - New method: ``Project.build_job_statepoint_index()``
 - New method: ``Project.build_job_search_index()``
 - The ``Project.find_jobs()`` method allows to filter by job document.

Changed
+++++++

 - The ``SignacProjectCrawler`` indexes all jobs, not only those with non-empty job documents.
 - The ``signac.fetch_one()`` function returns ``None`` if no associated object can be fetched.
 - The tutorial is restructured into multiple parts.
 - Instructions for installation are separated from the guide.

Removed
+++++++

 - Remove previously deprecated *crawl* keyword argument in index export functions.
 - Remove previously deprecated function common.config.write_config().

Version 0.3
===========

[0.3.0] -- 2016-06-23
---------------------

Added
+++++

 - Add contributing agreement and guidelines.

Changed
+++++++

 - Change license from MIT to BSD 3-clause license.

Version 0.2
===========

[0.2.9] -- 2016-06-06
---------------------

Added
+++++

 - Addition of the ``signac config`` command line API.
 - Password updates are encrypted with ``bcrypt`` when ``passlib`` is installed.
 - The user is prompted to enter missing credentials (username/password) in case that they are not stored in the configuration.
 - The ``$ signac confg`` tool provides the ``--update-pw`` argument, which allows users to update their own password.
 - Added MIT license, in addition, all source code files contain a short licensing header.

Changed
+++++++

 - Improved documentation on how to configure signac.
 - The OSI classifiers are updated, including an upgrade of the development status to '4 - beta'.

Fixed
+++++

 - Nested job state points can no longer get corrupted. This bug occurred when trying to operate on nested state point mappings.

Deprecated
++++++++++

 - Deprecated pymongo versions 2.x are no longer supported.

[0.2.8] -- 2016-04-18
---------------------

Added
+++++

 - ``Project`` is now in the root namespace.
 - Add ``index()`` method to `Project`.
 - Add ``create_access_module()`` method to ``Project``.
 - Add ``find_variable_parameters()`` method to ``Project``.
 - Add ``fn()`` method to ``Job``, which prepends the job's workspace path to a filename.
 - The documentation contains a comprehensive tutorial

Changed
+++++++

 - The ``crawl()`` function yields only the index documents and not a tuple of (_id, doc).
 - ``export()`` and ``export_pymongo()`` expect the index documents as first argument, not a crawler instance. The old API is still supported, but will trigger a DeprecationWarning.

[0.2.7] -- 2016-02-29
---------------------

Added
+++++

 - Add ``job.isfile()`` method

Changed
+++++++

 - Optimize ``project.find_statepoints()`` and ``project.repair()`` functions.

[0.2.6] -- 2016-02-20
---------------------

Added
+++++

 - Add ``job.reset_statepoint()`` and `job.update_statepoint()`
 - Add ``job.remove()`` function

Changed
+++++++

 - Sanitize filter argument in all ``project.find_*()`` methods.
 - The ``job.statepoint()`` function accurately represents saved statepoints, e.g. tuples are represented as lists, as there is no difference between tuples and lists in JSON.
 - signac-gui does not block on database operations.
 - signac-gui allows reload of databases and collections of connected hosts.

Fixed
+++++

 - ``RegexFileCrawler`` ``define()`` class function only acts upon the actual specialization and not globally on all ``RegexFileCrawler`` classes.
 - signac-gui does not crash when replica sets are configured.

[0.2.5] -- 2016-02-10
---------------------

Added
+++++

 - Added ``signac.get_project()``, ``signac.get_database()``, ``signac.fetch()`` and ``signac.fetch_one()`` to top-level namespace.
 - Added basic shell commands, see ``$ signac --help``.
 - Allow opening of jobs by id: ``project.open_job(id='abc123...')``.
 - Mirror data while crawling.
 - Use extra sources for ``fetch()`` and ``fetch_one()``.
 - Add file system handler: ``LocalFS``, handler for local file system.
 - Add file system handler: ``GridFS``, handler for MongoDB GridFS file system.
 - Crawler tags, to control which crawlers are used for a specific index.
 - Allow explicit job workspace creation with ``job.init()``.
 - Forwarding of pymongo host configuration via **signac** configuration.

Changed
+++++++

 - Major reorganization of the documentation, split into: Overview, Guide, Quick Reference and API.
 - Documentation: Add notes for system administrators about advanced indexing.
 - Warn about outdated pymongo versions.
 - Set *zip_safe* flag to true in *setup.py*.
 - Remove dependency on six module, by adding it to the common subpackage.

Deprecated
++++++++++

Fixed
+++++

 - Fixed hard import of pymongo bug (issue #24).
 - Crawler issues with malformed documents.

[0.2.4] -- 2016-01-11
---------------------

Added
+++++

 - Implement ``Project.repair()`` function for projects with corrupted workspaces.
 - Allow environment variables in workspace path definition.
 - Check and fix config permission errors.

Changed
+++++++

 - Increase robustness of job manifest file creation.

Fixed
+++++

 - Fix project crawler deep directory issue (hotfix).

[0.2.3] -- 2015-12-09
---------------------

Fixed
+++++

 - Fix a few bugs related to project views.

[0.2.2] -- 2015-11-30
---------------------

Fixed
+++++

 - Fix ``SignacProjectCrawler`` ``super()`` bug.

[0.2.1] -- 2015-11-29
---------------------

Added
+++++

 - Add support for Python 2.7
 - Add signac-gui (early alpha)
 - Allow specification of relative and default workspace paths
 - Add the ability to create project views
 - Add ``Project.find_*()`` functions to search the workspace
 - Add function to write and read state point hash tables

[0.2.0] -- 2015-11-05
---------------------

 - Major consolidation of the package.
 - Remove all hard dependencies, but six.