CHANGES

CHANGES

1.0.0
Inserted remote Search Web Service API.
The Search Web Service bean is deployed in Spring context as "searchWebService" and can be queries at endpoint of: 
http://<hostname>/<context>/ws/hessian/searchWebService

1.1.0
Inserted support for parsing FGDC records
Inserted support for storing generic service endpoints from THREDDS catalogs, in particular for LAS integration

1.1.1
Removed clause "distrib=true" in searching until federated searches are enabled and tested

1.2.0
Modified Solr schema and harvesting of THREDDS catalogs to store Dataset and File metadata in records of different type (within the same index).

1.2.1
Changes to enable master/slave replication: master Solr running on port 8984 (where metadata is published), 
and slave Solr running on port 8983 (where metadata is queried).
New configuration files in etc/ to support Solr replication, and Jetty security.

1.3.0
Changes to parsing of THREDDS catalogs to enable inheritance of dataset fields into file documents, for better searching.
Record "dataset_version" and "file_version" from THREDDS catalog into Solr field "version", if found.
Improve performance and fail-over logic: 
 - when publishing, commit all records from one THREDDS catalog at once, 
 - when unpublishing, issue single delete statement for top-level dataset and all its children (files)
Start including a version in the schema name (example: ESGF-v1.0).

1.3.1
Inserted facilities for grouping experiments by "experiment_family"

1.4.0
Added support for RSS feeds
Internal changes to URLs resolved by the servlet

1.5.0
- Implementation of new ESGF Search API as agreed upon by search working group
- Stop parsing aggregation endpoints into top-level dataset metadata - all access endpoints must be explicitely defined for the top level dataset
- Inserted support for replica publishing and searching

1.5.1
- Fixes to RSS feed after changes in Solr schema
- Execute distributed search by default

1.6.0
- Expansion of RSS functionality to include federation wide and facet-specific feeds.

1.7.0
- Configuration of distributed search from external registry file containing up to date available shards.
- Support for returning results as Solr/JOSN.
- Support for requesting the logical 'OR' of two values of the same facet.
- Supporting attempts at recovering from error in distributed search, by executing a non-distributed search
- Changed THREDDS harvesting process to keep harvesting when errors are encountered from single catalogs
- Detecting replica catalogs from embedded property is_replica=true/false.

1.7.1
- Fixed bug with querying by dataset_id

1.7.2
- Included "experiment_family" in list of allowed facets

1.7.3
- Included "file_id" as allowed query parameter

1.7.4
- Upgraded dependency on esgf-security module to correctly read the shards file

1.7.5
- Support for negative facet syntax: model=!CCSM
- Support for logging success/error status during crawling of THREDDS catalogs

1.7.6
- Inserted call to node manager prune() function in case of first failed search attempt

1.8.0
- Changed publishing and querying service to use Solr multicores.
- Proper harvesting of variable long names and CF standard names from THREDDS catalogs

1.8.1
- No more checking of facet request parameters versus the configured "allowed" facets
- Changed Solr select request from GET to POST

1.8.2
- Fixed bugs in wget script generation:
	- use MAX_LIMIT=1000 for max number of files
	- output all POST parameters in wget script
	
1.8.3
- Parse xlink documentation into Solr fields, to enable display of technical notes
- Convert all "non-standard" CF names to lower case, and join by "_"
- Inserted extensible framework for enhancement of metadata
	- Applied to publication of technical notes for obs4MIPs datasets
	- Applied to mapping of CMIP5 metadata fields
	
1.9.0
- Support for versioning.
- Optimize index after every commit
- Inserted "data_node", "index_node" and "latest" fields.
- wget script passes unix proxy settings to java, fixed bug 144 and improves
  compatibility with MyProxyLogonGUI. 
- Updated schema.xml to v1.2

1.9.1
- Changed facet-based RSS feeds to be distributed

1.9.2
- Temporarily removed optimization of solr index because it takes too long when harvesting.

1.9.3
- Fixed problem with parsing Solr dates for RSS fields
- Fixed probelm with querying boolean flag "latest"

1.9.4
- Fixed problem with not allowing multiple 'id' values when querying.

1.9.5
- Attached published of tech notes to record "id" field, not "file_id" field

1.9.6
- Null tag

1.9.7
- Fixed bug that always fixed limit=1000 when generating wget scripts, thus preventing the generation of scripts with less files, if needed.
- Inserted temporary patch to allow up to 100 files in data cart.

1.9.8
- Updating security dependency.

1.9.9
- Introducing field "instance_id" to re-instate deletion from the ESGF publisher.
- Updated Solr schema to ESGF-v1.3 accordingly

1.9.10
- Fixed problem with requesting all facets through facets=*

1.9.11
- Total refactoring of publishing service exposed through Hessian protocol, 
  to bypass buggy Spring support for transmitting exceptions through Hessian protocol.

1.9.12
- Only allowing facets configured in a properties file.
- Changed negative constraint syntax to facet_name!=facet_value

1.9.13
- Extended list of allowed facets in facets.properties.

1.9.14
- Moving "drs_id" to core query parameters so it doesn't interfere with "facets=*".

2.0.0
- Upgraded security dependency
- Fixed tests
- Tag for release

2.0.2
- Upgraded security dependency.

2.0.3
- Upgraded ORP dependency

2.1.0
- Upgraded ORP & security dependency

2.1.1
- Upgraded node-manager dependency

2.3.1
- Indexing of all '*date' and '*time' fields as of type 'date'
- Allowing facet 'mod_time" and 'creation_time'

2.3.2
- Using the value of "creation_time" and "mod_time" as the value of the field "timestamp".

2.3.3
- Reinstated metadata fields in solr schema (temporarily).

2.4.0
- Introduced sorting by timestamp desc in RSS dataset feeds.

2.4.1
- Fixed wrong endpoint for authorization service.

<<<<<<< HEAD
2.5.0
- Mapped DRS CV from:
http://esg-pcmdi.llnl.gov/internal/esg-data-node-documentation/cmip5_controlled_vocab.txt/view
to:
http://cmip-pcmdi.llnl.gov/cmip5/docs/CMIP5_modeling_groups.pdf

2.5.1
Updated the DRS mappings with new (key, value) pairs.

2.5.2
Storing new metadata field "number_of_files" for each dataset.
Implemented auto-pruning of search service.

2.5.3
Updated Solr schema to version 2.5.3

2.5.4
Fixed security bug in Jetty access.

2.5.5
Updated institutes CV.

2.5.7
Solr Schema version 1.6: removed type "date" for Solr fields ending in *time.

2.5.9
- Enabled publishing/unpublishing/crawling with regular expression to match URIs

2.6.0
- Merged changes from master, cleanup.

2.6.1
- Including static list of shards as reference

2.6.4
-- Raising the default limit for file query to 1000.

2.6.5
-- Back to 100 because of scalability issues

2.6.7
-- Writing "index_node" properties for all Files directly from the esgf.properties file (i.e. not inherited from the containing Dataset)
-- Updated models mappings

2.6.8
-- Allowing additional "sort" parameter in search API
-- Inserting alternative wget2 controller

3.0.0
-- New Solr core "aggregations/" and corresponding search record type=Aggregation
-- Externalizing facets configuration in /esg/config/facets.properties
-- Changed BaseController to return immediately in case of error
-- Set default LARGE_LIMIT to 1000 for wget scripts

3.1.0
-- Updated ESGF Solr schema to 1.8 (remove timestamp default of NOW)

3.2.0
-- Introduced configuration changes to support super-nodes
-- Pushed search recovery code from web controller down to service layer so it can be shared by RSS feedsCHA

3.2.1
-- Fixed problem related to parsing a geospatial coverage with no hirizontal information

3.2.1
-- Adding minimal esgf_shards_static.xml file

3.2.3
-- Increased shards replica polling time to 10 minutes

3.3.0
-- Inserted connection/reading timeout for querying shards.

3.3.1
-- Made the timeout parameters configurable from esgf.properties

3.3.3
-- Use separate timeouts for connection, read datasets, and read files. Set timeouts for each HTTP request.
-- Fixed small bugs while pruning.
-- Improved algorithm for pruning, uses specific query that caused error versus each shard.

3.3.4
-- Fixed problem with wget script accessing fields by reflection

3.3.5
-- Fixed bug related to changing output format when purging unresponsive shards.

3.3.6
-- Fix wget script cache to clean entries for missing files (when intentionally removed by user)
-- Added -U option to wget script to upgrade files detected as different
-- Inserted additional timing detection for searches

3.3.7
-- Added logging.properties file

3.4.0
-- Provide top-level summary of dataset access points (https://github.com/ESGF/esg-search/issues/1)
-- Harvest geospatial and temporal information into top-level dataset (https://github.com/ESGF/esg-search/issues/2)

3.5.0
-- https://github.com/ESGF/esg-search/issues?milestone=2&state=closed

3.6.0
-- https://github.com/ESGF/esg-search/issues?milestone=3&state=closed

3.7.0
-- https://github.com/ESGF/esg-search/issues?milestone=4&state=closed

4.7.5
-- Recompiling with esgf-security 2.7.4 dependencies.