Skip to content
This repository has been archived by the owner on Jul 30, 2024. It is now read-only.
/ NuGet.Jobs Public archive

Merge NuGet.Services.Metadata into NuGet.Jobs #908

Merged
merged 1,695 commits into from
Aug 11, 2020
Merged

Merge NuGet.Services.Metadata into NuGet.Jobs #908

merged 1,695 commits into from
Aug 11, 2020

Conversation

joelverhagen
Copy link
Member

@joelverhagen joelverhagen commented Jul 31, 2020

Address https://github.com/NuGet/Engineering/issues/2669.

The approach taken was to add NuGet.Services.Metadata as a remote on my local NuGet.Jobs clone, then merge NuGet.Services.Metadata's dev branch into NuGet.Jobs dev branch, allowing unrelated histories.

The merge conflicts were essentially places where both repositories had files with the same name. I logically merged the files that have the same name and merged the NuGet.Services.Metadata projects into NuGet.Jobs.sln.

All commits before merge commit 2fe83de are the originals from NuGet.Services.Metadata. Everything afterwards is reviewable on a commit-to-commit basis.

I plan on NOT squashing so we can retain history on a per-file basis.

image

Follow-up tasks:

  1. Test build and deployment:
    • Azure Search Service + Azure Search functional tests
    • Catalog2AzureSearch
  2. Update Azure Search Service release build to pull from NuGet.Jobs (this is a change to NuGetDeployment)
  3. Update release pipelines for NuGet.Services.Metadata jobs to pull artifacts from NuGet.Jobs builds
  4. Delete NuGet.Services.Metadata CI and release build
  5. Disable CI gate on NuGet.Services.Metadata
  6. Make a commit to NuGet.Services.Metadata deleting all content and giving a "moved" notice.
  7. Archive NuGet.Services.Metadata

joelverhagen and others added 30 commits June 17, 2019 11:51
* Initial implementation.
Added command line arguments to catalog2lucene to find flat container.
Added code that generates proper icon url.
Added tests.
* test fix
* Spelling, minor refactoring.
Merge branch 'dev' into master

Progress on NuGet/NuGetGallery#7293
* Hotfix binding redirects for search services (#536)

* #2532 Db2Catalog - alternate package version hotfix (#572)
[ReleasePrep][2019.06.26]FI of master into dev
[ReleasePrep][2019.06.26]RI of dev into master
Fix bug when UTC timestamp becomes local time
Progress on NuGet/NuGetGallery#7226
Merge branch 'dev' into master
AuxiliaryFileClient is currently only used by search service. Now it will be used by Auxiliary2AzureSearch as well.
Progress on NuGet/NuGetGallery#6447
This document will be used by Auxiliary2AzureSearh to update just the download count fields in the search index.
Progress on NuGet/NuGetGallery#6447
…uilder (#585)

This logic is already implemented for Owners2AzureSearch: get a package ID version list, find all of the applicable search filters, create a small document per search filter, yield the index actions. Any job that is just updated the search index can use this logic.
Progress on NuGet/NuGetGallery#6447
loic-sharma and others added 17 commits June 26, 2020 15:55
This adds functional tests for the new custom analyzer for packages' tags.
A 412 can happen if the downloads.v1.json gets written while the blob is being read
Also retry on an timeout from reading an HTTP-based cursor
Address NuGet/NuGetGallery#8076
* Add sort by downloads WIP

* Add sort by download in response builder

Use the auxiliary file to sort by downloads before returning a response

* Add Asc and Desc sorting for downloads

* Add functional tests for AdvancedSearch sorting

* Add unit tests for AdvancedSearch sorting

* Fix sorting variable in AdvancedSearch

* Add filtering parameter to AdvancedSearch

* Update comment on SearchReponseBuilder

* Add thenBy newest

* Fix tests for sorting order

* Simplified test cases for SearchResponseBuilder

* Fix tests

* Not using Random anymore for tests

* reverting packageType change

* Fix test styling

* Remove unused import

* Add packagetype parameter to V2 endpoint

* Add exceptions if invalid advanced search on Hijack index

* Add SearchController tests for packageType filter

* Add SearchParametersBuilder tests for packageType filter

* Add IndexOperationBuilder tests for packageType filter

* Add functional tests for package type filter

* V2 search now throws error 400 when using A-S on Hijack

* Update test case to catch the correct exception type
[ReleasePrep][2020.06.29]RI of dev into master
Add vulnerability info to db2catalog
[Search] Merge the A/B test for exactly matching on tags
[Advanced Search] Add sorting & filtering to the search service (#794)
[ReleasePrep][2020.07.08]RI of dev into master
[ReleasePrep][2020.07.14]FI master into dev
* Update SearchService README

* Update src/NuGet.Services.SearchService/README.md

Co-authored-by: Loïc Sharma <[email protected]>

* Update src/NuGet.Services.SearchService/README.md

Co-authored-by: Loïc Sharma <[email protected]>

Co-authored-by: Loïc Sharma <[email protected]>
# Conflicts:
#	.gitignore
#	.nuget/packages.config
#	LICENSE.txt
#	README.md
#	build.ps1
#	buildandtest.ps1
#	sign.thirdparty.props
#	test.ps1
@loic-sharma
Copy link
Contributor

loic-sharma commented Aug 1, 2020

It looks like we also need the following follow-up tasks:

Do you think it'd be worthwhile to deploy one job/service from each release build we migrate?

@joelverhagen joelverhagen changed the title [WIP] Merge NuGet.Services.Metadata into NuGet.Jobs Merge NuGet.Services.Metadata into NuGet.Jobs Aug 10, 2020
@joelverhagen joelverhagen merged commit 693f255 into dev Aug 11, 2020
@joelverhagen joelverhagen deleted the jver-merge branch August 11, 2020 00:37
joelverhagen added a commit that referenced this pull request Oct 26, 2020
NuGet.Services.Metadata history is left intact and merged into the NuGet.Jobs history.
Address NuGet/Engineering#2669
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.